NLP · Classification
A fitted email-triage classifier, not a keyword filter. Tune the auto-close threshold, inspect token-level contributions, and compare the model with a hand-written rule baseline on a labeled synthetic inbox.
POST /api/triagePersonal project on synthetic data · Not an employer system
What this showsNLP classification · Precision vs. recall trade-offs · Explainable predictions · Workload reduction
Optional. Leave empty to use the selected sample.
0 / 2,000Credential harvest · actual phishing
Unusual activity on your account. Verify your password immediately at http://203.0.113.44/login or we will suspend access.
Intercept -1.273 + contributions = logit 2.251
Champion scores by true class
Same 600-report test set · rules have a fixed operating point
| System | TNR | Recall | Precision | F1 | Workload |
|---|---|---|---|---|---|
| Fitted logistic | 97.5% | 99.0% | 95.0% | 0.970 | 33.5% |
| Keyword and URL rules | 58.2% | 34.7% | 28.3% | 0.312 | 39.5% |
The baseline is a hand-written keyword and URL heuristic, not a trained model.
Champion at this threshold
Synthetic data. 1,200 training and 600 test reports generated with a fixed seed.
True-negative rate 97.5%, recall 99.0%, 2 missed phishing. Send to analyst.
Security mailboxes fill with user-reported messages. Reviewing every item wastes analyst time, but auto-closing the wrong report misses phishing. A keyword filter fires on legitimate IT reminders and misses wire-request impersonation that contains no link.
Generate a public-safe inbox from templates, including overlapping cases (IT password reminders and link-free wire requests). Fit an L2-regularized logistic model on binary token presence plus URL structure features, then operate it at a tunable auto-close threshold. Reports below the threshold close as benign; the rest go to an analyst.
The baseline is a hand-written keyword and URL heuristic with a fixed operating point. Both systems are scored on the same 600 labeled reports so the comparison is reproducible.
True-negative rate (benign auto-closed), phishing recall, analyst workload against reviewing everything, missed-phishing counts, confusion counts, score histograms by class, and per-report logit contributions.
Templates are not a production mailbox. Linear token contributions explain this model’s output and are not causal, and the probabilities are uncalibrated. A threshold that looks efficient here can miss a novel lure that the vocabulary never saw.
Train on reviewed historical reports with a temporal split, add calibration and family-level slice metrics, keep a human review queue for scores near the threshold, and monitor delayed labels.