The problem
A classifier can look strong at a default threshold while producing unacceptable false positives, failing on shifted traffic, or hiding which inputs drive individual decisions.
Machine learning · MLOps
Sentinel ModelOps
A fitted intrusion classifier, not a hand-written score. Compare the champion against its baseline, tune the operating threshold, introduce drift, and explain an individual prediction.
- 1,400 train · 600 test rows
- Champion vs baseline
- Drift & explanations
POST /api/monitor
Personal project on synthetic data · Not an employer system
Architecture
- Logistic regression
- Champion vs baseline
- PSI drift
- Local explanations
- Fixed-seed data1,400 training and 600 test rows, UNSW-NB15-inspired features
- Two fitted modelsL2 logistic: 2-feature baseline vs 6-feature champion
- Operating thresholdPrecision, recall, F1, and false-positive rate at a chosen cut-off
- Drift monitorPSI between reference and shifted score distributions
- ExplanationsPer-example logit contributions from the fitted weights
Run it
- Raise population drift
- Watch PSI and precision react
- Move the threshold to trade recall for false alarms
What this showsModel training · Baseline comparison · Threshold tuning · Production monitoring
- Precision
- 82.1%champion
- Recall
- 87.6%champion
- F1
- 0.848baseline 0.498
- False-positive rate
- 10.3%40 of 390 normal rows
- Score drift (PSI)
- 0.196WATCH
Score distribution
Champion scores on reference vs. shifted traffic
threshold
Champion vs. baseline
Same 600-row test population and threshold
| Model | Precision | Recall | F1 | FPR |
|---|---|---|---|---|
| 6-feature logistic champion | 82.1% | 87.6% | 0.848 | 10.3% |
| 2-feature baseline | 72.1% | 38.1% | 0.498 | 7.9% |
Predicted attackPredicted normal
Actual attack184true positive26missed
Actual normal40false alarm350true negative
Prediction explanation
Test sample #12 · actual label: Normal
49.2%champion score
Allow- failed handshakes+1.544
- TTL delta+1.081
- destination fan-out+1.045
- source bytes+0.752
- connection rate+0.688
- service entropy+0.385
Intercept -5.524 + contributions = logit -0.029
Logit contribution = fitted weight × feature value for this connection.
Fitted champion
L2-regularized logistic regression · 260 epochs · seed 4815
| Feature | Weight |
|---|---|
| connection rate | 1.010 |
| source bytes | 1.180 |
| TTL delta | 3.581 |
| service entropy | 0.603 |
| failed handshakes | 4.052 |
| destination fan-out | 1.990 |
| bias | -5.524 |
Synthetic data. 1,400 training and 600 test rows generated with a fixed seed, inspired by the UNSW-NB15 feature domain; not official benchmark results.
Precision 82.1%, recall 87.6%, PSI 0.196, watch.
Decisions, evidence, and limits
How it works
Fit two L2-regularized logistic models on a reproducible public-safe benchmark inspired by the UNSW-NB15 feature domain. Compare a two-feature baseline with a six-feature champion, then rescore a fixed test population under controlled covariate shift.
Baseline
The baseline is trained with two traffic features; the champion uses all six. Both are evaluated on the same 600 labeled samples, with the same threshold and drift controls, so the comparison is reproducible.
What is measured
Precision, recall, F1, false-positive rate, confusion counts, PSI, score distributions, fitted weights, and per-example logit contributions are calculated by the live endpoint.
Failure modes
Synthetic features simplify real network behavior, PSI is sensitive to binning, and covariate drift does not necessarily imply quality loss. Linear contributions explain this model's output; they are not causal effects.
Production tradeoffs
Train on licensed source data with a temporal split, add calibration and attack-family slice metrics, log artifacts and data versions in MLflow, monitor delayed labels, and define champion-challenger rollback criteria.