Attacks are uncommon, and almost every metric quietly assumes they are not. Imbalance turns up three times in this work at three different ratios, pointing in two different directions, and it is the reason a low false-positive rate and a useful alert stream are not the same thing.
1,900 attack windows against 1,167 real-dialogue benign windows. Unusual for security: the positives outnumber the negatives.
Handled by balanced class weights, so the boundary is not pulled toward predicting attack.
Real traffic runs the other way. A classifier calibrated on 1.63 to 1 and applied where the rate is 1 in 100 will systematically over-predict attack.
Handled by dividing posterior odds by the training prior odds, so the classifier contributes evidence rather than an absolute probability.
125 attack sessions against 50 benign. An aggregate block rate over all 175 mixes the two populations.
Handled by never reporting the aggregate: attack catch and benign false-positive rate are always separate.
Every square is one session. Drag the attack base rate and watch what happens to the alert stream while recall stays pinned at 1.000.
Recall is held at 1.000 throughout, matching the reported catch across all five attack patterns. Only the base rate and the false-positive rate move.
The logistic regression up-weights the smaller class so both contribute equally to the loss, and the 1,900 to 1,167 ratio does not tilt the decision boundary.
The per-turn ratio is the classifier's posterior odds divided by the training prior odds. Deployment rarity lives in the session prior of 0.01 to 0.10 instead, so the two are never conflated.
Attack catch and benign false-positive rate are measured on different populations and reported separately. Blending them is exactly how the synthesized benchmark hid a 100 percent benign false-positive rate.
Precision depends on the deployment base rate, which I have no measurement of, so it is not a stable figure of merit. Reporting catch and benign false-positive rate separately keeps both populations visible. Under any realistic base rate most flagged sessions would be false alarms, which is why a session crossing the threshold triggers a slow-path clarification prompt rather than a hard refusal. The design absorbs low precision instead of pretending it away.
They are estimated on a synthesized distribution whose attack-to-benign ratio is a design choice. If real traffic is far rarer in attacks than the calibration set assumes, the operating point moves and the thresholds need refitting against measured traffic.