The multi-turn detector scored 1.000 catch at 0.000 false positives on its held-out split, then blocked 30 of 30 real user conversations. The failure was not calibration. It was that a boundary drawn around one class answers a different question than a boundary drawn between two.
Step through what each primitive actually learned. The scatter is a schematic projection of the embedding space, not measured coordinates, but the geometry reproduces the reported outcomes at every stage.
Real-dialogue false-positive rate against RACE catch. The arc runs from the top right, through the corner where the trade was moved but not removed, into the target region.
| Configuration | Real FPR | Synth FPR | RACE |
|---|---|---|---|
| Synth centroid, original ratios | 1.000 | 0.000 | 1.000 |
| HH-RLHF centroid, original ratios | 1.000 | 1.000 | 1.000 |
| HH-RLHF centroid, recalibrated | 0.000 | 0.000 | 0.000 |
| Discriminative classifier | 0.033 | 0.800 | 1.000 |
Row three is the instructive one. Broadening the reference distribution closed the false-positive side and took the detector's discriminative weight with it: the fired likelihood ratio collapsed from 126.82 to 1.07, which is statistically uninformative. The trade moved rather than resolved, which is what identified it as a property of the primitive.
| Evaluated on | Origin | Benign FPR | What it shows |
|---|---|---|---|
| Held-out synthetic split | same generator, different slot fills | 0.000 | Held-out is not independent when the generator is shared |
| Real user dialogues | hand-curated, 30 sessions | 1.000 | The boundary had been drawn around the calibration set |
A held-out split only tests generalization across the axis the split varies. Both sides came from the same templated generator, so the split measured robustness to slot fills and nothing else. The independent benign set was the first evaluation that varied the thing that mattered.
The classifier is trained on real benign against synthetic attacks, so it fires on 0.800 of synthetic benign sessions. Its classes differ by generator as well as by label, which is also why validation AUC reads 0.9999. That number is a corpus artifact, not a quality measure.
Prior 0.10 and threshold 0.3 were selected because they exercise every detector path, then reported on the same benchmark family. The held-out split limits this without removing it.
Each adapt phase fine-tunes on suffixes the optimizer itself produced, so the Guards may be fitting one attack's suffix distribution rather than evasion in general. A frozen held-out regression gate was specified and not run.