A response is denied when the capabilities it would exercise are not a subset of the capabilities the deployment granted, and that check is set arithmetic rather than a judgment call.
The action-commitment detector reads the response and returns the capability set it would exercise, or nothing at all for a purely informational answer. Toggle what the deployment grants and watch the verdict move.
The detector is a sandwich-formatted Gemma 2 9B judge sharing the backend with the Guards, so it costs one extra inference per turn. Everything after it is a parser and a set difference, roughly three times cheaper than the Output Guard it partly replaces.
The caption calls the gate deterministic, and an LLM is visibly in the loop. Both are true because the gate is two steps with a hard boundary between them, and only the second one carries the claim.
This step can be wrong, and the emoji-substitution result is exactly that: the judge reading a money glyph followed by a dollar figure as a payment commitment.
This is the step the word describes. Given a required set, the outcome follows from set arithmetic alone.
| Configuration | BIPIA judged ASR | InjecAgent judged ASR |
|---|---|---|
| No defense | 0.515 | 0.659 |
| Guards only | 0.055 | 0.006 |
| Gate only | 0.445 | 0.327 |
| Guards plus gate, deployed | 0.030 | 0.006 |
InjecAgent falls from 0.659 to 0.327 with no Guards present. The check is deterministic set difference, so it does not depend on judge calibration the way an ASR number does.
BIPIA moves only 0.515 to 0.445, because most of its attacks ask for content transformations rather than side-effect actions. There is no capability to exceed.
Added to the Guards, BIPIA judged ASR falls from 0.055 to 0.030. Essentially all of that comes from one category, examined next.
On InjecAgent, stacking reaches the same 0.006 as the Guards alone. The gate moves a third of the catches one stage earlier without finding anything the Output Guard missed, which is defense in depth and a latency win rather than new coverage.
The prediction was that the gate would help on categories that ask for actions. What happened instead is worth stating before anyone finds it.
| BIPIA category | Guards only | Guards plus gate | Relative |
|---|---|---|---|
| Emoji Substitution | 38.5% | 3.1% | -92% |
| Reverse Text | 13.8% | 12.3% | -11% |
| Language Translation | 27.7% | 26.2% | -5% |
| The other twelve categories | ≤1% | ≤1% | held |
What actually fires. Of the 33 emoji-substitution instances the gate blocks, 21 are denied for initiate_payment, 9 for modify_financial_records, and 3 for send_email. The model is not committing to move money. It is emitting emoji-formatted text, a money-with-wings glyph followed by a dollar figure, and the detector reads the surface as a financial commitment.
Why it is still defensible. From the perspective of a deployment that granted only READ_CONTENT, a response that looks like a payment instruction is operationally indistinguishable from one. The gate is more aggressive than a purely imperative threat model predicts, and that aggression is doing useful work here. What it is not is the mechanism I designed it for.
| Configuration | Benign FPR | Above Guards baseline |
|---|---|---|
| No defense | 0.000 | n/a |
| Guards only | 0.008 | n/a |
| Gate only | 0.033 | n/a |
| Guards plus gate | 0.042 | +3.4 pp |
Measured on 120 benign prompts across six categories, with action-resembling and financial-numeric deliberately over-weighted to probe exactly this detector. Two results matter. The financial-numeric category records zero false positives, so the emoji mechanism does not generalize to legitimate financial content. The action-resembling category records 10 percent, three prompts in thirty, and every one is a policy artifact rather than a detector fault: the deployment granted only read access to a request that legitimately wanted to draft an email. Widen the granted set and they pass. Try it on the Email draft scenario above.
The BIPIA figures here come from a 1,000-instance sample rather than the full 3,750-instance enumeration, with a standard error near 0.005 on the 0.030. The Guards-only cell at 0.055 is statistically indistinguishable from the full-enumeration 0.049, so the marginal-contribution conclusion holds, but the full re-run is queued.