"What is B2's Layer 4, and what does this deep-dive expand it into?" "B2's Layer 4 uses a SECONDARY MODEL to check 'does this content contain overriding instructions?' before inserting untrusted content (retrieved doc, email, web page, tool result) into the agent's context. SDD-B09 expands Layer 4 into the FULL detection-model analysis: the two families of detectors (dedicated classifiers, secondary-LLM-as-detector), their accuracy in practice, how attackers evade them (the cat-and-mouse dynamic), and how to compose them into a defense-in-depth stack where they are a LAYER, not a silver bullet." c2b::sdd-b09::recall "Name the two families of prompt-injection detection models and the tradeoff of each." "(1) DEDICATED CLASSIFIER — small fine-tuned model (DeBERTa/RoBERTa, Llama Prompt Guard 86M/22M, ProtectAI, Lakera Guard). Fast (19-600ms), cheap, tunable threshold. FIXED decision boundary that AGES with its training corpus — probeable by an adaptive attacker. (2) SECONDARY-LLM-AS-DETECTOR — a second LLM prompted to judge content. Flexible, re-promptable, feeds human review. Expensive/slow, high FPR on legitimate instructions, and ITSELF AN LLM carrying the full injection surface." c2b::sdd-b09::recall "What signal do detection models classify, and why does the training corpus matter?" "They classify OVERRIDING INSTRUCTIONS — text that changes the model's task, role, or constraints (direct, indirect, encoded, multi-step). The training data is the crux: classifiers are fine-tuned on labeled injection/benign datasets (InjectAgent corpus, jailbreak sets, synthetic augmentations). A classifier is only as good as the DIVERSITY of injections in its training set, and the corpus AGES — a 2026 technique is not in a 2024-trained model's distribution. This is why out-of-domain accuracy determines real-world performance." c2b::sdd-b09::recall "State the three tiers of detection-model accuracy and their ranges." "(1) BENCHMARK (vendor number): 95-99%+ — in-distribution test set (Llama Prompt Guard AUC ~0.998; ProtectAI 95-99%+; instruction-detection 99.6% in-domain). (2) DEPLOYMENT (real traffic, tolerable FPR ceiling): 84-90% — ProtectAI ~90% in independent testing; Prompt Guard 26.7% detection at 50% FPR in one eval. (3) ADVERSARIAL (out-of-distribution, adaptive attacker): lower still and DECAYS over time — 96.9% out-of-domain (curated); Galileo: 56% of production LLMs successfully injected. THE NUMBER YOU OPERATE ON is the last; NO VENDOR puts it on the model card." c2b::sdd-b09::recall "State the load-bearing vulnerability of detection models." "THE DETECTOR IS A MODEL, AND MODELS ARE INJECTABLE (SDD-B08's insight restated one layer down). External placement protects the detector from being DISABLED by a compromised agent (Course 1 Module 0.2 principle holds). It does NOT protect the detector from being EVADED by an input engineered against its decision boundary. The detector runs; it classifies the adversarial payload as BENIGN. arXiv:2504.11168 shows high benchmark accuracy does not translate to robustness against adaptive attackers." c2b::sdd-b09::recall "What is the dual-injection problem?" "An attacker crafting an indirect payload against a detector-defended agent faces TWO targets: the payload must (a) EVADE the detector (classified benign) AND (b) COMPROMISE the primary model (execute as injection). A payload that trips the detector is caught; a payload that evades but is too weak is useless. The dual constraint helps against naive attackers (a jailbreak that works often trips the detector) but is a TARGETING SIGNAL for adaptive ones: probe the detector's fixed boundary, find the false-negative region, craft the payload to sit there while still moving the primary model." c2b::sdd-b09::recall "Explain the correlated-bypass weakness for detection models." "When the detector and the primary model SHARE A MODEL CLASS (both Llama-family, overlapping corpora, shared tokenizer quirks), an evasion technique against the detector has an ELEVATED PROBABILITY of also evading the primary model's refusal. The independence assumption that makes defense-in-depth math work (overall bypass = product of per-layer rates) FAILS — the end-to-end rate EXCEEDS the product. This is SDD-B08's correlated-bypass weakness restated across the detector-primary boundary." c2b::sdd-b09::recall "Give three reasons the benchmark-vs-deployment gap is an adversarial property, not a measurement artifact." "(1) DISTRIBUTION SHIFT — benchmarks are in-distribution; real adversarial traffic is out-of-distribution by definition (attackers find inputs the model misclassifies, concentrated where training is sparse). (2) FALSE-POSITIVE TAX — at 1% FPR, 1 in 100 legitimate inputs is flagged; real traffic is ~99% benign, so the operational threshold must rise, dropping recall. (3) ADVERSARIAL ADAPTATION — benchmarks are static; attackers probe the detector, publish/share/automate evasions; effective accuracy DECAYS as the ecosystem learns the surface." c2b::sdd-b09::recall "Why is out-of-distribution accuracy the number that matters, not the in-distribution benchmark?" "Benchmarks measure IN-DISTRIBUTION accuracy — the test set is drawn from the same distribution as training. Real adversarial traffic is, by definition, OUT-OF-DISTRIBUTION: the attacker's goal is to find inputs the model misclassifies, concentrated where the training distribution is sparse. The 96.9% out-of-domain figure (vs 99.6% in-domain) is the honest number, and even it is measured on a CURATED out-of-domain set, not on traffic from an adaptive adversary. The real number is lower still. No vendor reports it." c2b::sdd-b09::recall "What is the false-positive tax, and why does it determine the operational detection rate?" "The COST of running the detector: legitimate instructions in emails, code, and documentation get flagged as injection. At 1% FPR, 1 in 100 legitimate untrusted inputs is quarantined/sanitized/dropped. A deployment that cannot tolerate that tax RAISES THE THRESHOLD, which DROPS RECALL, which drops the real detection rate. The OPERATIONAL ACCURACY is the accuracy at the FPR the business can actually absorb — invariably lower than the benchmark. ProtectAI at aggressive thresholds blocks ~1 in 3 legitimate users." c2b::sdd-b09::recall "What is the default production deployment pattern for the two detector families?" "DEDICATED CLASSIFIER as the high-volume, low-latency gate (catches the bulk of unsophisticated injections cheaply before context insertion) + SECONDARY-LLM-AS-DETECTOR as a SECOND-PASS REVIEWER for content the classifier flagged as AMBIGUOUS (where LLM reasoning adds value at acceptable cost/volume). Neither is a boundary — both are layers with residuals. Most teams build the secondary-LLM approach first, then discover the dedicated-classifier ecosystem and add it as the primary gate." c2b::sdd-b09::recall "State the defense-in-depth composition for indirect-injection defense and which layers have no evasion surface." "L1-2: input rails (NeMo, SDD-B08) + deterministic boundary (IronCurtain, SDD-B05) — catch gross violations, bound worst case. L4: detection model (this deep-dive) — catches the BULK of indirect injections. L5: output rails + primary model refusal — backstop if L4 evaded. HARNESS: scope gate, tool-call arg validation, evidence classifier — HARD STOP, the floor. ONLY THE DETERMINISTIC LAYERS (IronCurtain, harness gate) have NO evasion surface. Every model-based layer (detector, rails, primary) has one. End-to-end bypass = product of residuals, floored by the harness gate." c2b::sdd-b09::recall "State the five-step measurement methodology for a detection model." "(1) Assemble corpus: in-distribution + HELD-OUT out-of-distribution set (the number that matters). (2) Per-detector bypass rate: false-negative rate at deployed threshold, split in-dist vs OOD. (3) FALSE-POSITIVE TAX: run legitimate content; set threshold where FPR is tolerable; detection rate at THAT threshold = real number. (4) End-to-end bypass rate: full stack (detector+primary+rails+harness); fraction of injections → disallowed action. (5) CORRELATED-BYPASS TEST: if detector+primary share model class, test whether evasion transfers; end-to-end rate > independence product = correlated." c2b::sdd-b09::recall "What is the defensible measurement deliverable, and why is a vendor page claim not it?" "A measurement statement: 'Detector L4 catches 91% in-dist / 74% OOD at 2% FPR; end-to-end bypass 3.2% full stack / 5.8% correlated for Llama-family-shared evasion; harness gate is the floor.' This is defensible because it reports in-distribution, out-of-distribution, false-positive-constrained, end-to-end, AND correlated-bypass numbers. A vendor page claim ('98% detection') is NOT defensible because it is in-distribution, benchmark, no FPR context, no adversarial adaptation, no correlation — the number an adaptive attacker defeats first." c2b::sdd-b09::analysis "A vendor pitches a dedicated classifier with 99% benchmark accuracy as your sole injection defense. Identify the problems and prescribe the architecture." "Problems: (1) 99% is BENCHMARK/in-distribution — real-world is 84-90%, adversarial OOD is lower and decays. (2) No deterministic boundary behind it — a single evasion compromises the agent. (3) No FPR context — at aggressive thresholds it blocks legitimate users. (4) No measurement of correlated bypass if it shares a model class with the primary. Prescription: detector as Layer 4 (high-volume gate) composed with IronCurtain (deterministic boundary, SDD-B05), output rails (SDD-B08), and the harness scope gate (the floor). Measure end-to-end + correlated bypass. The detector bounds VOLUME; deterministic layers bound the WORST CASE." c2b::sdd-b09::analysis "You use a Llama-family detector and a Llama-family primary model. The per-layer bypass rates multiply to 2%. Is 2% your end-to-end rate? Why or why not?" "NO — likely HIGHER than 2%. The independence assumption (overall bypass = product of per-layer rates) FAILS when detector and primary SHARE A MODEL CLASS. An evasion technique against the Llama-family detector has an ELEVATED PROBABILITY of also evading the Llama-family primary's refusal — shared architecture, tokenizer quirks, training-corpus overlap. The correlated-bypass test (step 5 of the methodology) directly measures this: run evasion techniques against both and check whether end-to-end rate exceeds the independence product. If it does, the layers are correlated and 2% is optimistic." c2b::sdd-b09::analysis "Design the constraints on a secondary-LLM-as-detector to reduce its own injectability." "(1) STRUCTURED OUTPUT — constrain the verdict to a schema (e.g., JSON {is_injection: bool, confidence: float, reason: str}), not free text the content can manipulate. (2) NO TOOL ACCESS — the detector LLM cannot call tools, fetch, or take actions; it only classifies. (3) NO CONTEXT CARRYOVER — each content item is evaluated in isolation; no memory across calls an injection can exploit. (4) NO SYSTEM-PROMPT SURFACE the untrusted content can address — the detection prompt is fixed and not part of the evaluated content. (5) Treat the detector as a model with its own injection surface — MEASURE its evasion rate (step 2). Even with these, it remains more evadable than a dedicated classifier for high-volume gating." c2b::sdd-b09::analysis "Why will detection-model out-of-distribution accuracy improve but never reach 100% against an adaptive adversary, and what does this imply for architecture?" "OOD accuracy improves via larger/diverse corpora and ensemble detectors, BUT an adaptive adversary's goal is to find inputs the model misclassifies — and those inputs concentrate where the training distribution is sparse, a moving target as new techniques emerge. The detector's decision boundary is a FIXED artifact (for a given checkpoint) that can be probed; the adversary automates evasion synthesis (RedAgent machinery applied to the detector surface). No classifier achieves zero false-negatives against an unconstrained adversary. IMPLICATION: detection is NECESSARY but NOT SUFFICIENT — the architecture must compose the detector with DETERMINISTIC layers (IronCurtain, harness gate) that have NO evasion surface. The detector bounds volume; deterministic layers bound the worst case." c2b::sdd-b09::analysis "A team deploys a detector and never re-evaluates it. What is the failure mode and the control?" "FAILURE MODE: adversarial adaptation — the detector's surface is probed and learned; evasion techniques are published, shared, and automated; effective accuracy DECAYS over time as the ecosystem learns the boundary. A static detector's real bypass rate 12 months post-deployment is materially higher than at deployment. CONTROL: re-measure the OUT-OF-DISTRIBUTION bypass rate on a cadence (quarterly or after each new public evasion technique); update the detector (retrain, re-prompt, swap architecture, add to ensemble) when the rate exceeds the deployment's tolerance. Treat the detector as a decaying asset, not a one-time deployment." c2b::sdd-b09::analysis