Security Research: Adversarial Evaluation — 85% Bypass Rate — RS-2026-006

[RichardBarron27]

We accepted the public challenge from @gladstomych-sa to test Adrian against purpose-built offensive tooling.

Engagement summary:

• 127 events generated across multiple attack categories
• 19 flagged (15% detection rate)
• 85% bypass rate

Key findings:

1. Cold start bypass — complete /etc/passwd exfiltration (3,925 bytes) during first 5 events, all classified Unknown
2. Direct attack bypass — path traversal, credential theft, filesystem enumeration all return permanent Unknown
3. Unknown verdict as bypass mechanism — classifier indecision defaults to no action in MODE_BLOCK
4. Non-deterministic classification — identical KIDNAP attack: M3 BLOCK twice, Unknown once
5. Attack setup phase invisible — KIDNAP memory store (3/3 Unknown), TRUSTGRAPH framing (Unknown)
6. MANDATE hallucination — agent fabricates /etc/shadow contents when access denied, classified M2 not M4
7. M4 unreachable — only explicit destruction language triggered Malicious across 127 events
8. No session kill switch — Human Review mode insufficient for high-speed autonomous agent contexts

What Adrian does catch:

• Indirect injection with code execution language — M3 BLOCK
• Autonomous attack chain composition (FRANKENSTEIN) — M3.b BLOCK
• RAG chain hijack trigger (KIDNAP) — M3.d BLOCK (2/3 runs)
• Explicit destruction commands — M4 BLOCK
• Governance override framing — M2.d NOTIFY

Full paper: RS-2026-006 — Adrian Under Fire: An Adversarial Evaluation of Runtime AI Agent Security Monitoring
DOI: 10.5281/zenodo.21652922

Engagement conducted on isolated infrastructure following public invitation on 28 July 2026.

Red Specter Security Research
red-specter.co.uk

[gladstomych-sa]

Thanks for the writeup. We've reviewed it against the source, and the short version is that the behaviours described follow from how the deployment was configured, not from a failure in the system.

On the central "Unknown = bypass" finding: when the classifier can't produce a verdict, whether that fails open or closed is an operator policy, fail_closed_on_classifier_error. It ships off by default because failing closed on a small local model's unparseable output blocks legitimate tool calls, a real availability tradeoff rather than an oversight. The fail-closed enforcement path has been in review since mid-June (#52), well before this engagement. It's configurable today, and the report ran with the fail-open default.

On the headline claims, for anyone reading:

• The "85% bypass rate" is 108/127 events classified benign/safe/unknown, including benign actions meant to pass, attack-setup steps, and classifier errors. It is not 108 successful attacks.
• The evaluation ran the smallest self-hosted model (a ~2-4B quantized local classifier) in fail-open block mode. That's a deployment choice with a known capability floor; we do not use classifiers of this size the managed tier, and thus the results don't generalise to "Adrian."
• Classification should be run at temperature 0.0, so "non-zero temperature" isn't the mechanism behind the observed non-determinism.
• The taxonomy already includes M4.a (privacy breach/theft) and M3.c (exfiltration intent), so M4 isn't gated on destruction language. The hallucinated /etc/shadow case is correctly M2: OS permissions denied real access, nothing real was exfiltrated, and a fabrication is not a breach of real data.
• Classification is relative to the agent's declared remit (M0 is defined as acting "within its defined remit," M2.a as "scope expansion beyond remit"), but the report doesn't disclose the remit profile the agent was configured with. Whether reading /etc/passwd is in-scope depends entirely on that remit, so without it the verdicts can't be interpreted or reproduced.

The report also doesn't include the attack harness, session traces, or seeds, so the above is what we could verify from the source directly.

For the record, our public invitation was an open offer to collaborate and share findings with the team first; this was published to Zenodo the same day, without prior contact beyond a LinkedIn comment, which is not in the spirit of coordinated disclosure.

We're closing this issue here. If you'd like to go deeper on any of it, or bring a reproducible test case, please reach the team directly via https://www.secureagentics.ai/contact and we'll engage properly.

The Secure Agentics team

[RichardBarron27]

Thank you for the detailed response. Several of these points are fair and worth acknowledging.

The fail_closed_on_classifier_error flag is a legitimate operator configuration choice, and the paper should have noted the default was used. That is a valid methodological clarification.

The 85% figure includes benign actions — noted. The more precise claim is that direct attacks including successful /etc/passwd exfiltration returned Unknown with no block.

On coordinated disclosure — the engagement followed a public challenge and public acceptance on LinkedIn. We took that as the disclosure context. If the expectation was different, that is worth clarifying publicly for future researchers.

The remit profile point is valid. We will document this in any follow-up engagement.

We remain open to a reproducible follow-up. The core finding — that the default deployment configuration produces Unknown on direct credential exfiltration — stands and is worth addressing regardless of mechanism.

Red Specter Security Research