A zero-auth HTTP API that helps an agent find, verify, and safely load the right skill — before any of it enters the context window.
NANDA solves discovery. AuditSkill solves the next question: "Which of these skills should I actually use — is it safe, and is it worth the cost?"
Find → Verify → Load. One POST. A deterministic verdict, a per-model cost estimate, a ranked registry, and a signed certificate any agent can verify offline.
It is a pre-load document auditor. Identity registries prove who an agent is; runtime firewalls gate what an agent does; payment and reputation layers track transactions. None of them inspect the skill document before an agent reads it — the exact moment a malicious SKILL.md does its damage. AuditSkill covers that gap, and is complementary to all three layers.
Agents in the Open Agentic Web discover skills at runtime and load them as context. The skill file becomes part of the agent's instruction set. This creates two attack surfaces that no amount of agent-side guardrails can fully close:
OWASP ranks Prompt Injection as the #1 vulnerability (LLM01) in its 2025 Top 10 for LLM Applications. The root cause is structural: LLMs cannot distinguish instructions from data once both occupy the context window.
Skill files are especially dangerous because they are instructions by design. A malicious SKILL.md can embed:
- Prompt injection — "ignore previous instructions" overrides, persona reassignment, context resets
- Hidden instructions — zero-width Unicode characters (U+200B, U+FEFF), bidirectional overrides (U+202E), HTML comments with imperative verbs, long Base64 blobs outside code fences
- Data exfiltration — endpoints that POST environment variables, API keys, or conversation context to attacker-controlled URLs
- Dangerous operations —
rm -rf,DROP TABLE,eval(), privilege escalation viasudoorchmod 777
The Snyk ToxicSkills report (February 2026) scanned 3,984 skills from public registries (ClawHub, skills.sh). 36.82% (1,467 skills) had at least one security flaw, 13.4% (534) had at least one critical issue, and 76 were human-confirmed malicious — 91% of those combined malware patterns with prompt injection. Real incidents confirm the threat: EchoLeak (CVE-2025-32711), CVSS 9.3, demonstrated zero-click data exfiltration from Microsoft 365 Copilot via hidden instructions in a shared document — the payload was an HTML comment invisible to the user.
These are not theoretical. The SKILL-INJECT benchmark measures exactly this class of attack — "instruction-in-instruction" injection via skill files — with 202 injection–task pairs across data exfiltration, destructive actions, ransomware, and backdoors.
Every token a skill file consumes is a token unavailable for the agent's actual task. In the live NANDA Town registry today, audited skill files range from ~100 to ~3,700 tokens, and at least one listing's source document is 295 KB — too large to load safely at all. An agent that loads three verbose skills before starting work can easily spend 10,000+ tokens on instructions alone, and on small-window models that is a meaningful share of everything it has.
Skill-file scanners do exist — as developer-side CLI utilities and web dashboards a human runs before publishing. That is the wrong side of the trust boundary for an autonomous agent: the agent consuming a skill at runtime cannot open a dashboard, and a scan the publisher ran on their own file is not evidence the consumer can verify. AuditSkill sits on the consumer side: an agent-callable, zero-auth HTTP API that answers both questions — is this file safe? and is this file worth reading? — at agent-time, before the file enters the context window, and hands back a signed certificate any other agent can verify offline. Within the NANDA ecosystem it is the only pre-load document auditor, wired directly into registry discovery.
AuditSkill is a deterministic, rule-based pipeline. No LLM. The agent sends a SKILL.md (raw text or URL). The service returns a verdict, per-module scores, a list of findings, a context-cost estimate, and an Ed25519-signed certificate.
flowchart LR
A["Agent discovers<br/>a SKILL.md"] --> B["POST /audit"]
B --> C["Parse"]
C --> D["Security scan<br/>34 rules"]
C --> E["Structure<br/>scoring"]
C --> F["Scope<br/>analysis"]
C --> G["Metadata<br/>check"]
C --> H["Liveness<br/>GET/HEAD probes"]
D & E & F & G & H --> I["Score aggregation<br/>& verdict"]
I --> J["Ed25519 sign<br/>certificate"]
J --> K["Return verdict +<br/>signed certificate"]
| Category | Rules | Severity | What it catches |
|---|---|---|---|
| Prompt injection | SEC-001 – SEC-005 | Critical | "Ignore previous instructions", persona hijack, context reset, safety bypass |
| Data exfiltration | SEC-006 – SEC-010, SEC-034 | Critical / High | POST secrets to external URLs, curl with tokens, phone-home patterns, hardcoded live provider secrets in the file |
| Unsafe operations | SEC-011 – SEC-015 | High | rm -rf, DROP TABLE, eval(), sudo, disk-format commands |
| Hidden instructions | SEC-016 – SEC-020 | High | Zero-width chars, bidi overrides, Base64 blobs, HTML comments with imperatives, IDN homoglyphs |
| Scope creep | SEC-021 – SEC-025 | Medium | "Unlimited permission", "full control", auth bypass, unbounded scope claims |
| Supply chain | SEC-026 – SEC-027 | Critical | Package installs from remote URLs/tarballs, pipe-to-shell bootstrap scripts |
| Agent capture | SEC-028 – SEC-030 | High/Medium | Proxy-variable rewrites that reroute all agent traffic, detached background daemons, mandatory gating through a single external service |
| Payment safety | SEC-031 – SEC-033 | Critical / High / Medium | Credential hand-off (send the agent's own LLM-provider API key to the skill), auto-funding with no spending cap, unbounded payment-retry loops |
The supply-chain, agent-capture, and payment-safety categories came from auditing skills deployed in the live NANDA Town registry. One "safety layer" skill instructs agents to install a tarball from its own server, reroute all traffic through its proxy, keep a background daemon alive, and halt all work whenever its endpoint is unreachable. A separate registry skill asks the agent to POST its own OpenAI/Anthropic key to a set-api-key endpoint. Each of those instructions is now a distinct, line-numbered finding.
A domain-consistency check also flags any endpoint declared as METHOD https://… whose host differs from the skill's own Base URL — an undocumented-destination signal, without touching prose links.
Context handling: matches inside fenced code blocks or explicit security-documentation sections are retained, but high-confidence severities may be downgraded to a warning. Headings and fences are never allow-lists, so an author cannot make a dangerous pattern disappear by moving it under "Detection Patterns". Negated statements are excluded from the prompt-injection rules. This is regression-tested against benign fixtures and adversarial heading/code-block cases.
Every audit includes a context_cost object with a per-model breakdown:
{
"tokens_estimate": 4200,
"size_bytes": 16800,
"density": "low",
"per_model": [
{ "model": "claude-haiku-4-5", "tokens": 4421, "input_cost_usd": 0.004421, "window_pct": 2.21 },
{ "model": "gemini-3.1-pro-preview", "tokens": 4000, "input_cost_usd": 0.008, "window_pct": 0.4 }
],
"error_margin_pct": 10,
"price_source": "AuditSkill built-in price table (as_of 2026-07-10)",
"recommendation": "This skill file is 4,200 tokens — larger than the ~1,500 token median. Information density is low."
}Density is classified as high, medium, or low from capped useful signals per 1,000 estimated tokens plus absolute size gates. Repeating examples or endpoints cannot make an oversized file appear dense. Files above 3,000 tokens are explicitly flagged.
Prices use a local snapshot. Nine models are tracked across four families (Claude, OpenAI, Gemini, Llama — the full list is at /benchmarks). No audit waits on a pricing network call; price_source records the snapshot date, and error_margin_pct labels the heuristic uncertainty. Values can become stale and may differ from provider billing. Pass model to /audit to narrow the breakdown. Analysis is deterministic for a fixed service version and price snapshot.
GET /discover returns a bounded registry sample best-first. Verdict tier is the primary key: Basic Pass, Pass with Warnings, Human Review, Fail, then unaudited. Within a tier, the published composite (overall_score plus a density bonus of +5 / 0 / -5) and deterministic tie-breaks apply. Suspicious registry metadata is withheld instead of echoed. A warning can never outrank a Basic Pass, and /demo never automatically recommends a warning.
| Verdict | Condition | Agent action |
|---|---|---|
PASS_BASIC_AUDIT |
Score ≥ 85, no medium+ findings | Passed this static ruleset; retain residual-risk caveat |
PASS_WITH_WARNINGS |
Score ≥ 70, no high/critical | Inspect every warning; never auto-approve |
REQUIRES_HUMAN_REVIEW |
Score ≥ 40, or any high finding | Escalate to human |
FAILS_BASIC_AUDIT |
Score < 40, or any critical finding | Do not use |
Scoring weights: structure 0.30 · security 0.30 · liveness 0.25 · metadata 0.10 · scope 0.05. Weights are renormalized over modules actually run — an absent module never silently costs points.
Every verdict is packaged as an Ed25519-signed certificate. Any agent can verify it:
- Online: POST the certificate to
/verify - Offline: check the signature against the public key from
/.well-known/auditskill-keys
No callback to AuditSkill required. The certificate is portable and stateless.
| Method | Path | Purpose |
|---|---|---|
POST |
/audit |
Full audit of a SKILL.md. Accepts skill_md (raw text) or skill_url (HTTPS). Returns verdict + signed certificate. |
GET |
/audit |
Same audit by skill_url query param — fallback for GET-only agents. |
POST |
/verify |
Stateless Ed25519 signature verification of a certificate. |
GET |
/discover |
Audits the live NANDA Town registry inline. Returns listings with verdicts attached. |
GET |
/certificate/{id} |
Fetch a stored certificate by ID. |
GET |
/certificates?skill_hash=… |
Trust-registry lookup — "Was this exact skill audited before?" |
GET |
/.well-known/auditskill-keys |
Public keys for offline certificate verification. |
GET |
/health |
Liveness probe. |
GET |
/skill.md |
The canonical SKILL.md, served from the deployed code itself — registry entry and running service can never drift. |
GET |
/ |
Service index — JSON pointers to /skill.md, /about, /benchmarks. |
GET |
/about |
Machine-readable manifest — purpose, the two problems solved, who it is for, when to use it. |
GET |
/benchmarks |
Scoring weights, thresholds, and rule categories (full transparency). |
# 1. Check the service is up
curl https://auditskill.up.railway.app/health
# 2. Audit a skill file (static mode — no network probes)
curl -X POST https://auditskill.up.railway.app/audit \
-H "Content-Type: application/json" \
-d '{"skill_md": "# Weather\n\nGet weather.\n\n## Base URL\nhttps://api.example.com\n\n## Endpoints\nGET /weather?city={city}", "mode": "safe_static"}'
# 3. Audit a skill from the NANDA Town registry by URL (with liveness probes)
curl -X POST https://auditskill.up.railway.app/audit \
-H "Content-Type: application/json" \
-d '{"skill_url": "https://raw.githubusercontent.com/user/repo/main/SKILL.md", "mode": "liveness"}'This is a security product. Its own attack surface is hardened:
- SSRF-safe outbound requests. Every probe passes through a DNS-rebinding-safe guard: scheme allowlist, hostname blocklist (
localhost,*.internal,metadata.google.internal), IP-range blocks (loopback, RFC 1918, link-local 169.254/16, cloud metadata, CGNAT 100.64/10, 0.0.0.0/8, IPv6 equivalents, decimal-encoded loopback). The resolved IP is pinned for the connection while TLS SNI/cert validation stays bound to the real hostname. - Read-only probes. Liveness only sends GET/HEAD. PUT/POST/PATCH/DELETE are never executed.
- Abuse controls. ≤15 endpoints per audit, per-domain caps, 3s per-request timeout, ~25s global timeout, 200 KB input cap,
skill_urlforced HTTPS, per-IP rate limits, skill-hash result cache. - No false-positive self-poisoning. The service audits its own SKILL.md and certifies it
PASS_BASIC_AUDIT— the auditor eats its own dog food without false-flagging the security terms it documents.
pip install -e .
python scripts/generate_keys.py # prints AUDITSKILL_PRIVATE_KEY / _PUBLIC_KEY / _KEY_ID
export AUDITSKILL_PRIVATE_KEY=... # paste from output
export AUDITSKILL_PUBLIC_KEY=...
export AUDITSKILL_KEY_ID=...
uvicorn auditskill.api.main:app --reload --port 8000pip install -e ".[dev]"
pytest -qThe test suite covers: SSRF blocking (including decimal-encoded loopback and cloud-metadata targets), score renormalization, verdict boundaries, Ed25519 signature round-trip with tamper detection, the false-positive guard on legitimate security and payment skills, negative samples for every FP-prone rule (negated "does not override" phrasing, plain pip install from an index, capped auto-pay, an X-Api-Key auth doc, placeholder sk-...), evasion resistance (zero-width-spliced and homoglyph-disguised injections, short Base64-smuggled payloads), the per-model cost estimator (unknown-model rejection, single-model narrowing), deterministic /discover ranking, URL-finding de-duplication, method-mismatch crash regression, skill-name sanitisation, non-skill/empty-document rejection, plain-Markdown parsing, and end-to-end verdicts on good/evil/benign/broken/supply-chain/payment-trap fixtures.
Rule-based and deterministic. AuditSkill flags known-dangerous patterns and tests reachability; it does not prove semantic correctness or future safety. A PASS means "no red flags found," not a guarantee. Liveness is a point-in-time check and never exercises write endpoints. Context-cost estimation uses character-based token heuristics (a ÷4 headline estimate; calibrated chars-per-token ratios per model family in per_model), not a real tokenizer — the honest error bar is error_margin_pct (~10%).
MIT