From 8931c1337567c6b8cbf7bc255701790b7394345c Mon Sep 17 00:00:00 2001 From: alexsoyes Date: Sat, 25 Jul 2026 14:02:26 +0200 Subject: [PATCH 1/3] feat(aidd-dev): rebuild audit around core questions Focus audit effort on nine evidence-gated pillars and durable reports. --- .../phase-1.md | 134 ++++++++++++++ .../phase-2.md | 134 ++++++++++++++ .../phase-3.md | 128 +++++++++++++ .../phase-4.md | 113 ++++++++++++ .../plan.md | 47 +++++ .../review.md | 59 ++++++ plugins/aidd-dev/CATALOG.md | 22 ++- plugins/aidd-dev/README.md | 2 +- plugins/aidd-dev/skills/04-audit/SKILL.md | 68 ++++--- .../skills/04-audit/actions/00-core.md | 31 ++++ .../04-audit/actions/01-code-quality.md | 35 ++-- .../04-audit/actions/02-architecture.md | 37 ++-- .../skills/04-audit/actions/03-security.md | 41 +++-- .../04-audit/actions/04-dependencies.md | 30 --- .../skills/04-audit/actions/05-performance.md | 40 ++-- .../skills/04-audit/actions/06-tests.md | 42 +++-- .../aidd-dev/skills/04-audit/actions/07-ui.md | 31 ---- .../skills/04-audit/actions/08-north-star.md | 34 ++++ .../actions/09-rules-and-principles.md | 38 ++++ .../actions/10-memory-and-documentation.md | 34 ++++ .../skills/04-audit/actions/11-decisions.md | 39 ++++ .../skills/04-audit/actions/12-automation.md | 39 ++++ .../04-audit/assets/audit-actions-template.md | 35 ++++ .../assets/audit-challenge-template.md | 36 ++++ .../assets/audit-coverage-template.md | 44 +++++ .../04-audit/assets/audit-index-template.md | 40 ++++ .../04-audit/assets/audit-scope-template.md | 49 +++++ .../assets/audit-synthesis-template.md | 32 ++++ .../skills/04-audit/assets/audit-template.md | 73 ++++---- .../04-audit/assets/audit-validator.yml | 171 ++++++++++++++++++ .../04-audit/references/audit-contract.md | 79 ++++++++ .../04-audit/references/question-packs.md | 86 +++++++++ .../04-audit/references/question-protocol.md | 89 +++++++++ .../04-audit/references/report-contract.md | 102 +++++++++++ 34 files changed, 1806 insertions(+), 208 deletions(-) create mode 100644 aidd_docs/tasks/2026_07/2026_07_24_question-driven-audit-v2/phase-1.md create mode 100644 aidd_docs/tasks/2026_07/2026_07_24_question-driven-audit-v2/phase-2.md create mode 100644 aidd_docs/tasks/2026_07/2026_07_24_question-driven-audit-v2/phase-3.md create mode 100644 aidd_docs/tasks/2026_07/2026_07_24_question-driven-audit-v2/phase-4.md create mode 100644 aidd_docs/tasks/2026_07/2026_07_24_question-driven-audit-v2/plan.md create mode 100644 aidd_docs/tasks/2026_07/2026_07_24_question-driven-audit-v2/review.md create mode 100644 plugins/aidd-dev/skills/04-audit/actions/00-core.md delete mode 100644 plugins/aidd-dev/skills/04-audit/actions/04-dependencies.md delete mode 100644 plugins/aidd-dev/skills/04-audit/actions/07-ui.md create mode 100644 plugins/aidd-dev/skills/04-audit/actions/08-north-star.md create mode 100644 plugins/aidd-dev/skills/04-audit/actions/09-rules-and-principles.md create mode 100644 plugins/aidd-dev/skills/04-audit/actions/10-memory-and-documentation.md create mode 100644 plugins/aidd-dev/skills/04-audit/actions/11-decisions.md create mode 100644 plugins/aidd-dev/skills/04-audit/actions/12-automation.md create mode 100644 plugins/aidd-dev/skills/04-audit/assets/audit-actions-template.md create mode 100644 plugins/aidd-dev/skills/04-audit/assets/audit-challenge-template.md create mode 100644 plugins/aidd-dev/skills/04-audit/assets/audit-coverage-template.md create mode 100644 plugins/aidd-dev/skills/04-audit/assets/audit-index-template.md create mode 100644 plugins/aidd-dev/skills/04-audit/assets/audit-scope-template.md create mode 100644 plugins/aidd-dev/skills/04-audit/assets/audit-synthesis-template.md create mode 100644 plugins/aidd-dev/skills/04-audit/assets/audit-validator.yml create mode 100644 plugins/aidd-dev/skills/04-audit/references/audit-contract.md create mode 100644 plugins/aidd-dev/skills/04-audit/references/question-packs.md create mode 100644 plugins/aidd-dev/skills/04-audit/references/question-protocol.md create mode 100644 plugins/aidd-dev/skills/04-audit/references/report-contract.md diff --git a/aidd_docs/tasks/2026_07/2026_07_24_question-driven-audit-v2/phase-1.md b/aidd_docs/tasks/2026_07/2026_07_24_question-driven-audit-v2/phase-1.md new file mode 100644 index 000000000..5bde8e7ad --- /dev/null +++ b/aidd_docs/tasks/2026_07/2026_07_24_question-driven-audit-v2/phase-1.md @@ -0,0 +1,134 @@ +--- +status: done +--- + + + +# Instruction: Agree the V2 contract + +## Architecture projection + +> Tree of the final files. ✅ create · ✏️ modify · ❌ delete + +```txt +plugins/aidd-dev/skills/04-audit/ +└── references/ + ├── audit-contract.md ✅ canonical inputs, states, outputs, invariants, and failure modes + ├── question-protocol.md ✅ question -> hypothesis -> investigation -> verdict protocol + ├── question-packs.md ✅ core prompts plus project-supplied principle packs + └── report-contract.md ✅ numbered files, ownership, cross-links, freshness, and document assembly +``` + +## User Journey + +```mermaid +flowchart TD + A["User states the audit goal and target"] --> B["Skill inventories available code, runtime, history, rules, and principles"] + B --> C["Skill proposes question packs and exposes missing sources"] + C --> D["User confirms the contract and scope"] + D --> E["Implementation phases may start"] +``` + +## Test Scope + +```mermaid +--- +title: Test scope +--- +journey + section Happy path + User supplies goals and principles => contract exposes choice and automation questions => user approves the numbered report: 5: User, Planner + section Edge case - missing decision history + No decision source exists => reconstructed choice is labelled inferred or unknown: 1: Planner + section Edge case - missing North Star + No current North Star resolves => North Star pillar is skipped with reason => remaining pillars continue: 1: Planner + section Edge case - runtime unavailable + Static evidence cannot settle a non-critical concern => no E2E runs and the concern remains unknown: 1: Planner +``` + +## Tasks to do + +### `1)` Define the audit state machine + +> Make introspection useful without turning model sentiment into a verdict. + +1. Define six states: `question`, `impression`, `hypothesis`, `investigation`, `finding`, and `unknown`. +2. Require every impression to name what evidence would confirm or refute it. +3. Permit promotion to `finding` only after evidence is inspected. +4. Route unsupported concerns to `unknown`; never silently discard or present them as facts. +5. Label reconstructed implementation decisions as inference unless history, ADRs, tickets, or source comments establish them. +6. Let decision quality determine inspection depth: underspecified, low-confidence, local-only, or irreversible choices receive deeper inspection. +7. Define runtime inputs as optional target path, scope, North Star sources, rule sources, memory path, current architecture sources, and budget. +8. Default the target to the current repository; discover source candidates there and record what resolved. +9. A missing source degrades only its dependent pillar to `unknown` or `skipped`; it never blocks unrelated pillars. + +### `2)` Define the question contract + +> Ask uncomfortable questions that reveal hidden assumptions and regrettable decisions. + +1. Include pride and discomfort: “Would you defend this code?”, “What feels clever rather than clear?”, “What would embarrass you during an incident?” +2. Include blindness: “What can you not know from the available context?”, “Which assumptions have no validating observation?” +3. Include regret and counterfactuals: “Which decision was expedient but suboptimal?”, “What would you not choose again?”, “What became harder because of it?” +4. Include choice provenance: “Which choices came from the plan?”, “Which choices did the agent make because the work was underspecified?”, “Which choices are only inferred after the fact?” +5. Include generality: “Did this solve the class of problem or only the observed example?”, “Which local constant, buffer, threshold, or special case hides an unresolved cause?” +6. Include coherence: “Which two parts tell different stories?”, “Which rule is followed literally but violated in spirit?” +7. Include user and operational harm: “Where can the dashboard mislead?”, “Which failure looks like success?”, “What breaks silently?” +8. Include test value: “Which tests can pass while the product is broken?”, “Which tests protect no meaningful risk?”, “What important behavior has no witness?” +9. Include automation leverage: “Will this class recur?”, “What permanent guardrail would prevent it?”, “Could a fresh agent make the right choice from repository context alone?” +10. Allow a project to add questions sourced from AIDD rules, golden principles, architecture decisions, or domain constraints. +11. Exclude style preference, harmless drift, and low-impact completeness work from the default profile. + +### `3)` Define the evidence and verdict contract + +> Make every conclusion traceable and reproducible. + +1. Accept evidence from code locations, runtime observations, screenshots, logs, commands, traces, coverage, git history, tickets, and normative documents. +2. Record criterion, observation, interpretation, confidence, impact, likelihood, reach, reproduction, and suggested next action. +3. Distinguish `pass`, `partial`, `fail`, `unknown`, and `disputed` control outcomes. +4. Replace the current health shortcut with a risk summary that cannot be “good” solely because no critical finding exists. +5. Define duplicate and contradiction rules before parallel execution is designed. +6. Set runtime policy to static-first: no general E2E; allow one bounded, read-only runtime probe only for a plausible critical defect that static evidence cannot settle. +7. For every decision, record provenance, confidence, alternatives, assumptions, scope, generality, reversibility, affected surfaces, and verification evidence. +8. Prefer a decision log captured by the implementing context; when absent, reconstruct from plans, history, code, and tickets and label every reconstructed choice. +9. For every recurring issue, select the strongest suitable infrastructure target: + - executable invariant => type, schema, lint, static analysis, test, or CI; + - domain or architectural guidance => rule, skill, memory, code comment, or canonical documentation; + - runtime behavior => assertion, telemetry, alert, or operational check. +10. Keep E2E as a possible remediation recommendation when it is the right permanent guardrail, while preserving the audit's no-general-E2E execution policy. +11. Use this source hierarchy: + - current code, configuration, schemas, lockfiles, and tests as implementation truth; + - user-confirmed North Star sources as product truth; + - active host rules and instructions as behavioral constraints; + - `aidd_docs/memory/**` as maintained project knowledge to verify against code; + - explicitly current architecture records as intended structure. +12. Exclude `aidd_docs/tasks/**`, old plans, past reviews, and other historical artifacts unless the user names one as a current normative source. +13. Limit each pillar to its five highest-impact confirmed findings; record material uncertainty, but omit cosmetic nits and exhaustive low-value inventories. + +### `4)` Define the report package + +> One folder, many small owned files, one deterministic reading order. + +1. Require `aidd_docs/tasks//_audit_/`. +2. Reserve `00-summary.md` for the executive view and `01` to `14` for scope, nine core pillars, challenge, synthesis, actions, and coverage. +3. Give every file one exclusive owner role, stable finding IDs, source dependencies, and a last-verified marker. +4. Cross-link canonical findings instead of copying them between pillar, summary, and action files. +5. Define deterministic concatenation order so the folder can become one document without rewriting its contents. +6. Allow one pillar file to be refreshed without invalidating unrelated files; mark dependent synthesis files stale until regenerated. + +### `5)` Hold the contract gate + +> No audit implementation before the human agrees on what a valid audit means. + +1. Present the question packs, evidence rules, report schema, and unresolved tradeoffs. +2. Ask for corrections to tone, rigor, project customisation, and acceptable runtime cost. +3. Revise the four reference files until the user explicitly approves the contract. + +## Test acceptance criteria + +| Task | Acceptance criteria | +| ---- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| 1 | Runtime inputs are discovered from the current repository; missing sources degrade only dependent pillars and unsupported impressions remain unknown. | +| 2 | The contract covers pride, blind spots, regret, choice provenance, generality, rules, memory truth, code hotspots, test value, and automation leverage. | +| 3 | Findings accept static and runtime evidence, carry confidence and risk dimensions, and controls can remain unknown or disputed. | +| 4 | The report contract defines `00` to `14`, exclusive ownership, stable links, independent refresh, and deterministic document assembly. | +| 5 | The contract files are treated as unapproved until the user explicitly accepts them; phases 2 to 4 do not start before that gate. | diff --git a/aidd_docs/tasks/2026_07/2026_07_24_question-driven-audit-v2/phase-2.md b/aidd_docs/tasks/2026_07/2026_07_24_question-driven-audit-v2/phase-2.md new file mode 100644 index 000000000..9c477df30 --- /dev/null +++ b/aidd_docs/tasks/2026_07/2026_07_24_question-driven-audit-v2/phase-2.md @@ -0,0 +1,134 @@ +--- +status: done +--- + + + +# Instruction: Rebuild the audit recipe + +## Architecture projection + +> Tree of the final files. ✅ create · ✏️ modify · ❌ delete + +```txt +plugins/aidd-dev/ +├── README.md ✏️ describe question-led, evidence-gated audits +└── skills/04-audit/ + ├── SKILL.md ✏️ route profiles and enforce the approved V2 contract + ├── actions/ + │ ├── 00-core.md ✅ run and assemble the standalone 80/20 report + │ ├── 01-code-quality.md ✏️ replace flat scan with lens-specific questions and probes + │ ├── 02-architecture.md ✏️ add decision regret, drift, and counterfactual probes + │ ├── 03-security.md ✏️ add threat assumptions and unknown-boundary probes + │ ├── 04-dependencies.md ❌ remove non-core lens + │ ├── 05-performance.md ✏️ add budget, measurement, and silent-degradation probes + │ ├── 06-tests.md ✏️ audit defect-detection value, assertions, realism, and cost + │ ├── 07-ui.md ❌ remove non-core lens + │ ├── 08-north-star.md ✅ audit code against current product intent and critical outcomes + │ ├── 09-rules-and-principles.md ✅ evaluate supplied rules and principles in letter and spirit + │ ├── 10-memory-and-documentation.md ✅ audit knowledge completeness, freshness, and code alignment + │ ├── 11-decisions.md ✅ audit provenance, confidence, generality, and reversibility + │ └── 12-automation.md ✅ convert recurring risks and tacit knowledge into infrastructure + ├── assets/ + │ ├── audit-actions-template.md ✅ prioritised remediation sequence + │ ├── audit-challenge-template.md ✅ accepted, rejected, merged, and disputed candidates + │ ├── audit-coverage-template.md ✅ scanned, skipped, stale, inaccessible, and unknown + │ ├── audit-template.md ✏️ render one owned pillar shard + │ ├── audit-index-template.md ✅ ordered links, owners, status, and freshness + │ ├── audit-scope-template.md ✅ target, resolved sources, exclusions, budget, and policy + │ ├── audit-synthesis-template.md ✅ systemic patterns and canonical finding links + │ └── audit-validator.yml ✅ closed report schema and required fields + └── references/ + ├── audit-contract.md ✏️ only if implementation exposes an approved ambiguity + ├── evidence-rubric.md ✅ evidence strength, confidence, risk, and promotion rules + ├── question-protocol.md ✏️ connect the approved protocol to action execution + └── question-packs.md ✏️ define the nine-pillar 80/20 profile plus optional lenses +``` + +## User Journey + +```mermaid +flowchart TD + A["Audit checker receives target, goal, and question packs"] --> B["Ask one question and record an impression"] + B --> C["Turn the impression into a falsifiable hypothesis"] + C --> D["Inspect the cheapest decisive evidence"] + D --> E{"Evidence settles it?"} + E -->|"yes"| F["Record pass, partial, or finding"] + E -->|"no"| G["Record unknown or disputed"] + F --> H["Continue until declared scope is covered"] + G --> H +``` + +## Test Scope + +```mermaid +--- +title: Test scope +--- +journey + section Happy path + Caller selects a lens => questions expose choices and risks => evidence settles hypotheses => one valid shard is written: 5: Audit checker + section Edge case - unsupported impression + A concern has no decisive evidence => it is recorded as unknown and never promoted to finding: 1: Audit checker + section Edge case - recurring failure + The same issue class can recur => the strongest suitable automation layer is proposed: 1: Audit checker + section Edge case - UI concern + A UI suspicion is not plausibly critical => no runtime or E2E probe runs: 1: Audit checker +``` + +## Tasks to do + +### `1)` Make the router profile-driven + +> Preserve the useful pillars while allowing project-specific inquiry. + +1. Accept the complete nine-pillar `core`, one named core pillar, or a custom question pack mapped to core pillars. +2. Resolve the target from arguments or the current repository, then discover North Star candidates, active host instructions, scoped project rules, memory, current architecture records, and user-supplied principles. +3. Show source conflicts and precedence uncertainty; never invent a canonical source. +4. Keep the recipe read-only and caller-agnostic; do not spawn agents here. +5. Require the audit contract and relevant action file to be read before scanning. + +### `2)` Convert each action to inquiry plus verification + +> The model should reason broadly, then prove narrowly. + +1. Give every lens opening questions, likely probes, decisive evidence, and explicit non-goals. +2. Replace generic “scan for” instructions with the approved state machine. +3. Upgrade the tests lens from coverage counting to risk protection and defect-detection value. + - Map each meaningful test group to the user or system risk it protects. + - Ask which plausible regression or mutation would make the test fail. + - Compare signal quality with runtime, flakiness, fixture complexity, duplication, and maintenance cost. + - Classify it as `protective`, `redundant`, `brittle`, `ceremonial`, or `misleading`. + - Recommend `keep`, `rewrite`, `merge`, or `delete`; recommend deletion only when no distinct protected risk remains. +4. Prohibit a general E2E pass; permit one targeted runtime probe only for a plausible critical defect that static evidence cannot settle. +5. Make performance conclusions depend on a stated budget or measurement; otherwise retain them as hypotheses or unknowns. +6. Add conformance checks that cite the exact normative statement and distinguish literal compliance from intent compliance. +7. Audit choices before implementation detail: inventory specified, agent-made, and retrospectively inferred choices; deepen inspection around low-confidence, local-only, or irreversible choices. +8. Audit automation leverage: treat repeated review feedback, tribal knowledge, recurring fixes, and extra prompting as candidates for permanent repository infrastructure. +9. Make the default profile ignore historical task artifacts, skip cosmetic findings, stop each pillar at five high-impact findings, and expose no non-core lens. + +### `3)` Replace the report contract + +> Preserve the audit's reasoning, not just its conclusions. + +1. Give controls and findings stable IDs. +2. Record the originating question, criterion, evidence kind, evidence reference, interpretation, confidence, impact, likelihood, reach, reproduction, next action, and effort. +3. Add separate sections for confirmed findings, passed controls, unknowns, disputes, systemic patterns, coverage, and top actions. +4. Validate the report against a closed schema without requiring every proof to be a `file:line`. +5. Keep the report concise by linking evidence rather than pasting logs or duplicating source text. +6. Add a decision table with provenance, confidence, alternatives, assumptions, generality, reversibility, evidence, and verdict. +7. Add an automation-candidate table with recurrence, current human or token cost, proposed enforcement layer, leverage, false-positive risk, and owner. + +## Test acceptance criteria + +| Task | Acceptance criteria | +| ---- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 1 | A caller can run the complete nine-pillar core, one named core pillar, or a custom question pack mapped to core pillars; the recipe never spawns an agent. | +| 1 | With no explicit paths, the skill audits the current repository, records discovered sources, and skips only pillars whose required source is absent. | +| 2 | Every lens follows question → hypothesis → evidence → verdict, and conformance checks cite both the rule and implementation evidence. | +| 2 | Choice review leads implementation inspection but never replaces targeted verification of undeclared choices, drift, correctness, or security. | +| 2 | A repeated issue yields an automation candidate at the strongest suitable layer, while one-off preferences do not become permanent machinery. | +| 2 | The default profile ignores old plans, caps each pillar at five consequential findings, and emits no cosmetic style nits. | +| 3 | Test findings name the protected risk, plausible regression, signal quality, and cost; coverage percentage alone is never a finding. | +| 3 | A delete recommendation proves that the test protects no distinct risk beyond another cheaper test; uncertainty never becomes a deletion recommendation. | +| 3 | Runtime-only conclusions cite runtime evidence, unsupported concerns remain unknown, and the rendered report passes its schema validator. | diff --git a/aidd_docs/tasks/2026_07/2026_07_24_question-driven-audit-v2/phase-3.md b/aidd_docs/tasks/2026_07/2026_07_24_question-driven-audit-v2/phase-3.md new file mode 100644 index 000000000..c508a3e3d --- /dev/null +++ b/aidd_docs/tasks/2026_07/2026_07_24_question-driven-audit-v2/phase-3.md @@ -0,0 +1,128 @@ +--- +status: pending +--- + + + +# Instruction: Orchestrate independent audit checkers + +## Architecture projection + +> Tree of the final files. ✅ create · ✏️ modify · ❌ delete + +```txt +plugins/aidd-orchestrator/ +├── .claude-plugin/plugin.json ✏️ register the audit orchestration skill +├── README.md ✏️ document the audit use case and portability behavior +├── CATALOG.md ✏️ regenerate the plugin catalog +└── skills/01-audit/ + ├── SKILL.md ✅ frame → fan-out → challenge → synthesize router + ├── actions/ + │ ├── 01-frame.md ✅ inventory surfaces, sources, questions, budget, and shards + │ ├── 02-fan-out.md ✅ one isolated checker per non-overlapping audit shard + │ ├── 03-challenge.md ✅ reject weak, duplicate, and contradictory candidate findings + │ └── 04-synthesize.md ✅ write the sole merged report from accepted shards + └── references/ + ├── orchestration-contract.md ✅ ownership, concurrency, fallback, and stop conditions + └── synthesis-rubric.md ✅ deduplication, conflict handling, ranking, and completeness +aidd_docs/tasks//_audit_/ +├── 00-summary.md ✅ synthesizer: executive view and ordered contents +├── 01-scope-and-system-map.md ✅ framer: target, surfaces, sources, exclusions, budgets +├── 02-north-star.md ✅ checker instance: current product intent, critical outcomes, code divergence +├── 03-rules.md ✅ checker instance: active rules, violations, letter and spirit +├── 04-memory.md ✅ checker instance: memory claims versus current code and usage +├── 05-architecture-and-decisions.md ✅ checker instance: boundaries, divergence, uncertainty, generality +├── 06-code-hotspots.md ✅ checker instance: oversized, complex, duplicated, dead, or risky code +├── 07-tests-value.md ✅ checker instance: protected risk, assertions, redundancy, flake, cost +├── 08-security-and-data.md ✅ checker instance: auth, privacy, integrity, unsafe boundaries +├── 09-performance-and-reliability.md ✅ checker instance: slow paths, heavy files, failure modes, observability +├── 10-automation-and-knowledge-infrastructure.md ✅ checker instance: recurring work, tacit knowledge, permanent guardrails +├── 11-challenge-ledger.md ✅ checker: accepted, rejected, duplicate, disputed candidates +├── 12-systemic-findings.md ✅ synthesizer: cross-pillar causes and canonical finding links +├── 13-prioritized-actions.md ✅ synthesizer: highest-leverage fixes and automation investments +└── 14-coverage-and-unknowns.md ✅ synthesizer: examined, skipped, stale, inaccessible, uncertain +``` + +## User Journey + +```mermaid +flowchart TD + A["User requests a broad audit"] --> B["Orchestrator frames questions, sources, surfaces, and budget"] + B --> C["Independent checkers investigate exclusive audit shards in parallel"] + C --> D["Adversarial checker challenges every candidate"] + D --> E["Synthesizer deduplicates, clusters causes, and ranks accepted findings"] + E --> F["User receives one report with findings, passes, unknowns, disputes, and coverage"] +``` + +## Test Scope + +```mermaid +--- +title: Test scope +--- +journey + section Happy path + User confirms pillars => checkers write exclusive shards in parallel waves => fresh checkers challenge findings => synthesis writes 00 to 14: 5: User, Orchestrator + section Edge case - no agent support + Agent spawning is unavailable => shards run serially => independence limitation is disclosed: 1: Orchestrator + section Edge case - duplicate findings + Two shards report one root cause => challenge ledger keeps one canonical finding and records the merge: 1: Checker + section Edge case - incomplete shard + A shard times out => synthesis marks it skipped or unknown instead of inventing coverage: 1: Orchestrator +``` + +## Tasks to do + +### `1)` Frame a bounded audit + +> Parallelism begins only after scope and ownership are deterministic. + +1. Inventory current code surfaces, North Stars, active rules, memory, data boundaries, and only the history needed to establish a material architectural decision. +2. Resolve the target from the user's argument or current repository; treat every other source path as an optional runtime discovery result. +3. Select question packs based on the stated audit goal, using files `02` to `10` as the default 80/20 pillar set. +4. Let the user add, remove, split, merge, or reorder pillars before spawning; filenames become stable after confirmation. +5. Split by independent lens or surface; give each shard an identifier, exclusive output file, and explicit exclusions. +6. Set agent count, concurrency cap, tool/time budget, runtime access, and stop conditions. +7. Ask one blocking question only when unresolved precedence would materially change a high-impact verdict; otherwise mark the dependent pillar unknown or skipped. +8. Set `runtime-policy: static-first`; never schedule a broad E2E pass as an audit shard. + +### `2)` Fan out without contaminating evidence + +> Independent contexts should create independent judgments. + +1. Spawn one existing `checker` instance per pillar shard and brief it to invoke the audit recipe. +2. Discover the audit recipe by capability, never by a hard-coded sibling plugin name. +3. Give each checker the approved contract, target, question pack, normative sources, exclusions, evidence budget, stop conditions, and unique shard path. +4. Prevent agents from editing application code or the merged report. +5. Run more pillars than available concurrency in waves; do not reduce the pillar set to the host's slot count. +6. When agents are unavailable, run shards serially and mark independence as unavailable. + +### `3)` Challenge candidate findings + +> Treat eloquent self-critique as guilty until proven. + +1. Spawn fresh checker instances over non-overlapping groups of completed shards; give them findings and evidence, not originating hidden reasoning. +2. Reject findings that lack a criterion, decisive evidence, reproducibility, or a plausible impact. +3. Merge duplicates only when they share the same underlying cause. +4. Mark incompatible interpretations as disputed; do not average them into certainty. +5. Write every challenge decision to `11-challenge-ledger.md`; preserve rejected candidates with the rejection reason outside final findings. + +### `4)` Synthesize one risk-ranked report + +> One owner converts shards into a coherent system view. + +1. Validate every shard against the V2 report schema and its exclusive owner. +2. Cluster accepted findings into systemic causes and isolated symptoms. +3. Rank by impact, likelihood, reach, confidence, and remediation effort. +4. Report positive controls, unknowns, disputes, skipped surfaces, runtime limitations, and independence level. +5. Write `00` and `12` to `14` only after every shard is completed, skipped with reason, or timed out. +6. Assemble a transferable document by concatenating `00` to `14`; never maintain a second copy of the prose. + +## Test acceptance criteria + +| Task | Acceptance criteria | +| ---- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 1 | The user can confirm the default `02` to `10` pillar set, and every confirmed surface belongs to one named shard or is explicitly skipped with a reason. | +| 2 | One checker instance owns each pillar file and runs the audit recipe in parallel waves; unavailable agent support produces a disclosed serial run. | +| 3 | Speculative, duplicate, and contradictory candidates are respectively rejected, merged by cause, or marked disputed with the challenge decision retained. | +| 4 | `00` to `14` form one ordered, cross-linked report package that can be refreshed per shard and concatenated without duplicate source prose. | diff --git a/aidd_docs/tasks/2026_07/2026_07_24_question-driven-audit-v2/phase-4.md b/aidd_docs/tasks/2026_07/2026_07_24_question-driven-audit-v2/phase-4.md new file mode 100644 index 000000000..ce8dfb5b3 --- /dev/null +++ b/aidd_docs/tasks/2026_07/2026_07_24_question-driven-audit-v2/phase-4.md @@ -0,0 +1,113 @@ +--- +status: pending +--- + + + +# Instruction: Prove behavior and publish + +## Architecture projection + +> Tree of the final files. ✅ create · ✏️ modify · ❌ delete + +```txt +scripts/ +├── skill-eval.mjs ✏️ accept cases from skills outside aidd-refine +└── skill-eval/ + ├── README.md ✏️ document audit and orchestration evals + └── cases.json ✏️ add deterministic and judged V2 audit cases +plugins/ +├── aidd-dev/CATALOG.md ✏️ regenerate entries for the V2 files +└── aidd-orchestrator/CATALOG.md ✏️ regenerate entries for the audit orchestrator +README.md ✏️ sync plugin skill counts if generated checks require it +``` + +## User Journey + +```mermaid +flowchart TD + A["Maintainer runs deterministic evals"] --> B{"Schema and invariants pass?"} + B -->|"no"| C["Repair the contract implementation"] + C --> A + B -->|"yes"| D["Maintainer opts into judged behavioral evals"] + D --> E{"Question quality and evidence discipline pass?"} + E -->|"no"| C + E -->|"yes"| F["Regenerate catalogs and publish the documented capability"] +``` + +## Test Scope + +```mermaid +--- +title: Test scope +--- +journey + section Happy path + Maintainer runs deterministic and judged cases => audit invariants pass => catalogs and documentation are regenerated: 5: Maintainer + section Edge case - no metered judge + Authenticated judge is unavailable => semantic cases are reported skipped while deterministic gates still run: 1: Maintainer + section Edge case - source mutation + An audit changes fixture application bytes => behavioral validation fails: 1: Maintainer + section Edge case - local-only fix + Example passes but the problem class remains => generality eval rejects the finding resolution: 1: Maintainer +``` + +## Tasks to do + +### `1)` Generalise the behavioral harness + +> Test the worktree skill, never a globally installed copy. + +1. Let each case identify its source plugin and skill folder. +2. Preserve unique temporary skill names and isolated project settings. +3. Add deterministic assertions for files, closed sections, statuses, evidence references, and forbidden unsupported findings. +4. Keep semantic quality behind the explicit metered judge mode. + +### `2)` Add adversarial audit cases + +> Test failure modes, not only the happy report shape. + +1. A “not proud” impression without evidence remains an unknown. +2. A documented rule plus violating code produces a conformance finding citing both. +3. A high coverage suite with weak assertions produces a test-value finding without treating coverage itself as the problem. +4. Two audit checkers reporting one root cause produce one merged finding. +5. Conflicting interpretations remain disputed. +6. Missing runtime or agent support is disclosed and never fabricated. +7. A screenshot, log, or runtime observation is accepted as evidence without a fake `file:line`. +8. A normal UI audit performs no E2E; only a plausible critical suspicion can trigger one bounded runtime probe with its escalation reason recorded. +9. A declared high-confidence decision narrows inspection but cannot suppress contradictory implementation evidence. +10. A local fix that passes its example while leaving the general cause intact is rejected as non-general. +11. Repeated review feedback produces a concrete knowledge-infrastructure candidate; a one-off preference does not. +12. Historical plans containing obsolete statements do not produce findings unless explicitly declared current. +13. A stale memory claim contradicted by current code produces a memory finding. +14. A pillar with more than five valid issues reports only its five highest-impact findings and records the prioritisation boundary. +15. With no target argument, the current repository is audited. +16. With no North Star source, only the North Star pillar is skipped and every source-independent pillar still completes. + +### `3)` Run structural and behavioral validation + +> Prove portability, report validity, and question-to-evidence discipline. + +1. Run Markdown link, JSON/YAML, frontmatter, argument-hint, and catalog checks. +2. Run deterministic audit evals. +3. Run judged evals when an authenticated headless model is available; report metered checks as skipped otherwise. +4. Execute one fixture audit across at least three lenses and inspect its merged report. +5. Verify the application fixture remains byte-for-byte unchanged. + +### `4)` Publish the new contract + +> Make the feature discoverable without duplicating its canonical rules. + +1. Update the two plugin landing pages with their distinct responsibilities. +2. Link documentation to the canonical audit contract and orchestration contract. +3. Regenerate catalogs and derived counts. +4. Record known limitations: model variance, runtime access, historical intent inference, cost, and host agent support. + +## Test acceptance criteria + +| Task | Acceptance criteria | +| ---- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 1 | Eval cases can load both development and orchestration skills from the worktree under unique temporary names. | +| 2 | The suite catches unsupported sentiment-as-fact, duplicated roots, hidden disagreement, fabricated runtime evidence, and undisclosed serial fallback. | +| 3 | Structural checks and deterministic evals pass; metered judged eval status is reported; the fixture source is unchanged after a full audit. | +| 4 | Plugin documentation exposes the V2 capability and its limitations while linking, rather than copying, the canonical contracts. | diff --git a/aidd_docs/tasks/2026_07/2026_07_24_question-driven-audit-v2/plan.md b/aidd_docs/tasks/2026_07/2026_07_24_question-driven-audit-v2/plan.md new file mode 100644 index 000000000..a912eabf2 --- /dev/null +++ b/aidd_docs/tasks/2026_07/2026_07_24_question-driven-audit-v2/plan.md @@ -0,0 +1,47 @@ +--- +objective: "The audit capability uses independently spawned checker agents to find the highest-leverage divergences between current code, North Stars, active rules, and memory, then publishes one maintainable folder of numbered Markdown reports." +status: in-progress +--- + + + +# Plan: Question-driven audit V2 + +## Overview + +| Field | Value | +| ---------- | ------------------------------------------------------------------------------------------------------ | +| **Goal** | Produce a static-first 80/20 audit of current code and normative truth, with small independently owned Markdown shards. | +| **Source** | User conversation on 2026-07-24, including two supplied X posts about knowledge-as-infrastructure and auditing AI-made choices. | + +## Phases + +| # | Phase | File | +| --- | -------------------------------- | ---------------------------- | +| 1 | Agree the V2 contract | [`phase-1.md`](./phase-1.md) | +| 2 | Rebuild the audit recipe | [`phase-2.md`](./phase-2.md) | +| 3 | Orchestrate audit checkers | [`phase-3.md`](./phase-3.md) | +| 4 | Prove behavior and publish | [`phase-4.md`](./phase-4.md) | + +## Decisions + +| Decision | Why | +| ----------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- | +| Questions generate hypotheses; only verified hypotheses become findings | Model self-critique is useful for discovery but is not evidence and can confidently invent intent, history, or causality. | +| Audit declared and reconstructed choices before implementation details | Choice review compresses the search, but undeclared choices and implementation drift still require targeted evidence. | +| Record choice provenance as `specified`, `agent-made`, or `retrospectively-inferred` | A post-hoc model account must not be presented as reliable implementation history. | +| Prefer an implementation-time decision log, but accept labelled retrospective reconstruction | Choices are most reliable while their context is alive; existing systems still need an honest fallback. | +| Turn recurring findings into knowledge-infrastructure candidates | A one-off fix spends reasoning again; types, lint, CI, tests, rules, skills, memory, or docs can prevent the whole class. | +| Ignore historical task documentation by default; scan only current code, North Stars, active rules, memory, and declared current architecture | Old plans are implementation history, not present-tense truth, and their drift is expected. | +| Discover the target and normative sources at audit runtime | Repository paths, North Stars, rules, and memory are project inputs; hard-coding them would make the skill non-portable. | +| Cap each pillar at five high-impact findings and omit cosmetic nits | The audit optimises for leverage and decision quality, not exhaustive defect inventory. | +| Keep audit methodology in `aidd-dev`; put agent fan-out and synthesis in `aidd-orchestrator` | Project architecture assigns domain recipes to their concern and spawning to the coordination layer. | +| Make question packs configurable from project principles instead of hard-coding only universal axes | AIDD rules, golden principles, product constraints, and dashboard-specific risks differ by project. | +| Give each audit checker an exclusive report shard; only the synthesizer writes the merged report | Parallel agents must not race on one file or silently overwrite another agent's evidence. | +| Publish `00-summary.md` plus stable `01` to `14` Markdown files in one audit folder | Nine core pillars plus challenge and synthesis stay compact, refreshable, and deterministically exportable. | +| Spawn the existing `checker` with the audit recipe instead of adding an `auditor` role | The skill owns audit expertise; the agent adds isolation, and `checker` already permits invoking the audit capability. | +| Add an adversarial challenge pass between investigation and synthesis | Duplicate, speculative, and contradictory findings need independent rejection before ranking. | +| Report confidence, evidence kind, impact, likelihood, and reach separately | One severity label cannot compare a likely local smell with a rare systemic failure or an unverified concern. | +| Preserve explicit unknowns and disagreements instead of forcing a verdict | “I do not know” is a valid audit result when history, runtime access, telemetry, or a normative source is missing. | +| Degrade visibly to serial execution when the host cannot spawn agents | Portability matters, but a single-agent run must never be presented as independent parallel review. | +| Default to static inspection; permit only a targeted runtime check for an otherwise unresolvable critical suspicion | A broad E2E pass is expensive, noisy, and outside the desired audit behavior. | diff --git a/aidd_docs/tasks/2026_07/2026_07_24_question-driven-audit-v2/review.md b/aidd_docs/tasks/2026_07/2026_07_24_question-driven-audit-v2/review.md new file mode 100644 index 000000000..9b4dda4a4 --- /dev/null +++ b/aidd_docs/tasks/2026_07/2026_07_24_question-driven-audit-v2/review.md @@ -0,0 +1,59 @@ +# Review: Question-driven audit V2 + +- Verdict: approved +- Diff: `working-tree` +- Axes run: code, functional, relevancy +- Date: 2026_07_25 +- Findings: 0 critical, 0 warning, 0 minor + +## Phases + +### Phase 1: Agree the V2 contract + +- [x] Runtime inputs default to the current repository, resolved sources are recorded, missing sources skip only dependent pillars, and unsupported impressions remain unknown. +- [x] The question protocol covers pride, blind spots, regrettable choices, provenance, generality, rules, memory, hotspots, test value, and automation. +- [x] Findings separate evidence, observation, interpretation, impact, likelihood, reach, confidence, reproduction, and falsification. +- [x] The report contract defines `00` through `14`, canonical ownership, stable links, refresh semantics, and deterministic order. +- [x] The contract gate was completed before implementation. + +### Phase 2: Rebuild the audit recipe + +- [x] A caller can run the complete nine-pillar core, one named core pillar, or a custom question pack mapped to core pillars; the recipe never spawns an agent. +- [x] No explicit path defaults to the current repository; resolved and missing sources are recorded. +- [x] Every pillar uses question → hypothesis → evidence → verdict, and rule checks require normative plus implementation evidence. +- [x] Choice review leads inspection without replacing contradictory implementation evidence. +- [x] Recurring issues trigger proportionate automation candidates while one-off preferences are excluded. +- [x] The default profile ignores historical task artifacts, caps consequential findings, and excludes cosmetic nits. +- [x] Test findings require protected risk, plausible regression, signal failure, and cost; coverage alone is insufficient. +- [x] Test deletion requires proof that no unique protection remains. +- [x] Runtime-only conclusions require runtime evidence, unsupported concerns remain unknown, and the report has a closed structural contract. + +### Phase 3: Orchestrate the parallel audit + +- [ ] Confirm the default shard set and explicit skips — not-applicable: orchestration remains a separate pending phase. +- [ ] Assign one checker per shard with disclosed serial fallback — not-applicable: orchestration remains a separate pending phase. +- [ ] Challenge, merge, reject, or dispute candidate findings — not-applicable: orchestration remains a separate pending phase. +- [ ] Assemble and refresh the parallel report package — not-applicable: orchestration remains a separate pending phase. + +### Phase 4: Prove the workflow + +- [ ] Load isolated development and orchestration fixtures — not-applicable: behavioural evaluation was explicitly deferred. +- [ ] Exercise the deterministic failure cases — not-applicable: behavioural evaluation was explicitly deferred. +- [ ] Pass structural and behavioural checks without fixture mutation — not-applicable: behavioural evaluation was explicitly deferred. +- [ ] Publish the final documented workflow — not-applicable: phase 4 remains pending. + +## Findings + +No findings. + +## Verification + +- In-scope acceptance criteria verified: 14/14, 100%. +- Core contract check: 9 pillars, 16 required finding fields, and 3 structured schemas. +- YAML validation passed for `audit-validator.yml`. +- Skill argument hints match the action files. +- Markdown link validation found 0 broken links across 551 files. +- `git diff --check` returned no output. +- The generic Codex skill validator is not applicable to this Claude Code plugin format because the framework requires `argument-hint` and `model` frontmatter. +- Behavioural execution remains deferred to phase 4. +- Unplanned changes: none found. diff --git a/plugins/aidd-dev/CATALOG.md b/plugins/aidd-dev/CATALOG.md index 399feb29a..cf738dd8e 100644 --- a/plugins/aidd-dev/CATALOG.md +++ b/plugins/aidd-dev/CATALOG.md @@ -87,15 +87,29 @@ Auto-generated index of skills, agents, references and assets shipped by the `ai | Group | File | Description | |-------|------|---| +| `actions` | [00-core.md](skills/04-audit/actions/00-core.md) | - | | `actions` | [01-code-quality.md](skills/04-audit/actions/01-code-quality.md) | - | | `actions` | [02-architecture.md](skills/04-audit/actions/02-architecture.md) | - | | `actions` | [03-security.md](skills/04-audit/actions/03-security.md) | - | -| `actions` | [04-dependencies.md](skills/04-audit/actions/04-dependencies.md) | - | | `actions` | [05-performance.md](skills/04-audit/actions/05-performance.md) | - | | `actions` | [06-tests.md](skills/04-audit/actions/06-tests.md) | - | -| `actions` | [07-ui.md](skills/04-audit/actions/07-ui.md) | - | -| `assets` | [audit-template.md](skills/04-audit/assets/audit-template.md) | `Codebase audit report template` | -| `-` | [SKILL.md](skills/04-audit/SKILL.md) | `Audit a codebase read-only across seven quality pillars into one ranked report. Use when the user wants to assess, health-check, or audit a codebase or one pillar. Not for fixing findings, reviewing a change, or checking a feature works.` | +| `actions` | [08-north-star.md](skills/04-audit/actions/08-north-star.md) | - | +| `actions` | [09-rules-and-principles.md](skills/04-audit/actions/09-rules-and-principles.md) | - | +| `actions` | [10-memory-and-documentation.md](skills/04-audit/actions/10-memory-and-documentation.md) | - | +| `actions` | [11-decisions.md](skills/04-audit/actions/11-decisions.md) | - | +| `actions` | [12-automation.md](skills/04-audit/actions/12-automation.md) | - | +| `assets` | [audit-actions-template.md](skills/04-audit/assets/audit-actions-template.md) | - | +| `assets` | [audit-challenge-template.md](skills/04-audit/assets/audit-challenge-template.md) | - | +| `assets` | [audit-coverage-template.md](skills/04-audit/assets/audit-coverage-template.md) | - | +| `assets` | [audit-index-template.md](skills/04-audit/assets/audit-index-template.md) | - | +| `assets` | [audit-scope-template.md](skills/04-audit/assets/audit-scope-template.md) | - | +| `assets` | [audit-synthesis-template.md](skills/04-audit/assets/audit-synthesis-template.md) | - | +| `assets` | [audit-template.md](skills/04-audit/assets/audit-template.md) | - | +| `references` | [audit-contract.md](skills/04-audit/references/audit-contract.md) | `Runtime inputs, source policy, invariants, and completion rules for question-led audits.` | +| `references` | [question-packs.md](skills/04-audit/references/question-packs.md) | `High-leverage questions and evidence targets for the nine core audit pillars.` | +| `references` | [question-protocol.md](skills/04-audit/references/question-protocol.md) | `Converts introspective audit questions into falsifiable, evidence-gated conclusions.` | +| `references` | [report-contract.md](skills/04-audit/references/report-contract.md) | `Ordered audit folder, chapter ownership, finding identity, freshness, and synthesis rules.` | +| `-` | [SKILL.md](skills/04-audit/SKILL.md) | `Audit current code through a nine-pillar 80/20 core covering North Stars, active rules, memory, architecture and decisions, code hotspots, test value, security, performance, and automation. Use for a broad codebase health-check or one named core pillar. Question-led, static-first, read-only, and evidence-gated.` | #### `skills/05-review` diff --git a/plugins/aidd-dev/README.md b/plugins/aidd-dev/README.md index c214a1cc0..a5b302f38 100644 --- a/plugins/aidd-dev/README.md +++ b/plugins/aidd-dev/README.md @@ -18,7 +18,7 @@ Covers the full SDLC coding loop: orchestrator, planning, implementation, assert | [2.1] | [plan](skills/01-plan/SKILL.md) | Turn a request, ticket, or file into a phased implementation plan, gathering the source first and optionally wireframing a screen before planning. | | [2.2] | [implement](skills/02-implement/SKILL.md) | Execute an implementation plan phase by phase, recipe-style, iterating until 100% completeness. | | [2.3] | [assert](skills/03-assert/SKILL.md) | Assert features work as intended - general assertions, architecture conformance, and frontend UI validation. | -| [2.4] | [audit](skills/04-audit/SKILL.md) | Perform deep codebase analysis to identify technical debt, dead code, and improvement opportunities. | +| [2.4] | [audit](skills/04-audit/SKILL.md) | Run a question-led, static-first nine-pillar core audit of current code against North Stars, active rules, memory, architectural decisions, test value, and high-leverage risks. | | [2.5] | [review](skills/05-review/SKILL.md) | Review a diff along three axes: code quality, feature behavior against the plan, and relevancy (fit to the need, declared-rule conformance, no rot). | | [2.6] | [test](skills/06-test/SKILL.md) | Write and iterate on tests until they pass, and validate user journeys end-to-end in the browser. | | [2.7] | [refactor](skills/07-refactor/SKILL.md) | Optimize code for performance and fix security vulnerabilities following OWASP guidelines. | diff --git a/plugins/aidd-dev/skills/04-audit/SKILL.md b/plugins/aidd-dev/skills/04-audit/SKILL.md index e38b7dc64..550e9c55d 100644 --- a/plugins/aidd-dev/skills/04-audit/SKILL.md +++ b/plugins/aidd-dev/skills/04-audit/SKILL.md @@ -1,37 +1,59 @@ --- name: 04-audit -description: Audit a codebase read-only across seven quality pillars into one ranked report. Use when the user wants to assess, health-check, or audit a codebase or one pillar. Not for fixing findings, reviewing a change, or checking a feature works. -argument-hint: code-quality | architecture | security | dependencies | performance | tests | ui +description: Audit current code through a nine-pillar 80/20 core covering North Stars, active rules, memory, architecture and decisions, code hotspots, test value, security, performance, and automation. Use for a broad codebase health-check or one named core pillar. Question-led, static-first, read-only, and evidence-gated. +argument-hint: core | code-quality | architecture | security | performance | tests | north-star | rules-and-principles | memory-and-documentation | decisions | automation model: opus --- # Skill: audit -Diagnose a codebase against quality pillars and emit one ranked findings report. Read-only: it identifies and ranks problems, never changes code. +Find the few divergences that matter. Questions expose choices and blind spots; evidence decides whether they become findings. The skill writes audit reports and never changes application code. -## Actions +## Default profile -| # | Action | Pillar | Lens | -| --- | -------------- | ------------ | -------------------------------------------------------------------- | -| 01 | `code-quality` | code-quality | Clean code (naming, SOLID, DRY, readability, smells) and tech debt (dead code, complexity, file/function size, error handling) | -| 02 | `architecture` | architecture | Conformance to C4 / ADRs, coupling, boundaries, layering | -| 03 | `security` | security | OWASP risks, authz, input validation, secrets in code | -| 04 | `dependencies` | dependencies | CVEs, licenses, outdated and unused deps, supply chain | -| 05 | `performance` | performance | N+1 queries, hot paths, bundle size, heavy operations | -| 06 | `tests` | tests | Critical-path coverage, flakiness, test pyramid balance | -| 07 | `ui` | ui | Loading/error/empty states, visual hierarchy, design-system drift, responsive, a11y | +`core` is the 80/20 audit. Read `actions/00-core.md`, then the action files mapped below. It writes one numbered chapter per pillar, capped at five high-impact findings except `rules`, which evaluates every active rule. -Run the one pillar named, or offer all seven when the request is unscoped. -Before running an action, read its file in `actions/`, not only the table or assets. +| File | Pillar | Read | +| --- | --- | --- | +| `02-north-star.md` | Current product intent versus code | `actions/08-north-star.md` | +| `03-rules.md` | Active rules, letter and spirit | `actions/09-rules-and-principles.md` | +| `04-memory.md` | Maintained memory versus current usage | `actions/10-memory-and-documentation.md` | +| `05-architecture-and-decisions.md` | Boundaries, choices, divergence, generality | `actions/02-architecture.md`, `actions/11-decisions.md` | +| `06-code-hotspots.md` | Oversized, complex, duplicated, dead, risky code | `actions/01-code-quality.md` | +| `07-tests-value.md` | Risk protection, signal, redundancy, flake, cost | `actions/06-tests.md` | +| `08-security-and-data.md` | Auth, privacy, integrity, unsafe boundaries | `actions/03-security.md` | +| `09-performance-and-reliability.md` | Slow paths, heavy work, failure modes, observability | `actions/05-performance.md` | +| `10-automation-and-knowledge-infrastructure.md` | Recurring work and permanent guardrails | `actions/12-automation.md` | -## Transversal rules +Before running a pillar, read every action file named by its row. Before any run, read: -- Read-only: diagnose and rank, never edit code. -- Scope: run the one named pillar, or for an unscoped request ask once "all seven pillars, or one?" before running. Never silently default to one pillar, never blind-run all without offering the choice. -- One folder per run, `aidd_docs/tasks//_audit/`, like a feature folder. Every pillar that runs always writes its own `.md` there, alone or in a full run. A full run additionally writes a merged `report.md`: one Findings table (category = pillar, severity-first), one Top-actions list, and one Coverage section over all seven pillars. -- Unscannable pillar: skip it, record it under `Coverage > Skipped` with the reason, and never invent findings for it. -- Every finding row carries a severity, its pillar, a concrete `file:line`, the issue, a suggested fix, and an effort. +- `references/audit-contract.md` +- `references/question-protocol.md` +- `references/report-contract.md` +- the relevant section of `references/question-packs.md` -## Assets +## Routing -- `assets/audit-template.md`: the report structure both run modes fill. +1. A named pillar runs only that pillar. +2. `core`, an unscoped audit, or a broad health-check runs all nine core pillars. +3. A custom scope or question pack must map to one or more core pillars and records the customisation in `01-scope-and-system-map.md`. +4. The recipe never spawns agents. A caller may run chapters in isolated agents and later populate the same report package. + +## Output + +Write or refresh: + +`aidd_docs/tasks//_audit_/` + +A full run uses `00-summary.md` through `14-coverage-and-unknowns.md` per `references/report-contract.md`. A single-pillar run always writes `01-scope-and-system-map.md`, its numbered pillar file, and `14-coverage-and-unknowns.md`. + +## Non-negotiable rules + +- Current truth only: code, configuration, North Stars, active rules, `aidd_docs/memory/**`, and explicitly current architecture sources. +- Ignore `aidd_docs/tasks/**`, old plans, old reviews, and historical documentation unless the user declares one normative. +- Static-first. Never run a general E2E journey or launch the site as normal audit work. +- One bounded, read-only runtime probe is allowed only for a plausible critical defect that static evidence cannot settle; record the escalation reason. +- No finding from sentiment, naming, coverage percentage, or model recollection alone. +- No cosmetic nits. Prefer five consequential findings to fifty observations. +- Report uncertainty. Missing evidence produces `unknown`, not confidence theater. +- Recurring problems must consider a permanent automation or knowledge-infrastructure response. diff --git a/plugins/aidd-dev/skills/04-audit/actions/00-core.md b/plugins/aidd-dev/skills/04-audit/actions/00-core.md new file mode 100644 index 000000000..ab095a7a6 --- /dev/null +++ b/plugins/aidd-dev/skills/04-audit/actions/00-core.md @@ -0,0 +1,31 @@ +# 00 - Core audit + +Run the complete static-first 80/20 audit and assemble the numbered report package. Read-only outside the report folder. + +## Input + +Optional target, scope, North Star sources, rule sources, memory path, architecture sources, and budget. Defaults come from `../references/audit-contract.md`. + +## Output + +`aidd_docs/tasks//_audit_/`, containing `00-summary.md` through `14-coverage-and-unknowns.md`. + +## Process + +1. **Contract.** Read every audit reference and the applicable templates before scanning. +2. **Frame.** Resolve runtime inputs and exclusions into `01-scope-and-system-map.md`. +3. **Investigate.** Run the nine core pillars in the router table and write only their owned chapters. +4. **Challenge.** Recheck every candidate against criterion, observation, interpretation, impact, likelihood, reach, confidence, reproduction, and falsification. Write `11-challenge-ledger.md`. In one context set `independence: none`. +5. **Cluster.** Link supported cross-pillar causes in `12-systemic-findings.md`. Similar wording without a shared cause stays separate. +6. **Prioritise.** Write `13-prioritized-actions.md`, ranking cause-level fixes and automation by impact, likelihood, reach, confidence, and effort. +7. **Close coverage.** Write `14-coverage-and-unknowns.md` with scanned, skipped, stale, inaccessible, and unresolved surfaces. +8. **Summarise.** Write `00-summary.md` last from canonical finding IDs. Never copy full finding prose. +9. **Validate.** Check each document against its section contract in `../assets/audit-validator.yml`; remove placeholders and unsupported claims. + +## Test + +- `00` through `14` exist, or a skipped chapter exists with its reason. +- Every finding exists canonically in one pillar chapter. +- Summary, challenge, systemic, and action chapters link IDs instead of duplicating findings. +- `11-challenge-ledger.md` states the achieved independence honestly. +- No file outside the audit report folder changed. diff --git a/plugins/aidd-dev/skills/04-audit/actions/01-code-quality.md b/plugins/aidd-dev/skills/04-audit/actions/01-code-quality.md index 3dcc5e3eb..0bc32c36e 100644 --- a/plugins/aidd-dev/skills/04-audit/actions/01-code-quality.md +++ b/plugins/aidd-dev/skills/04-audit/actions/01-code-quality.md @@ -1,28 +1,33 @@ -# 01 - Code-quality audit +# 01 - Code hotspots -Read-only audit of the `code-quality` pillar, clean-code craftsmanship and tech debt. Reports findings, never edits code. +Find the few code structures that impose disproportionate correctness or change cost. Read-only. ## Input -An optional scope, a directory or file glob. Defaults to the entire codebase. +Target, optional scope, and resolved sources from `01-scope-and-system-map.md`. ## Output -The `code-quality` findings, written to `code-quality.md` in the run's audit folder. +`06-code-hotspots.md`, following `../assets/audit-template.md`. + +## Questions + +- Which files or functions are abnormal relative to project siblings in size, complexity, churn, dependencies, or responsibility? +- Which duplicated paths, dead branches, vestigial flags, or swallowed errors create material risk? +- Which abstraction makes a common change touch unrelated areas? +- What feels clever rather than clear, and does evidence show a consequence? ## Process -1. **Scope.** Default to the full codebase when no scope is given. Otherwise restrict scanning to the provided glob or directory. -2. **Scan.** Cover the two lenses below, using the project's conventions and coding rules already in context. Stay in this pillar: coupling belongs to `02-architecture`, runtime cost to `05-performance`, coverage to `06-tests`, CVEs to `04-dependencies`. - - **Clean code**: naming clarity, single-responsibility and SOLID, DRY (copy-pasted logic, re-implemented stdlib helpers), readability, abstraction level, magic numbers, dead or misleading comments, code smells. - - **Tech debt**: dead and unreachable code, unused exports, types, and helpers, stale TODOs, vestigial flags, cyclomatic complexity and file, function, or component length above project thresholds, nesting depth, error handling caught at the wrong boundary or silently swallowed. - - Use a dedicated tool when available, for example an unused-export finder. Never assert dead code without evidence. -3. **Rate.** Give each finding a severity and an effort per the `@../assets/audit-template.md` legend, with a concrete `file:line`. The category is always `code-quality`. -4. **Write.** Fill `@../assets/audit-template.md` into the pillar file: the Findings table (one row per issue, severity-first), the ranked Top actions, and the Coverage section. In a full run, also add the rows to the merged `report.md` in the same folder. Emit the report and stop. +1. Read `../references/audit-contract.md`, `../references/question-protocol.md`, and the Code hotspots pack in `../references/question-packs.md`. +2. Build a cheap hotspot map with repository search and available static tools. Prefer relative outliers over universal line-count thresholds. +3. Inspect call sites, imports, error paths, and current change surfaces for the strongest leads. +4. Falsify each lead: a large generated table or cohesive module is not a finding merely because it is large. +5. Report at most five verified root problems. Architecture coupling belongs to `02`; runtime cost belongs to `05`. ## Test -- The output file exists at the reported path. -- It has the `## Findings`, `## Top actions`, and `## Coverage` sections. -- Every Findings row carries a severity, category `code-quality`, a concrete `file:line`, and an effort. -- Coverage lists `code-quality` as scanned, and no code was changed. +- Every finding proves a material consequence beyond file size or personal style. +- No dead code is asserted without call-site or tooling evidence. +- `06-code-hotspots.md` has the required sections and at most five findings. +- Application files are unchanged. diff --git a/plugins/aidd-dev/skills/04-audit/actions/02-architecture.md b/plugins/aidd-dev/skills/04-audit/actions/02-architecture.md index 69baec047..3c19fd0f1 100644 --- a/plugins/aidd-dev/skills/04-audit/actions/02-architecture.md +++ b/plugins/aidd-dev/skills/04-audit/actions/02-architecture.md @@ -1,29 +1,34 @@ -# 02 - Architecture audit +# 02 - Architecture -Read-only audit of the `architecture` pillar, conformance to documented architecture, module coupling, and layer or boundary violations. Reports findings, never edits code. +Audit current boundaries, dependency direction, ownership, and decision quality. Read-only. ## Input -An optional scope, a directory or file glob. Defaults to the entire codebase. +Current architecture sources, memory, code structure, and the decision analysis from `11-decisions.md` when this pillar is composed. ## Output -The `architecture` findings, written to `architecture.md` in the run's audit folder. +`05-architecture-and-decisions.md`, following `../assets/audit-template.md` with a `## Decision register`. + +## Questions + +- Where does code diverge from an explicitly current boundary or dependency direction? +- Which module owns unrelated responsibilities or leaks internal details? +- Which choice solves one observed case but leaves the problem class intact? +- Which consequential choice is inferred, weakly supported, or hard to reverse? ## Process -1. **Scope.** Default to the full codebase when no scope is given. Otherwise restrict scanning to the provided glob or directory. -2. **Scan.** Read the architecture documents already in context (`aidd_docs/memory`, ADRs, C4 diagrams). Stay in this pillar: intra-file craftsmanship belongs to `01-code-quality`, runtime cost to `05-performance`, CVEs to `04-dependencies`. - - **Conformance**: map the actual code structure against the documented modules, layers, and C4 boundaries. Flag any divergence from the stated architecture. - - **Coupling**: identify modules that import from layers they should not depend on, a wrong dependency direction or a circular reference across bounded contexts. - - **God-modules**: detect modules with an abnormally large surface area, too many exports or responsibilities, that signal architectural erosion. - - When ADRs or C4 diagrams are absent, note "no architecture docs found, conformance check skipped" in `Coverage > Skipped` and limit the scan to observable coupling heuristics. -3. **Rate.** Give each finding a severity and an effort per the `@../assets/audit-template.md` legend, with a concrete `file:line`. The category is always `architecture`. -4. **Write.** Fill `@../assets/audit-template.md` into the pillar file: the Findings table (one row per issue, severity-first), the ranked Top actions, and the Coverage section. In a full run, also add the rows to the merged `report.md` in the same folder. Emit the report and stop. +1. Read the audit contract, question protocol, and Architecture and decisions pack. +2. Use only explicitly current architecture records. Ignore old plans and historical task reports. +3. Map observable modules, imports, public surfaces, data flow, and ownership. +4. Compare declared intent with implementation. When no current declaration exists, audit observable coupling without inventing a desired architecture. +5. Apply the generality gate and inspect contradictory evidence. +6. Group symptoms by architectural cause and report at most five findings. ## Test -- The output file exists at the reported path. -- It has the `## Findings`, `## Top actions`, and `## Coverage` sections. -- Every Findings row carries a severity, category `architecture`, a concrete `file:line`, and an effort. -- Coverage lists `architecture` as scanned, and no code was changed. +- Every conformance finding cites both intended boundary and implementation evidence. +- Inferred intent is labelled and cannot produce a conformance finding by itself. +- The decision register records provenance, confidence, generality, reversibility, and evidence. +- `05-architecture-and-decisions.md` contains at most five root findings. diff --git a/plugins/aidd-dev/skills/04-audit/actions/03-security.md b/plugins/aidd-dev/skills/04-audit/actions/03-security.md index 4935fa618..4fff0f16b 100644 --- a/plugins/aidd-dev/skills/04-audit/actions/03-security.md +++ b/plugins/aidd-dev/skills/04-audit/actions/03-security.md @@ -1,32 +1,35 @@ -# 03 - Security audit +# 03 - Security and data -Read-only audit of the `security` pillar, OWASP top risks and code-level security weaknesses. Reports findings, never edits code. +Find material trust-boundary, authorization, privacy, and data-integrity failures. Read-only. ## Input -An optional scope, a directory or file glob. Defaults to the entire codebase. +Target code, configuration, schemas, persistence paths, manifests, lockfiles, existing scanner output, and declared trust assumptions. ## Output -The `security` findings, written to `security.md` in the run's audit folder. +`08-security-and-data.md`, following `../assets/audit-template.md`. + +## Questions + +- Where does untrusted data cross a boundary without validation, sanitisation, or authorization? +- Which path can disclose, corrupt, overwrite, or silently lose data? +- Which security property exists only by convention? +- Which default broadens access or weakens isolation? +- Which direct dependency creates a material, evidenced trust or supply-chain risk? ## Process -1. **Scope.** Default to the full codebase when no scope is given. Otherwise restrict scanning to the provided glob or directory. -2. **Scan.** Use static code analysis. Stay in this pillar: known-CVE and vulnerable-dependency findings belong to `04-dependencies`, coupling to `02-architecture`. - - **Input validation at trust boundaries**: check that every external input (HTTP requests, env vars, file paths, user-supplied data) is validated or sanitised before use. - - **Authn/authz gates**: verify authentication and authorisation checks are enforced consistently at every protected route or operation. - - **Secrets in code**: flag hardcoded credentials, API keys, tokens, or passwords anywhere in the scanned files. - - **Injection risks**: SQL, command, XSS, LDAP, or template injection. Identify concatenated or unescaped values passed to interpreters. - - **Unsafe deserialization**: flag `eval`, unsafe YAML/pickle/JSON reviver patterns, or object deserialization from untrusted sources. - - **Insecure defaults**: missing TLS enforcement, overly permissive CORS, disabled security headers, debug flags left on in non-dev code. - - Use a static-analysis tool when available. Flag only findings supported by code evidence, never inferred from naming alone. -3. **Rate.** Give each finding a severity and an effort per the `@../assets/audit-template.md` legend, with a concrete `file:line`. The category is always `security`. -4. **Write.** Fill `@../assets/audit-template.md` into the pillar file: the Findings table (one row per issue, severity-first), the ranked Top actions, and the Coverage section. In a full run, also add the rows to the merged `report.md` in the same folder. Emit the report and stop. +1. Read the audit contract, question protocol, and Security and data pack. +2. Trace external inputs through validation, authorization, persistence, serialization, and output. +3. Inspect secrets, injection sinks, unsafe deserialization, CORS/TLS/security defaults, and destructive operations where relevant. +4. Use existing static-security or dependency-scanner output when available. Do not perform a general dependency inventory or guess current CVEs, versions, provenance, or licences. +5. Attempt to falsify exploitability and impact. Names or theoretical patterns alone remain unknown. +6. Report at most five highest-risk findings. ## Test -- The output file exists at the reported path. -- It has the `## Findings`, `## Top actions`, and `## Coverage` sections. -- Every Findings row carries a severity, category `security`, a concrete `file:line`, and an effort. -- Coverage lists `security` as scanned, and no code was changed. +- Every finding names the trust boundary, evidence, plausible consequence, and contradictory evidence checked. +- No secret value is copied into the report; cite only its location and kind. +- Dependency findings cite authoritative current evidence and a material affected path. +- Application files are unchanged. diff --git a/plugins/aidd-dev/skills/04-audit/actions/04-dependencies.md b/plugins/aidd-dev/skills/04-audit/actions/04-dependencies.md deleted file mode 100644 index 50f3128f9..000000000 --- a/plugins/aidd-dev/skills/04-audit/actions/04-dependencies.md +++ /dev/null @@ -1,30 +0,0 @@ -# 04 - Dependencies audit - -Read-only audit of the `dependencies` pillar, CVEs, license compliance, outdated packages, and supply-chain integrity. Reports findings, never edits code. - -## Input - -An optional scope, a directory or file glob. Defaults to the entire codebase. - -## Output - -The `dependencies` findings, written to `dependencies.md` in the run's audit folder. - -## Process - -1. **Scope.** Default to the full codebase when no scope is given. Otherwise restrict scanning to the provided glob or directory. -2. **Scan.** Use the appropriate dependency scanner. Stay in this pillar: application-code security belongs to `03-security`, runtime cost to `05-performance`. - - **Known CVEs**: run `npm audit`, `pip-audit`, `cargo audit`, or the equivalent for the project's package manager. When no scanner or lockfile is present, record "scanner absent" or "no lockfile found" in `Coverage > Skipped` and do not guess CVEs. - - **License compliance**: check declared licenses against the project's accepted-license list. Flag GPL/AGPL or unknown licenses in a commercial codebase. - - **Outdated packages**: identify packages significantly behind their latest stable release, especially those with security-relevant changelogs. - - **Unused declared dependencies**: flag packages listed in the manifest with no import found in the scanned source. - - **Lockfile integrity and supply chain**: verify the lockfile is present and committed. Flag direct git or URL dependencies and any package with no integrity hash. -3. **Rate.** Give each finding a severity and an effort per the `@../assets/audit-template.md` legend, with a concrete `file:line` in the manifest or lockfile. The category is always `dependencies`. -4. **Write.** Fill `@../assets/audit-template.md` into the pillar file: the Findings table (one row per issue, severity-first), the ranked Top actions, and the Coverage section. In a full run, also add the rows to the merged `report.md` in the same folder. Emit the report and stop. - -## Test - -- The output file exists at the reported path. -- It has the `## Findings`, `## Top actions`, and `## Coverage` sections. -- Every Findings row carries a severity, category `dependencies`, a concrete `file:line`, and an effort. -- Coverage lists `dependencies` as scanned, and no code was changed. diff --git a/plugins/aidd-dev/skills/04-audit/actions/05-performance.md b/plugins/aidd-dev/skills/04-audit/actions/05-performance.md index e19e1aed1..49c767371 100644 --- a/plugins/aidd-dev/skills/04-audit/actions/05-performance.md +++ b/plugins/aidd-dev/skills/04-audit/actions/05-performance.md @@ -1,32 +1,34 @@ -# 05 - Performance audit +# 05 - Performance and reliability -Read-only audit of the `performance` pillar, runtime cost, query patterns, and rendering efficiency. Reports findings, never edits code. +Find material slow paths, amplification, silent failure, and missing recovery without running a general E2E flow. Read-only. ## Input -An optional scope, a directory or file glob. Defaults to the entire codebase. +Target code, current budgets when defined, and existing profiler, build, bundle, query, CI, or telemetry artifacts. ## Output -The `performance` findings, written to `performance.md` in the run's audit folder. +`09-performance-and-reliability.md`, following `../assets/audit-template.md`. + +## Questions + +- Which path repeats I/O, loads unbounded data, or performs avoidable heavy work? +- Which file, asset, dependency, or initialization path materially affects build, startup, or runtime? +- Which failure becomes worse through retries, fan-out, missing timeouts, or absent backpressure? +- Which important failure is invisible or unrecoverable? ## Process -1. **Scope.** Default to the full codebase when no scope is given. Otherwise restrict scanning to the provided glob or directory. -2. **Scan.** Prefer runtime tools (profiler, bundle analyzer, query explain) when available. Stay in this pillar: cyclomatic complexity belongs to `01-code-quality`, coupling to `02-architecture`. - - **N+1 queries**: detect loops that issue a database or network call on each iteration without batching. - - **Unbatched heavy operations**: flag heavy computations or I/O repeated individually where a batch or bulk API exists. - - **Unpaginated large payloads**: identify endpoints or queries that fetch unbounded result sets without a limit or pagination. - - **Bundle size**: use a bundle analyzer when available. Flag large or duplicated dependencies that inflate the JS or CSS payload. - - **Render thrash**: detect layout-thrashing DOM patterns, missing memoization on computed values used in hot render paths, or component trees that re-render without a guard on reference-stable props. - - **Missing memoization on hot paths**: flag expensive pure computations inside render or tight loops that are not memoized. - - When no profiler or bundle analyzer is available, degrade to static heuristics and record "no profiler, static heuristics only" in `Coverage > Skipped`. Never assert a runtime bottleneck without evidence. -3. **Rate.** Give each finding a severity and an effort per the `@../assets/audit-template.md` legend, with a concrete `file:line`. The category is always `performance`. -4. **Write.** Fill `@../assets/audit-template.md` into the pillar file: the Findings table (one row per issue, severity-first), the ranked Top actions, and the Coverage section. In a full run, also add the rows to the merged `report.md` in the same folder. Emit the report and stop. +1. Read the audit contract, question protocol, and Performance and reliability pack. +2. Inspect static call paths and existing measurements first: N+1 patterns, unbounded reads, repeated serialization, synchronous heavy work, large bundles, retry loops, timeouts, and recovery. +3. Prefer a stated budget or existing measurement. A clear complexity argument may support a finding; otherwise record a hypothesis or unknown. +4. Do not launch the site, generate a new profile, or run E2E by default. +5. A bounded runtime probe must pass the critical escalation gate in `audit-contract.md`. +6. Report at most five material findings. ## Test -- The output file exists at the reported path. -- It has the `## Findings`, `## Top actions`, and `## Coverage` sections. -- Every Findings row carries a severity, category `performance`, a concrete `file:line`, and an effort. -- Coverage lists `performance` as scanned, and no code was changed. +- Every bottleneck finding cites measurement or a decisive complexity path. +- “Large” is compared with a project budget, artifact, or meaningful consequence. +- Reliability findings name the failure mode and missing detection or recovery. +- No general runtime or E2E flow runs. diff --git a/plugins/aidd-dev/skills/04-audit/actions/06-tests.md b/plugins/aidd-dev/skills/04-audit/actions/06-tests.md index 36b42e1d9..fc441fd89 100644 --- a/plugins/aidd-dev/skills/04-audit/actions/06-tests.md +++ b/plugins/aidd-dev/skills/04-audit/actions/06-tests.md @@ -1,31 +1,37 @@ -# 06 - Tests audit +# 06 - Test value -Read-only audit of the `tests` pillar, coverage gaps, test quality, and suite health. Reports findings, never edits code. +Audit whether tests protect meaningful risks at a proportionate cost. Coverage percentage is not the goal. Read-only. ## Input -An optional scope, a directory or file glob. Defaults to the entire codebase. +Test code, production behavior, fixtures, mocks, test configuration, CI history, and existing coverage or timing artifacts. ## Output -The `tests` findings, written to `tests.md` in the run's audit folder. +`07-tests-value.md`, following `../assets/audit-template.md`. + +## Questions + +- Which user or system risk does each important test group protect? +- What plausible regression would make it fail? +- Which tests can stay green while the claimed behavior is broken? +- Which tests duplicate cheaper protection or cost more than their unique signal? +- Which critical behavior has no effective witness? ## Process -1. **Scope.** Default to the full codebase when no scope is given. Otherwise restrict scanning to the provided glob or directory. -2. **Scan.** Prefer a coverage report when available. Stay in this pillar: whether a feature behaves correctly is a separate concern, runtime cost belongs to `05-performance`. - - **Critical-path coverage gaps**: identify code paths (auth flows, data mutations, error handling) with no test. Use a coverage report when available, degrade to static inspection of test-file presence when absent. - - **Tests asserting implementation, not behavior**: flag tests coupled to internal method names, private state, or implementation details rather than observable outputs. - - **Flaky tests**: flag tests that use arbitrary `sleep` calls, rely on timing, or have known intermittent failures recorded in CI history or inline comments. - - **Skipped tests without a reason**: flag `skip`, `xit`, `xfail`, `.todo`, or equivalent markers that lack an explanatory comment or issue reference. - - **Test pyramid imbalance**: flag suites with disproportionately many end-to-end or integration tests and few unit tests, raising maintenance cost and slowing feedback. - - When no coverage tool is available, record "no coverage tool, static inspection only" in `Coverage > Skipped` and limit findings to structurally observable issues. Do not invent coverage numbers. -3. **Rate.** Give each finding a severity and an effort per the `@../assets/audit-template.md` legend, with a concrete `file:line`. The category is always `tests`. -4. **Write.** Fill `@../assets/audit-template.md` into the pillar file: the Findings table (one row per issue, severity-first), the ranked Top actions, and the Coverage section. In a full run, also add the rows to the merged `report.md` in the same folder. Emit the report and stop. +1. Read the audit contract, question protocol, and Test value pack. +2. Start from critical code paths and risks, then map the tests that claim to protect them. +3. Inspect assertions, mocks, fixtures, timing dependencies, skips, snapshots, and overlap. Do not classify from filenames alone. +4. Classify material groups as `protective`, `redundant`, `brittle`, `ceremonial`, or `misleading`. +5. Compare signal with runtime, flake, fixture complexity, duplication, and maintenance cost. +6. Recommend `delete` only when no distinct protected risk remains beyond a cheaper test. Otherwise prefer `keep`, `rewrite`, `merge`, or `unknown`. +7. Do not run E2E. Use existing CI, coverage, timing, and flake evidence; do not generate new coverage by default. +8. Report at most five highest-leverage findings. ## Test -- The output file exists at the reported path. -- It has the `## Findings`, `## Top actions`, and `## Coverage` sections. -- Every Findings row carries a severity, category `tests`, a concrete `file:line`, and an effort. -- Coverage lists `tests` as scanned, and no code was changed. +- Every finding names protected risk, plausible regression, signal failure, and cost. +- Coverage percentage alone never produces a finding. +- A delete recommendation proves the absence of unique protection. +- Critical untested behavior cites the production path and missing effective witness. diff --git a/plugins/aidd-dev/skills/04-audit/actions/07-ui.md b/plugins/aidd-dev/skills/04-audit/actions/07-ui.md deleted file mode 100644 index 3f11a649d..000000000 --- a/plugins/aidd-dev/skills/04-audit/actions/07-ui.md +++ /dev/null @@ -1,31 +0,0 @@ -# 07 - UI audit - -Read-only audit of the `ui` pillar, missing states, design-system drift, responsive gaps, and accessibility. Reports findings, never edits code. - -## Input - -An optional scope, a directory or file glob, defaulting to the entire codebase. An optional running-frontend URL enables a runtime axe accessibility pass. - -## Output - -The `ui` findings, written to `ui.md` in the run's audit folder. - -## Process - -1. **Scope.** Default to the full codebase when no scope is given. Otherwise restrict scanning to the provided glob or directory. When a URL is given, also run a browser-based accessibility pass against the live frontend. -2. **Scan.** Use static component and markup analysis, plus a runtime axe-style accessibility pass when a URL is given. Stay in this pillar: redesign and implementation fixes are a separate concern. - - **Missing loading, error, and empty states**: flag UI surfaces (lists, forms, async data areas) with no visible feedback for in-progress loading, failed fetches, or zero-result datasets. - - **Visual hierarchy**: flag layouts where primary actions are visually subordinate, or where typographic hierarchy (heading levels, heading-to-body contrast) is broken. - - **Design-token drift**: identify hardcoded colour, spacing, or typography values that deviate from the project's design tokens or component library. - - **Responsive breakpoint gaps**: flag components or layouts that break or overflow at standard breakpoints (mobile, tablet, desktop), detectable from markup or media queries. - - **Accessibility (WCAG)**: contrast-ratio failures, missing keyboard navigation, incorrect or absent aria roles and labels, missing `alt` text. Run axe or equivalent against the URL when given, otherwise inspect markup statically. - - When no URL is given, record "no url provided, runtime a11y pass skipped, static inspection only" in `Coverage > Skipped`. Do not invent runtime findings from static analysis. -3. **Rate.** Give each finding a severity and an effort per the `@../assets/audit-template.md` legend, with a concrete `file:line`. The category is always `ui`. -4. **Write.** Fill `@../assets/audit-template.md` into the pillar file: the Findings table (one row per issue, severity-first), the ranked Top actions, and the Coverage section. In a full run, also add the rows to the merged `report.md` in the same folder. Emit the report and stop. - -## Test - -- The output file exists at the reported path. -- It has the `## Findings`, `## Top actions`, and `## Coverage` sections. -- Every Findings row carries a severity, category `ui`, a concrete `file:line`, and an effort. -- Coverage lists `ui` as scanned, and no code was changed. diff --git a/plugins/aidd-dev/skills/04-audit/actions/08-north-star.md b/plugins/aidd-dev/skills/04-audit/actions/08-north-star.md new file mode 100644 index 000000000..acc8b4207 --- /dev/null +++ b/plugins/aidd-dev/skills/04-audit/actions/08-north-star.md @@ -0,0 +1,34 @@ +# 08 - North Star + +Audit whether current code serves the user-confirmed product intent and critical outcomes. Read-only. + +## Input + +Explicit North Star sources, or candidates discovered in the current repository. + +## Output + +`02-north-star.md`, following `../assets/audit-template.md`. + +## Questions + +- Which critical outcome does the implementation serve differently from the North Star? +- Which shipped behavior contradicts, obscures, or omits a primary outcome? +- Which technical choice optimises a local concern at the expense of product intent? +- Where can the application appear successful while failing the intended outcome? + +## Process + +1. Read the audit contract, question protocol, and North Star pack. +2. Resolve explicit sources first. Otherwise discover candidates, show ambiguity, and never silently canonise one. +3. If no current North Star resolves, write the chapter as `skipped` and continue other pillars. +4. Extract only material outcomes and constraints, not every sentence. +5. Trace them into routes, commands, schemas, public APIs, and critical code paths. +6. Report at most five verified divergences. + +## Test + +- Every finding cites both a confirmed North Star statement and implementation evidence. +- Missing or conflicting sources produce skipped, unknown, or disputed status rather than invented intent. +- Old plans are ignored unless explicitly declared normative. +- `02-north-star.md` contains no cosmetic product suggestions. diff --git a/plugins/aidd-dev/skills/04-audit/actions/09-rules-and-principles.md b/plugins/aidd-dev/skills/04-audit/actions/09-rules-and-principles.md new file mode 100644 index 000000000..3060507ee --- /dev/null +++ b/plugins/aidd-dev/skills/04-audit/actions/09-rules-and-principles.md @@ -0,0 +1,38 @@ +# 09 - Rules and principles + +Evaluate every active rule against current code and maintained project state. Read-only. + +## Input + +Active host instructions, scoped repository rules, user-supplied principles, and their resolved precedence. + +## Output + +`03-rules.md`, following `../assets/audit-template.md` with a `## Rule control matrix`. + +| Control ID | Rule | Source | Scope | Precedence | Letter | Spirit | Status | Evidence | +| --- | --- | --- | --- | --- | --- | --- | --- | --- | +| `RULE-C001` | Short normative statement | `path:line` | Evaluated surface | Resolved priority | pass / partial / fail / unknown / disputed | pass / partial / fail / unknown / disputed | Overall status | Implementation references | + +## Questions + +- Which active rule is violated by code, configuration, tests, or maintained memory? +- Which rule is followed literally but violated in spirit? +- Which rules conflict, are impossible to evaluate, or have ambiguous scope? +- Which repeated violation should become executable enforcement? + +## Process + +1. Read the audit contract, question protocol, and Rules pack. +2. Inventory every active rule with source, scope, precedence, and a stable control ID. +3. Mark each `pass`, `partial`, `fail`, `unknown`, or `disputed`, citing implementation evidence. +4. Group repeated violations of one rule into one root finding, but keep every violating location linked in the control matrix. +5. Report every active rule in the matrix. The five-finding cap does not hide material rule failures; prioritise the five root causes in the Findings section and link remaining failed controls. +6. Suggest automation when a rule can be enforced more reliably by type, lint, test, CI, hook, or skill. + +## Test + +- Every active rule appears exactly once in the control matrix. +- Every fail cites the exact rule source and concrete implementation evidence. +- Observed convention is never promoted to a rule. +- Conflicting precedence is marked disputed, not resolved by guess. diff --git a/plugins/aidd-dev/skills/04-audit/actions/10-memory-and-documentation.md b/plugins/aidd-dev/skills/04-audit/actions/10-memory-and-documentation.md new file mode 100644 index 000000000..3037ed80c --- /dev/null +++ b/plugins/aidd-dev/skills/04-audit/actions/10-memory-and-documentation.md @@ -0,0 +1,34 @@ +# 10 - Memory + +Audit whether maintained project memory still matches code and current usage. Historical task documentation is out of scope. Read-only. + +## Input + +`aidd_docs/memory/**` or an explicitly supplied memory path, plus current code and configuration. + +## Output + +`04-memory.md`, following `../assets/audit-template.md`. + +## Questions + +- Which memory claim is stale, false, or no longer used? +- Which missing current fact would cause a fresh agent to choose incorrectly? +- Which memory statement duplicates and contradicts a canonical source? +- Which command, dependency, boundary, or workflow named in memory no longer exists? + +## Process + +1. Read the audit contract, question protocol, and Memory pack. +2. If no maintained memory resolves, write the chapter as `skipped`. +3. Prioritise memory that guides architecture, commands, stack, testing, deployment, and critical workflows. +4. Verify each high-impact claim against current code, configuration, manifests, and scripts. +5. Ignore `aidd_docs/tasks/**`, old plans, and historical reviews even when they contradict memory. +6. Report at most five stale or missing facts with the greatest agent-decision impact. + +## Test + +- Every stale-memory finding cites memory and current implementation evidence. +- “Missing memory” explains the wrong decision a fresh agent could make. +- No historical task artifact is treated as current truth. +- The audit never edits memory. diff --git a/plugins/aidd-dev/skills/04-audit/actions/11-decisions.md b/plugins/aidd-dev/skills/04-audit/actions/11-decisions.md new file mode 100644 index 000000000..2c6c997a4 --- /dev/null +++ b/plugins/aidd-dev/skills/04-audit/actions/11-decisions.md @@ -0,0 +1,39 @@ +# 11 - Decisions + +Interrogate material implementation choices, uncertainty, generality, and reversibility. This action composes with `02-architecture`. Read-only. + +## Input + +Decision logs when present, North Stars, active rules, current architecture sources, code, and limited history needed to establish a choice. + +## Output + +The `## Decision register` and decision-related findings in `05-architecture-and-decisions.md`. + +| Decision ID | Choice | Provenance | Confidence | Alternatives | Assumptions | Generality | Reversibility | Evidence | Verdict | +| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | +| `DEC-001` | Material choice | specified / agent-made / retrospectively-inferred | high / medium / low | Credible alternatives | Required assumptions | general / local / unknown | easy / moderate / hard | References | pass / finding / unknown / disputed | + +## Questions + +- Which choices were specified, explicitly agent-made, or retrospectively inferred? +- Which choices were made because the task was underspecified? +- Which choice are you least confident in, and what evidence would change it? +- Which decision fixes the observed example but not the class? +- Which decision is expensive to reverse or broad in reach? + +## Process + +1. Read the audit contract, question protocol, and Architecture and decisions pack. +2. Prefer implementation-time decision logs. Otherwise reconstruct only material choices and label them `retrospectively-inferred`. +3. Record provenance, confidence, alternatives, assumptions, scope, generality, reversibility, affected surfaces, and evidence. +4. Apply the generality gate to constants, thresholds, buffers, special cases, and narrowly placed fixes. +5. Use decision risk to focus code inspection; never let a declared decision suppress contradictory evidence. +6. Feed verified decision findings into the shared architecture chapter. + +## Test + +- Retrospective inference is never presented as original intent. +- Every non-general finding proves the class remains unresolved beyond the observed example. +- High-confidence declarations cannot override contradictory code evidence. +- The decision register contains no hidden chain-of-thought. diff --git a/plugins/aidd-dev/skills/04-audit/actions/12-automation.md b/plugins/aidd-dev/skills/04-audit/actions/12-automation.md new file mode 100644 index 000000000..199e9e570 --- /dev/null +++ b/plugins/aidd-dev/skills/04-audit/actions/12-automation.md @@ -0,0 +1,39 @@ +# 12 - Automation and knowledge infrastructure + +Turn recurring audit findings and tacit knowledge into proportionate permanent guardrails. Read-only. + +## Input + +Confirmed findings from other pillars, active automation, rules, skills, memory, CI, and repository tooling. + +## Output + +`10-automation-and-knowledge-infrastructure.md`, following `../assets/audit-template.md` with `## Automation candidates`. + +| Candidate ID | Finding | Recurrence | Current cost | Enforcement layer | Leverage | False-positive risk | Maintenance cost | Owner | +| --- | --- | --- | --- | --- | --- | --- | --- | --- | +| `AUTO-C001` | Canonical finding ID | Evidenced recurrence | Human, agent, or CI cost | Type, lint, test, CI, context, or runtime | Expected class-level benefit | high / medium / low | high / medium / low | Responsible role | + +## Questions + +- Which issue class already consumes repeated human or agent reasoning? +- Which review rejection indicates missing repository knowledge? +- Could a fresh agent make the right choice with zero extra prompt? +- Which invariant belongs in executable enforcement rather than prose? +- Which guardrail has the best leverage after false-positive and maintenance cost? + +## Process + +1. Read the audit contract, question protocol, and Automation pack. +2. Start from confirmed findings and repeated patterns; do not invent automation for hypothetical preferences. +3. Map each candidate to the strongest proportionate layer: type/schema, lint/static analysis, test, CI, rule/skill/memory/docs, or runtime assertion/telemetry. +4. Record recurrence, current human or token cost, proposed layer, expected leverage, false-positive risk, maintenance cost, and owner. +5. Prefer one class-level guardrail over multiple one-off fixes. +6. Rank at most five automation candidates. + +## Test + +- Every candidate links to a confirmed finding or repeated evidenced pattern. +- One-off preferences do not become permanent machinery. +- Proposed enforcement is placed at the strongest proportionate layer. +- Expected leverage and maintenance or false-positive risk are both stated. diff --git a/plugins/aidd-dev/skills/04-audit/assets/audit-actions-template.md b/plugins/aidd-dev/skills/04-audit/assets/audit-actions-template.md new file mode 100644 index 000000000..844a2296f --- /dev/null +++ b/plugins/aidd-dev/skills/04-audit/assets/audit-actions-template.md @@ -0,0 +1,35 @@ +--- +audit: scope-slug +chapter: 13-prioritized-actions +status: complete +owner: current-context +last_verified: yyyy-mm-dd +sources: + - 11-challenge-ledger.md + - 12-systemic-findings.md +depends_on: + - 11-challenge-ledger.md + - 12-systemic-findings.md +--- + +# 13 - Prioritized actions + +## Action sequence + +### ACT-001 - Action title + +- **Resolves**: Canonical finding and systemic IDs +- **Outcome**: Observable risk or recurring cost removed +- **Why now**: Impact, likelihood, reach, and confidence +- **Approach**: Cause-level direction, not a speculative implementation plan +- **Automation**: Permanent guardrail candidate or `none` +- **Effort**: S | M | L +- **Depends on**: Action ID or `none` + +## Deferred + +- Canonical finding ID and reason it falls outside the 80/20 boundary. + +## Order rationale + +One short explanation of dependencies and leverage. Do not rank by severity alone. diff --git a/plugins/aidd-dev/skills/04-audit/assets/audit-challenge-template.md b/plugins/aidd-dev/skills/04-audit/assets/audit-challenge-template.md new file mode 100644 index 000000000..fc97390ba --- /dev/null +++ b/plugins/aidd-dev/skills/04-audit/assets/audit-challenge-template.md @@ -0,0 +1,36 @@ +--- +audit: scope-slug +chapter: 11-challenge-ledger +status: complete +owner: current-context +last_verified: yyyy-mm-dd +sources: + - 02-10 +depends_on: + - 02-10 +--- + +# 11 - Challenge ledger + +## Independence + +`none`, `fresh-context`, or `multiple-fresh-contexts`. Never imply independent review when the same context challenged its own work. + +## Decisions + +| Candidate | Decision | Canonical ID | Reason | +| --- | --- | --- | --- | +| `ID` | accept, reject, merge, or dispute | `ID` or `none` | Evidence, missing criterion, duplicate cause, or contradiction | + +## Rejected candidates + +- Candidate ID, original concern, and decisive rejection reason. + +## Disputes + +- Candidate ID, competing interpretations, evidence for each, and what would settle them. + +## Coverage + +- Chapters challenged +- Chapters not independently challenged diff --git a/plugins/aidd-dev/skills/04-audit/assets/audit-coverage-template.md b/plugins/aidd-dev/skills/04-audit/assets/audit-coverage-template.md new file mode 100644 index 000000000..66a1c511c --- /dev/null +++ b/plugins/aidd-dev/skills/04-audit/assets/audit-coverage-template.md @@ -0,0 +1,44 @@ +--- +audit: scope-slug +chapter: 14-coverage-and-unknowns +status: complete +owner: current-context +last_verified: yyyy-mm-dd +sources: + - 01-scope-and-system-map.md + - 02-13 +depends_on: + - 01-13 +--- + +# 14 - Coverage and unknowns + +## Scanned + +| Pillar | Sources and surfaces | Status | +| --- | --- | --- | +| Pillar | Concrete paths or artifacts | complete or partial | + +## Skipped + +| Pillar or surface | Reason | Impact | +| --- | --- | --- | +| Name | Missing source, excluded history, inaccessible tool, or outside budget | What could remain unseen | + +## Unknowns + +- Linked unknown, missing evidence, and what would settle it. + +## Freshness + +- Fresh chapters +- Stale dependent chapters + +## Runtime and independence + +- Runtime probes: reason and result, or `none` +- Independence achieved + +## 80/20 boundary + +Why further scanning was unlikely to change the highest-leverage actions. diff --git a/plugins/aidd-dev/skills/04-audit/assets/audit-index-template.md b/plugins/aidd-dev/skills/04-audit/assets/audit-index-template.md new file mode 100644 index 000000000..dcc6678f1 --- /dev/null +++ b/plugins/aidd-dev/skills/04-audit/assets/audit-index-template.md @@ -0,0 +1,40 @@ +--- +audit: scope-slug +chapter: 00-summary +status: complete +owner: current-context +last_verified: yyyy-mm-dd +sources: + - 01-scope-and-system-map.md +depends_on: + - 02-10 + - 11-challenge-ledger.md + - 12-systemic-findings.md + - 13-prioritized-actions.md + - 14-coverage-and-unknowns.md +--- + +# Audit: scope + +## Verdict + +One sentence on current health against the audit's declared sources. State independence level. + +## Highest-leverage findings + +1. `ID` - linked title and consequence. + +## Highest-leverage actions + +1. Linked action from `13-prioritized-actions.md`. + +## Contents + +Ordered links to every written chapter, including skipped chapters. + +## Boundaries + +- Static-first policy +- Excluded historical artifacts +- Missing normative sources +- Finding cap and independence level diff --git a/plugins/aidd-dev/skills/04-audit/assets/audit-scope-template.md b/plugins/aidd-dev/skills/04-audit/assets/audit-scope-template.md new file mode 100644 index 000000000..f47fe8bac --- /dev/null +++ b/plugins/aidd-dev/skills/04-audit/assets/audit-scope-template.md @@ -0,0 +1,49 @@ +--- +audit: scope-slug +chapter: 01-scope-and-system-map +status: complete +owner: current-context +last_verified: yyyy-mm-dd +sources: + - current-repository +depends_on: + - none +--- + +# 01 - Scope and system map + +## Target + +- **Repository**: Resolved target +- **Scope**: Included application surfaces +- **Profile**: core or named pillar + +## Normative sources + +| Kind | Resolved source | Status | Precedence | +| --- | --- | --- | --- | +| North Star | path or none | confirmed, ambiguous, or absent | explicit or unresolved | +| Rules | paths or active host context | confirmed | resolved scope | +| Memory | path or none | present or absent | non-normative knowledge | +| Architecture | paths or none | current, ambiguous, or absent | explicit or unresolved | + +## Exclusions + +- Historical tasks, old plans, past reviews, generated output, vendored dependencies, general dependency inventory, and other excluded surfaces. + +## Budget + +- Pillars, finding cap, stop boundary, available tools, and inaccessible evidence. + +## Runtime policy + +- Static-first +- Runtime escalation reason or `none` + +## Independence + +- `none`, `parallel-shards`, or `parallel-shards-plus-fresh-challenge` + +## System map + +Concise modules, boundaries, critical paths, and data stores needed to understand the audit. No exhaustive file tree. diff --git a/plugins/aidd-dev/skills/04-audit/assets/audit-synthesis-template.md b/plugins/aidd-dev/skills/04-audit/assets/audit-synthesis-template.md new file mode 100644 index 000000000..f17d0f27a --- /dev/null +++ b/plugins/aidd-dev/skills/04-audit/assets/audit-synthesis-template.md @@ -0,0 +1,32 @@ +--- +audit: scope-slug +chapter: 12-systemic-findings +status: complete +owner: current-context +last_verified: yyyy-mm-dd +sources: + - 02-10 +depends_on: + - 11-challenge-ledger.md +--- + +# 12 - Systemic findings + +## Root causes + +### SYS-001 - Root cause + +- **Evidence**: Links to canonical finding IDs from at least two pillars +- **Pattern**: Shared cause, not repeated symptom text +- **Reach**: Affected pillars and surfaces +- **Consequence**: Material compounded impact +- **Confidence**: high | medium +- **Response**: One cause-level action + +## Isolated findings + +- Canonical finding IDs that do not share a supported cause. + +## Rejected clusters + +- Candidate cluster and why the evidence did not support one cause. diff --git a/plugins/aidd-dev/skills/04-audit/assets/audit-template.md b/plugins/aidd-dev/skills/04-audit/assets/audit-template.md index 6566f8ccc..3287b0095 100644 --- a/plugins/aidd-dev/skills/04-audit/assets/audit-template.md +++ b/plugins/aidd-dev/skills/04-audit/assets/audit-template.md @@ -1,49 +1,58 @@ --- -name: audit -description: Codebase audit report template -argument-hint: N/A +audit: scope-slug +chapter: number-and-name +status: complete +owner: current-context +last_verified: yyyy-mm-dd +sources: + - resolved-source +depends_on: + - none --- -# Codebase Audit: {{scope}} +# NN - Pillar name -{{one_line_summary}} +## Verdict -- **Date**: {{yyyy_mm_dd}} -- **Scope**: {{scope}} -- **Health**: {{good | fair | poor}} -- **Findings**: {{n_critical}} critical, {{n_warning}} warning, {{n_minor}} minor - -Health: `good` = no critical findings; `fair` = critical findings exist but are isolated and addressable; `poor` = systemic or widespread critical findings. +One sentence: what matters, why, and how confident the audit is. ## Findings -One row per issue. Every row MUST cite a concrete `file:line`. Sort by severity (critical first). Read-only: an audit reports, it never edits code. +Maximum five, highest impact first. Omit the example when no finding survives the evidence gate. -Severity (shared rubric across every audit pillar, so a full audit ranks consistently): -- 🔴 critical - exploitable security hole, data loss, or broken correctness. Fix now. -- 🟡 warning - real debt or risk that will bite later. Fix soon. -- 🟢 minor - nit or cleanup. Fix when convenient. +### PREFIX-001 - Finding title -Effort: `S` (under 1h), `M` (under 1d), `L` (over 1d). -Category (the audit pillar, one of): `code-quality`, `architecture`, `security`, `dependencies`, `performance`, `tests`, `ui`. +- **Priority**: P0 | P1 | P2 +- **Question**: The question that exposed the concern +- **Criterion**: The North Star, rule, invariant, risk, or expected property +- **Evidence kind**: code | configuration | test | command | artifact | runtime | history | normative +- **Evidence**: `path:line`, command result, existing artifact, or bounded observation +- **Observation**: What the evidence directly establishes +- **Interpretation**: Why the observation diverges from the criterion +- **Impact**: Material user, system, maintenance, or agent-productivity consequence +- **Likelihood**: high | medium | low +- **Reach**: local | module | system | user-base +- **Confidence**: high | medium +- **Reproduction**: Minimal reproduction or `not-applicable` with reason +- **Falsification**: Contradictory evidence checked and result +- **Next action**: Smallest action that addresses the cause +- **Effort**: S | M | L +- **Automation**: Candidate ID or `none` -| Sev | Category | Location | Issue | Suggested fix | Effort | -| --- | ------------ | ------------------------- | -------------------------------------- | ------------------------------------ | ------ | -| 🔴 | security | `src/api/user.ts:30` | Request body used without validation | Validate with the project schema lib | S | -| 🟡 | code-quality | `src/views/login.tsx:45` | Toast logic copy-pasted across 3 views | Extract a `useToast` helper | M | -| 🟢 | code-quality | `src/legacy/utils.ts:120` | Unused export `formatLegacyDate` | Delete the function and its imports | S | +## Confirmed strengths -## Top actions +- Important control or decision verified with its evidence. Do not add praise without consequence. -Highest impact first. Each action names the finding rows it resolves and, when a fix is wanted, the act-skill to hand off to (refactor, test, impeccable - the audit itself never edits code). +## Disputes -1. {{action_1}} -2. {{action_2}} -3. {{action_3}} +- Question with credible conflicting evidence, the competing interpretations, and what would settle it. -## Coverage +## Unknowns -Proves each pillar was examined. A pillar with no findings is still scanned and listed here. A pillar that could not be examined (missing tool or runtime) is listed under Skipped with the reason - never silently dropped. +- Question that could not be settled, missing evidence, and what would settle it. + +## Coverage -- **Scanned**: {{pillars examined, comma-separated}} -- **Skipped**: {{pillars not examined + reason, or "none"}} +- **Scanned**: Sources and surfaces actually examined +- **Skipped**: Excluded or inaccessible surfaces with reasons +- **Boundary**: Why further scanning was unlikely to change the top findings diff --git a/plugins/aidd-dev/skills/04-audit/assets/audit-validator.yml b/plugins/aidd-dev/skills/04-audit/assets/audit-validator.yml new file mode 100644 index 000000000..159a3af7d --- /dev/null +++ b/plugins/aidd-dev/skills/04-audit/assets/audit-validator.yml @@ -0,0 +1,171 @@ +required_frontmatter: + - audit + - chapter + - status + - owner + - last_verified + - sources + - depends_on + +status_values: + - complete + - partial + - skipped + - stale + +document_sections: + summary: + required: + - Verdict + - Highest-leverage findings + - Highest-leverage actions + - Contents + - Boundaries + scope: + required: + - Target + - Normative sources + - Exclusions + - Budget + - Runtime policy + - Independence + - System map + pillar: + required: + - Verdict + - Findings + - Confirmed strengths + - Disputes + - Unknowns + - Coverage + optional: + - Rule control matrix + - Decision register + - Automation candidates + challenge: + required: + - Independence + - Decisions + - Rejected candidates + - Disputes + - Coverage + synthesis: + required: + - Root causes + - Isolated findings + - Rejected clusters + actions: + required: + - Action sequence + - Deferred + - Order rationale + coverage: + required: + - Scanned + - Skipped + - Unknowns + - Freshness + - Runtime and independence + - 80/20 boundary + +finding_required_fields: + - Priority + - Question + - Criterion + - Evidence kind + - Evidence + - Observation + - Interpretation + - Impact + - Likelihood + - Reach + - Confidence + - Reproduction + - Falsification + - Next action + - Effort + - Automation + +finding_field_values: + Priority: + - P0 + - P1 + - P2 + Evidence kind: + - code + - configuration + - test + - command + - artifact + - runtime + - history + - normative + Likelihood: + - high + - medium + - low + Reach: + - local + - module + - system + - user-base + Confidence: + - high + - medium + Effort: + - S + - M + - L + +structured_sections: + Rule control matrix: + required_columns: + - Control ID + - Rule + - Source + - Scope + - Precedence + - Letter + - Spirit + - Status + - Evidence + status_values: + - pass + - partial + - fail + - unknown + - disputed + Decision register: + required_columns: + - Decision ID + - Choice + - Provenance + - Confidence + - Alternatives + - Assumptions + - Generality + - Reversibility + - Evidence + - Verdict + Automation candidates: + required_columns: + - Candidate ID + - Finding + - Recurrence + - Current cost + - Enforcement layer + - Leverage + - False-positive risk + - Maintenance cost + - Owner + +finding_limits: + default_maximum: 5 + rules: all_active_rules + +forbidden: + - unsupported_finding + - cosmetic_nit + - historical_plan_as_normative_without_user_override + - general_e2e_without_critical_escalation + - application_source_edit diff --git a/plugins/aidd-dev/skills/04-audit/references/audit-contract.md b/plugins/aidd-dev/skills/04-audit/references/audit-contract.md new file mode 100644 index 000000000..c9b3b76e5 --- /dev/null +++ b/plugins/aidd-dev/skills/04-audit/references/audit-contract.md @@ -0,0 +1,79 @@ +--- +name: audit-contract +description: Runtime inputs, source policy, invariants, and completion rules for question-led audits. +--- + +# Audit contract + +## Inputs + +All inputs are optional. + +| Input | Default | +| --- | --- | +| Target | Current repository | +| Scope | Current application code and configuration | +| North Stars | User-supplied sources, then discovered candidates | +| Rules | Active host instructions and scoped repository rules | +| Memory | `aidd_docs/memory/**` when present | +| Architecture | Explicitly current records and observable boundaries | +| Budget | Core profile, five findings per pillar | + +Record resolved inputs and unresolved conflicts in `01-scope-and-system-map.md`. A missing source skips only its dependent pillar. + +## Source policy + +Use, in order: + +1. Current code, configuration, schemas, lockfiles, and tests as implementation truth. +2. User-confirmed North Stars as product truth. +3. Active host instructions and scoped repository rules as behavioral constraints. +4. `aidd_docs/memory/**` as maintained knowledge to verify. +5. Explicitly current architecture records as intended structure. + +Exclude `.git`, vendored dependencies, build output, generated artifacts, `aidd_docs/tasks/**`, old plans, and past reviews unless explicitly placed in scope. Do not treat an observed convention as a rule. + +When normative sources conflict, apply an explicit precedence supplied by the host or user. Otherwise record `disputed`; never choose silently. + +## Audit states + +Every concern moves through: + +`question → impression → hypothesis → investigation → pass | finding | unknown | disputed` + +- An impression is a search lead, never a conclusion. +- A hypothesis states what evidence would confirm or refute it. +- A finding cites decisive evidence and a material consequence. +- `unknown` means evidence is absent or inconclusive. +- `disputed` means credible evidence supports incompatible interpretations. + +## 80/20 boundary + +- Maximum five findings per pillar, highest impact first. +- `rules` is the exception: evaluate every active rule, while grouping repeated violations by root cause. +- Omit style preferences, harmless drift, exhaustive inventories, and low-impact completeness work. +- Stop when further scanning is unlikely to change the top actions. +- A skipped or clean pillar is still recorded in coverage. + +## Read-only boundary + +The audit may write only its report folder. It never edits application source, tests, configuration, memory, rules, or normative documents. + +It may inspect existing test, coverage, CI, profiler, and build artifacts. It does not generate coverage, run a full suite, launch the site, or execute E2E by default. + +A bounded runtime probe requires all of: + +1. plausible critical correctness, security, privacy, or data-integrity impact; +2. static evidence cannot settle the hypothesis; +3. the probe is read-only and narrowly scoped; +4. `01-scope-and-system-map.md` records the reason before it runs. + +## Completion + +A pillar is complete when: + +- its sources and exclusions are named; +- every finding passed the evidence gate; +- material unknowns remain visible; +- coverage states what was and was not examined; +- the report contains no placeholder or unsupported claim. diff --git a/plugins/aidd-dev/skills/04-audit/references/question-packs.md b/plugins/aidd-dev/skills/04-audit/references/question-packs.md new file mode 100644 index 000000000..951f2d6d7 --- /dev/null +++ b/plugins/aidd-dev/skills/04-audit/references/question-packs.md @@ -0,0 +1,86 @@ +--- +name: question-packs +description: High-leverage questions and evidence targets for the nine core audit pillars. +--- + +# Question packs + +## North Star + +- Which critical outcome does the code implement differently from the current North Star? +- What shipped behavior is absent from or contrary to product intent? +- Which technical choice optimises a local concern at the expense of the primary outcome? + +Evidence: confirmed North Star sources, routes, commands, schemas, public APIs, critical application paths. + +## Rules + +- Which active rules fail in code, configuration, tests, or maintained memory? +- Which rules are followed literally but violated in spirit? +- Which two active rules conflict or cannot both be satisfied? + +Evidence: exact active rule, scoped target, concrete implementation reference. Evaluate every active rule. + +## Memory + +- Which maintained memory claim is stale, false, or no longer used? +- Which current architectural fact would mislead a fresh agent because memory omits it? +- Which duplicated memory statement disagrees with its canonical source? + +Evidence: `aidd_docs/memory/**` versus current code, configuration, commands, and dependencies. Ignore task history. + +## Architecture and decisions + +- Where does code cross a declared or observable boundary in the wrong direction? +- Which decision was expedient but local rather than general? +- Which module owns responsibilities that should be separated? +- Which inferred choice is least defensible or hardest to reverse? + +Evidence: current architecture sources, dependency graph, imports, public surfaces, data flow, history only when needed to establish a material choice. + +## Code hotspots + +- Which files or functions concentrate disproportionate size, complexity, churn, duplication, or error handling? +- Which dead paths, vestigial flags, or swallowed errors create material maintenance risk? +- Which abstraction makes common changes touch unrelated areas? + +Evidence: file and function size relative to project siblings, static tooling when available, imports, call sites, current TODOs, error paths. Size alone is a lead, not a finding. + +## Test value + +- Which meaningful risk does each important test group protect? +- What plausible regression would make it fail? +- Which tests stay green while the behavior they claim to protect is broken? +- Which tests duplicate cheaper protection or cost more than their unique signal? +- Which critical behavior has no effective witness? + +Classify: `protective`, `redundant`, `brittle`, `ceremonial`, or `misleading`. + +Evidence: test assertions, production behavior, fixtures, mocks, CI configuration, existing timing or flake history. Coverage percentage alone proves nothing. + +## Security and data + +- Where does untrusted data cross a boundary without validation or authorization? +- Which path can disclose, corrupt, overwrite, or silently lose data? +- Which security assumption exists only by convention? +- Which direct dependency creates a material, evidenced trust or supply-chain risk? + +Evidence: trust boundaries, schemas, auth gates, persistence, serialization, secrets, security configuration, manifests, lockfiles, and current authoritative scanner artifacts. Do not infer exploitability from names alone. + +## Performance and reliability + +- Which path repeats I/O, loads unbounded data, performs avoidable heavy work, or amplifies failure? +- Which oversized asset or module materially affects startup, build, or runtime? +- Which failure is invisible because timeouts, retries, logging, metrics, or recovery are absent? + +Evidence: static call paths and existing measurements first. Without a measurement or clear complexity argument, report a hypothesis or unknown, not a bottleneck. + +## Automation and knowledge infrastructure + +- Which issue class has already required repeated human or agent reasoning? +- Which review rejection reveals missing repository knowledge? +- Could a fresh agent choose correctly with zero extra prompt? +- What invariant belongs in code rather than prose? +- What domain guidance belongs in a rule, skill, memory, or canonical document? + +Evidence: confirmed cross-pillar findings, repeated patterns, current rules and automation. Rank expected leverage against false-positive and maintenance cost. diff --git a/plugins/aidd-dev/skills/04-audit/references/question-protocol.md b/plugins/aidd-dev/skills/04-audit/references/question-protocol.md new file mode 100644 index 000000000..2de9b553d --- /dev/null +++ b/plugins/aidd-dev/skills/04-audit/references/question-protocol.md @@ -0,0 +1,89 @@ +--- +name: question-protocol +description: Converts introspective audit questions into falsifiable, evidence-gated conclusions. +--- + +# Question protocol + +## Loop + +For each high-leverage question: + +1. **Ask.** Use the pillar's question pack. +2. **Impression.** State the uncomfortable first read in one sentence. +3. **Hypothesis.** Name the suspected problem, expected consequence, confirming evidence, and refuting evidence. +4. **Investigate.** Inspect the cheapest decisive evidence first. +5. **Challenge.** Search once for contradictory evidence or a simpler explanation. +6. **Verdict.** Record `pass`, `finding`, `unknown`, or `disputed`. + +Do not expose hidden chain-of-thought. Reports contain only questions, concise hypotheses, evidence, and verdicts. + +## Choice questions + +- Which choices were specified by a North Star, rule, or accepted architecture source? +- Which choices appear to have been made because the work was underspecified? +- Which choices are low-confidence, irreversible, or unusually broad? +- Which local constant, threshold, buffer, special case, or abstraction solves only the observed example? +- What would a careful engineer choose differently today, and why? +- Which decision is being inferred retrospectively rather than established by history? + +Decision provenance: + +- `specified`: directly required by a normative source; +- `agent-made`: explicitly recorded by the implementation context; +- `retrospectively-inferred`: reconstructed from code or history. + +Retrospective inference never proves original intent. + +## Pride and blindness questions + +- Would you defend this design during an incident? +- What feels clever rather than clear? +- What are you least confident about? +- What important fact is unavailable from this repository? +- Where could the system fail while looking successful? +- Which two parts of the system tell incompatible stories? + +Treat answers as impressions until verified. + +## Evidence gate + +A finding needs: + +- a named criterion or risk; +- a concrete evidence kind and reference; +- a direct observation separated from its interpretation; +- material impact with likelihood and reach; +- a confidence level; +- a minimal reproduction, or a reason why reproduction does not apply; +- one attempt to falsify it. + +Confidence: + +- `high`: direct evidence with no credible contradiction; +- `medium`: converging evidence with a remaining assumption; +- `low`: useful concern, but insufficient for a finding; report as `unknown`. + +## Generality gate + +A fix or decision is non-general when it makes the observed example pass while the underlying class can still fail. Check: + +- nearby inputs, sizes, platforms, or call sites; +- duplicated local exceptions; +- unexplained constants or thresholds; +- invariant enforcement at the correct boundary; +- whether another caller can reproduce the same class. + +## Automation question + +For every recurring issue ask: “What should make this class impossible or automatically visible next time?” + +Choose the strongest proportionate layer: + +- type, schema, lint, static analysis; +- unit, property, integration, or E2E test; +- CI check or repository automation; +- rule, skill, memory, code comment, or canonical documentation; +- runtime assertion, telemetry, or alert. + +Do not automate a one-off preference or create machinery whose maintenance cost exceeds the protected risk. diff --git a/plugins/aidd-dev/skills/04-audit/references/report-contract.md b/plugins/aidd-dev/skills/04-audit/references/report-contract.md new file mode 100644 index 000000000..07d446eb8 --- /dev/null +++ b/plugins/aidd-dev/skills/04-audit/references/report-contract.md @@ -0,0 +1,102 @@ +--- +name: report-contract +description: Ordered audit folder, chapter ownership, finding identity, freshness, and synthesis rules. +--- + +# Report contract + +## Folder + +`aidd_docs/tasks//_audit_/` + +Reuse the same folder when refreshing the same scope on the same day. Never write audit prose outside it. + +## Ordered files + +| File | Purpose | +| --- | --- | +| `00-summary.md` | Executive verdict, highest-leverage findings, ordered contents | +| `01-scope-and-system-map.md` | Target, resolved sources, exclusions, budget, runtime and independence policy | +| `02-north-star.md` | Product intent versus code | +| `03-rules.md` | Active-rule control matrix and violations | +| `04-memory.md` | Maintained memory versus current usage | +| `05-architecture-and-decisions.md` | Architecture, choice quality, divergence, generality | +| `06-code-hotspots.md` | Material code-quality hotspots | +| `07-tests-value.md` | Test protection and cost | +| `08-security-and-data.md` | Security, privacy, data integrity | +| `09-performance-and-reliability.md` | Performance and operational resilience | +| `10-automation-and-knowledge-infrastructure.md` | Permanent guardrail candidates | +| `11-challenge-ledger.md` | Accepted, rejected, merged, and disputed candidate findings | +| `12-systemic-findings.md` | Cross-pillar root causes | +| `13-prioritized-actions.md` | Risk-ranked remediation and automation sequence | +| `14-coverage-and-unknowns.md` | Scanned, skipped, stale, inaccessible, unknown | + +A single-pillar run writes only `01`, its pillar file, and `14`. A standalone full run writes all files and marks `11` as `independence: none` unless a fresh context challenged the shards. + +Templates: + +- `assets/audit-index-template.md` for `00` +- `assets/audit-scope-template.md` for `01` +- `assets/audit-template.md` for `02` through `10` +- `assets/audit-challenge-template.md` for `11` +- `assets/audit-synthesis-template.md` for `12` +- `assets/audit-actions-template.md` for `13` +- `assets/audit-coverage-template.md` for `14` + +## Chapter frontmatter + +Every file starts with: + +```yaml +--- +audit: +chapter: +status: complete | partial | skipped | stale +owner: +last_verified: +sources: + - +depends_on: + - +--- +``` + +## Pillar body + +Use `assets/audit-template.md`. Keep exactly: + +1. `## Verdict` +2. `## Findings` +3. `## Confirmed strengths` +4. `## Disputes` +5. `## Unknowns` +6. `## Coverage` + +Rules additionally include `## Rule control matrix` using the schema in `actions/09-rules-and-principles.md`. Architecture additionally includes `## Decision register` using `actions/11-decisions.md`. Automation additionally includes `## Automation candidates` using `actions/12-automation.md`. + +## Finding identity + +IDs use a stable pillar prefix and sequence: `NS-001`, `RULE-001`, `MEM-001`, `ARCH-001`, `CODE-001`, `TEST-001`, `SEC-001`, `PERF-001`, `AUTO-001`. + +Each finding records: + +- priority `P0`, `P1`, or `P2`; +- originating question and criterion; +- evidence kind and reference; +- direct observation and its interpretation; +- material impact, likelihood, and reach; +- confidence `high` or `medium`; +- minimal reproduction or `not-applicable` with reason; +- falsification attempt; +- next action and effort `S`, `M`, or `L`; +- automation candidate or `none`. + +- `P0`: plausible severe correctness, security, privacy, data loss, or North Star failure. +- `P1`: material recurring risk, architectural divergence, or expensive friction. +- `P2`: clear leverage with bounded impact. Cosmetic work is omitted. + +## Canonical ownership + +A finding exists in exactly one pillar file. Summary, challenge, systemic, and action files link its ID; they never copy its full prose. + +When a pillar is refreshed, mark dependent synthesis chapters `stale` until regenerated. Preserve stable IDs when the underlying finding remains the same. From 4a39ca8102d4cddbfb1eebc223e339134a97b068 Mon Sep 17 00:00:00 2001 From: alexsoyes Date: Sat, 25 Jul 2026 14:15:44 +0200 Subject: [PATCH 2/3] fix(framework): install cli deps in new worktrees First pushes evaluate CLI hooks, so worktrees need both dependency sets. --- .claude/hooks/worktree-create.js | 30 +++++---- scripts/__tests__/worktree-create.test.js | 79 +++++++++++++++++++++++ 2 files changed, 98 insertions(+), 11 deletions(-) create mode 100644 scripts/__tests__/worktree-create.test.js diff --git a/.claude/hooks/worktree-create.js b/.claude/hooks/worktree-create.js index b647d5e54..04aa1d459 100644 --- a/.claude/hooks/worktree-create.js +++ b/.claude/hooks/worktree-create.js @@ -34,6 +34,19 @@ function readWorktreeName() { return payload.name; } +function installDependencies(directory, label) { + try { + execFileSync("pnpm", ["install", "--frozen-lockfile"], { + cwd: directory, + stdio: ["ignore", "inherit", "inherit"], + }); + } catch { + process.stderr.write( + `worktree-create: ${label} pnpm install skipped or failed; run it manually before committing or pushing\n`, + ); + } +} + function createWorktree() { const worktreeName = readWorktreeName(); const baseBranch = branchExists("next") ? "next" : "main"; @@ -44,17 +57,12 @@ function createWorktree() { git("worktree", "add", worktreePath, "-b", newBranch, baseBranch); - // A fresh worktree has no node_modules, so its first commit fails the - // commit-msg hook (commitlint not found). Install deps best-effort: never - // let a failed install block the worktree the user asked for. - try { - execFileSync("pnpm", ["install", "--frozen-lockfile"], { - cwd: worktreePath, - stdio: ["ignore", "inherit", "inherit"], - }); - } catch { - process.stderr.write("worktree-create: pnpm install skipped or failed; run it manually before committing\n"); - } + // A fresh worktree has no node_modules. Root hooks need the framework + // dependencies to commit, while first-push checks need the standalone CLI + // dependencies even when the feature itself does not touch cli/. + // Install both best-effort: dependency setup must not block worktree creation. + installDependencies(worktreePath, "root"); + installDependencies(join(worktreePath, "cli"), "CLI"); process.stdout.write(worktreePath + "\n"); } diff --git a/scripts/__tests__/worktree-create.test.js b/scripts/__tests__/worktree-create.test.js new file mode 100644 index 000000000..e7aad8711 --- /dev/null +++ b/scripts/__tests__/worktree-create.test.js @@ -0,0 +1,79 @@ +const assert = require("node:assert/strict"); +const { + chmodSync, + mkdtempSync, + mkdirSync, + readFileSync, + realpathSync, + rmSync, + writeFileSync, +} = require("node:fs"); +const { tmpdir } = require("node:os"); +const { delimiter, join, resolve } = require("node:path"); +const { spawnSync } = require("node:child_process"); +const test = require("node:test"); + +test("worktree-create installs root and CLI dependencies", () => { + const fixtureRoot = realpathSync(mkdtempSync(join(tmpdir(), "worktree-create-"))); + const binDir = join(fixtureRoot, "bin"); + const repoRoot = join(fixtureRoot, "repo"); + const pnpmLog = join(fixtureRoot, "pnpm.log"); + const hookPath = resolve(".claude/hooks/worktree-create.js"); + + mkdirSync(binDir, { recursive: true }); + mkdirSync(repoRoot, { recursive: true }); + + const fakeGit = `#!/usr/bin/env node +const { mkdirSync } = require("node:fs"); +const { join } = require("node:path"); +const args = process.argv.slice(2); +if (args[0] === "rev-parse" && args.includes("refs/heads/next")) process.exit(0); +if (args[0] === "rev-parse" && args[1] === "--show-toplevel") { + process.stdout.write(process.env.FAKE_REPO_ROOT + "\\n"); + process.exit(0); +} +if (args[0] === "worktree" && args[1] === "add") { + mkdirSync(join(args[2], "cli"), { recursive: true }); + process.exit(0); +} +process.exit(1); +`; + + const fakePnpm = `#!/usr/bin/env node +const { appendFileSync } = require("node:fs"); +appendFileSync( + process.env.FAKE_PNPM_LOG, + process.cwd() + "\\t" + process.argv.slice(2).join(" ") + "\\n", +); +`; + + writeFileSync(join(binDir, "git"), fakeGit); + writeFileSync(join(binDir, "pnpm"), fakePnpm); + chmodSync(join(binDir, "git"), 0o755); + chmodSync(join(binDir, "pnpm"), 0o755); + + try { + const result = spawnSync(process.execPath, [hookPath], { + cwd: repoRoot, + encoding: "utf8", + env: { + ...process.env, + FAKE_PNPM_LOG: pnpmLog, + FAKE_REPO_ROOT: repoRoot, + PATH: `${binDir}${delimiter}${process.env.PATH}`, + }, + input: JSON.stringify({ name: "audit" }), + }); + + assert.equal(result.status, 0, result.stderr); + + const worktreePath = join(repoRoot, ".claude", "worktrees", "audit"); + assert.equal(result.stdout, `${worktreePath}\n`); + assert.deepEqual(readFileSync(pnpmLog, "utf8").trim().split("\n"), [ + `${worktreePath}\tinstall --frozen-lockfile`, + `${join(worktreePath, "cli")}\tinstall --frozen-lockfile`, + ]); + } finally { + rmSync(fixtureRoot, { recursive: true, force: true }); + } +}); From ed4239a7f2d2609a5bbdd577b318b3f232f479a6 Mon Sep 17 00:00:00 2001 From: alexsoyes Date: Sun, 26 Jul 2026 13:24:34 +0200 Subject: [PATCH 3/3] refactor(aidd-dev): remove duplicated action checks Keep audit validation canonical in the shared contract and validator. --- docs/ARCHITECTURE.md | 2 +- .../aidd-dev/skills/04-audit/actions/00-core.md | 8 -------- .../skills/04-audit/actions/01-code-quality.md | 7 ------- .../skills/04-audit/actions/02-architecture.md | 7 ------- .../skills/04-audit/actions/03-security.md | 9 +-------- .../skills/04-audit/actions/05-performance.md | 7 ------- .../aidd-dev/skills/04-audit/actions/06-tests.md | 14 ++++---------- .../skills/04-audit/actions/08-north-star.md | 7 ------- .../04-audit/actions/09-rules-and-principles.md | 7 ------- .../actions/10-memory-and-documentation.md | 7 ------- .../skills/04-audit/actions/11-decisions.md | 7 ------- .../skills/04-audit/actions/12-automation.md | 7 ------- 12 files changed, 6 insertions(+), 83 deletions(-) diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index b355fc578..ba6f3b0dd 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -141,7 +141,7 @@ flowchart LR ActionN --> Out ``` -Each action is a self-contained markdown file with inputs, outputs, depends-on, process steps, and a test checklist. Actions can call other skills via the `Skill` tool, so a skill discovers a capability it needs at runtime (by matching skill descriptions, never by hardcoded plugin name) and delegates to it. +Each action is a self-contained markdown file with inputs, outputs, dependencies, and process steps. Keep a test checklist only when it adds action-specific validation that is not already enforced by a canonical contract or validator. Actions can call other skills via the `Skill` tool, so a skill discovers a capability it needs at runtime (by matching skill descriptions, never by hardcoded plugin name) and delegates to it. ## 🤖 Skills and agents diff --git a/plugins/aidd-dev/skills/04-audit/actions/00-core.md b/plugins/aidd-dev/skills/04-audit/actions/00-core.md index ab095a7a6..8a8a7a3e4 100644 --- a/plugins/aidd-dev/skills/04-audit/actions/00-core.md +++ b/plugins/aidd-dev/skills/04-audit/actions/00-core.md @@ -21,11 +21,3 @@ Optional target, scope, North Star sources, rule sources, memory path, architect 7. **Close coverage.** Write `14-coverage-and-unknowns.md` with scanned, skipped, stale, inaccessible, and unresolved surfaces. 8. **Summarise.** Write `00-summary.md` last from canonical finding IDs. Never copy full finding prose. 9. **Validate.** Check each document against its section contract in `../assets/audit-validator.yml`; remove placeholders and unsupported claims. - -## Test - -- `00` through `14` exist, or a skipped chapter exists with its reason. -- Every finding exists canonically in one pillar chapter. -- Summary, challenge, systemic, and action chapters link IDs instead of duplicating findings. -- `11-challenge-ledger.md` states the achieved independence honestly. -- No file outside the audit report folder changed. diff --git a/plugins/aidd-dev/skills/04-audit/actions/01-code-quality.md b/plugins/aidd-dev/skills/04-audit/actions/01-code-quality.md index 0bc32c36e..4643b6e8e 100644 --- a/plugins/aidd-dev/skills/04-audit/actions/01-code-quality.md +++ b/plugins/aidd-dev/skills/04-audit/actions/01-code-quality.md @@ -24,10 +24,3 @@ Target, optional scope, and resolved sources from `01-scope-and-system-map.md`. 3. Inspect call sites, imports, error paths, and current change surfaces for the strongest leads. 4. Falsify each lead: a large generated table or cohesive module is not a finding merely because it is large. 5. Report at most five verified root problems. Architecture coupling belongs to `02`; runtime cost belongs to `05`. - -## Test - -- Every finding proves a material consequence beyond file size or personal style. -- No dead code is asserted without call-site or tooling evidence. -- `06-code-hotspots.md` has the required sections and at most five findings. -- Application files are unchanged. diff --git a/plugins/aidd-dev/skills/04-audit/actions/02-architecture.md b/plugins/aidd-dev/skills/04-audit/actions/02-architecture.md index 3c19fd0f1..9a5950996 100644 --- a/plugins/aidd-dev/skills/04-audit/actions/02-architecture.md +++ b/plugins/aidd-dev/skills/04-audit/actions/02-architecture.md @@ -25,10 +25,3 @@ Current architecture sources, memory, code structure, and the decision analysis 4. Compare declared intent with implementation. When no current declaration exists, audit observable coupling without inventing a desired architecture. 5. Apply the generality gate and inspect contradictory evidence. 6. Group symptoms by architectural cause and report at most five findings. - -## Test - -- Every conformance finding cites both intended boundary and implementation evidence. -- Inferred intent is labelled and cannot produce a conformance finding by itself. -- The decision register records provenance, confidence, generality, reversibility, and evidence. -- `05-architecture-and-decisions.md` contains at most five root findings. diff --git a/plugins/aidd-dev/skills/04-audit/actions/03-security.md b/plugins/aidd-dev/skills/04-audit/actions/03-security.md index 4fff0f16b..2dca04568 100644 --- a/plugins/aidd-dev/skills/04-audit/actions/03-security.md +++ b/plugins/aidd-dev/skills/04-audit/actions/03-security.md @@ -22,14 +22,7 @@ Target code, configuration, schemas, persistence paths, manifests, lockfiles, ex 1. Read the audit contract, question protocol, and Security and data pack. 2. Trace external inputs through validation, authorization, persistence, serialization, and output. -3. Inspect secrets, injection sinks, unsafe deserialization, CORS/TLS/security defaults, and destructive operations where relevant. +3. Inspect secrets, injection sinks, unsafe deserialization, CORS/TLS/security defaults, and destructive operations where relevant. Cite a secret's location and kind, never its value. 4. Use existing static-security or dependency-scanner output when available. Do not perform a general dependency inventory or guess current CVEs, versions, provenance, or licences. 5. Attempt to falsify exploitability and impact. Names or theoretical patterns alone remain unknown. 6. Report at most five highest-risk findings. - -## Test - -- Every finding names the trust boundary, evidence, plausible consequence, and contradictory evidence checked. -- No secret value is copied into the report; cite only its location and kind. -- Dependency findings cite authoritative current evidence and a material affected path. -- Application files are unchanged. diff --git a/plugins/aidd-dev/skills/04-audit/actions/05-performance.md b/plugins/aidd-dev/skills/04-audit/actions/05-performance.md index 49c767371..69401547d 100644 --- a/plugins/aidd-dev/skills/04-audit/actions/05-performance.md +++ b/plugins/aidd-dev/skills/04-audit/actions/05-performance.md @@ -25,10 +25,3 @@ Target code, current budgets when defined, and existing profiler, build, bundle, 4. Do not launch the site, generate a new profile, or run E2E by default. 5. A bounded runtime probe must pass the critical escalation gate in `audit-contract.md`. 6. Report at most five material findings. - -## Test - -- Every bottleneck finding cites measurement or a decisive complexity path. -- “Large” is compared with a project budget, artifact, or meaningful consequence. -- Reliability findings name the failure mode and missing detection or recovery. -- No general runtime or E2E flow runs. diff --git a/plugins/aidd-dev/skills/04-audit/actions/06-tests.md b/plugins/aidd-dev/skills/04-audit/actions/06-tests.md index fc441fd89..8d7159e42 100644 --- a/plugins/aidd-dev/skills/04-audit/actions/06-tests.md +++ b/plugins/aidd-dev/skills/04-audit/actions/06-tests.md @@ -25,13 +25,7 @@ Test code, production behavior, fixtures, mocks, test configuration, CI history, 3. Inspect assertions, mocks, fixtures, timing dependencies, skips, snapshots, and overlap. Do not classify from filenames alone. 4. Classify material groups as `protective`, `redundant`, `brittle`, `ceremonial`, or `misleading`. 5. Compare signal with runtime, flake, fixture complexity, duplication, and maintenance cost. -6. Recommend `delete` only when no distinct protected risk remains beyond a cheaper test. Otherwise prefer `keep`, `rewrite`, `merge`, or `unknown`. -7. Do not run E2E. Use existing CI, coverage, timing, and flake evidence; do not generate new coverage by default. -8. Report at most five highest-leverage findings. - -## Test - -- Every finding names protected risk, plausible regression, signal failure, and cost. -- Coverage percentage alone never produces a finding. -- A delete recommendation proves the absence of unique protection. -- Critical untested behavior cites the production path and missing effective witness. +6. For critical untested behavior, cite the production path and the missing effective witness. +7. Recommend `delete` only when no distinct protected risk remains beyond a cheaper test. Otherwise prefer `keep`, `rewrite`, `merge`, or `unknown`. +8. Do not run E2E. Use existing CI, coverage, timing, and flake evidence; do not generate new coverage by default. +9. Report at most five highest-leverage findings. diff --git a/plugins/aidd-dev/skills/04-audit/actions/08-north-star.md b/plugins/aidd-dev/skills/04-audit/actions/08-north-star.md index acc8b4207..73932cc22 100644 --- a/plugins/aidd-dev/skills/04-audit/actions/08-north-star.md +++ b/plugins/aidd-dev/skills/04-audit/actions/08-north-star.md @@ -25,10 +25,3 @@ Explicit North Star sources, or candidates discovered in the current repository. 4. Extract only material outcomes and constraints, not every sentence. 5. Trace them into routes, commands, schemas, public APIs, and critical code paths. 6. Report at most five verified divergences. - -## Test - -- Every finding cites both a confirmed North Star statement and implementation evidence. -- Missing or conflicting sources produce skipped, unknown, or disputed status rather than invented intent. -- Old plans are ignored unless explicitly declared normative. -- `02-north-star.md` contains no cosmetic product suggestions. diff --git a/plugins/aidd-dev/skills/04-audit/actions/09-rules-and-principles.md b/plugins/aidd-dev/skills/04-audit/actions/09-rules-and-principles.md index 3060507ee..5150ebff3 100644 --- a/plugins/aidd-dev/skills/04-audit/actions/09-rules-and-principles.md +++ b/plugins/aidd-dev/skills/04-audit/actions/09-rules-and-principles.md @@ -29,10 +29,3 @@ Active host instructions, scoped repository rules, user-supplied principles, and 4. Group repeated violations of one rule into one root finding, but keep every violating location linked in the control matrix. 5. Report every active rule in the matrix. The five-finding cap does not hide material rule failures; prioritise the five root causes in the Findings section and link remaining failed controls. 6. Suggest automation when a rule can be enforced more reliably by type, lint, test, CI, hook, or skill. - -## Test - -- Every active rule appears exactly once in the control matrix. -- Every fail cites the exact rule source and concrete implementation evidence. -- Observed convention is never promoted to a rule. -- Conflicting precedence is marked disputed, not resolved by guess. diff --git a/plugins/aidd-dev/skills/04-audit/actions/10-memory-and-documentation.md b/plugins/aidd-dev/skills/04-audit/actions/10-memory-and-documentation.md index 3037ed80c..d89e6080b 100644 --- a/plugins/aidd-dev/skills/04-audit/actions/10-memory-and-documentation.md +++ b/plugins/aidd-dev/skills/04-audit/actions/10-memory-and-documentation.md @@ -25,10 +25,3 @@ Audit whether maintained project memory still matches code and current usage. Hi 4. Verify each high-impact claim against current code, configuration, manifests, and scripts. 5. Ignore `aidd_docs/tasks/**`, old plans, and historical reviews even when they contradict memory. 6. Report at most five stale or missing facts with the greatest agent-decision impact. - -## Test - -- Every stale-memory finding cites memory and current implementation evidence. -- “Missing memory” explains the wrong decision a fresh agent could make. -- No historical task artifact is treated as current truth. -- The audit never edits memory. diff --git a/plugins/aidd-dev/skills/04-audit/actions/11-decisions.md b/plugins/aidd-dev/skills/04-audit/actions/11-decisions.md index 2c6c997a4..02a97b0df 100644 --- a/plugins/aidd-dev/skills/04-audit/actions/11-decisions.md +++ b/plugins/aidd-dev/skills/04-audit/actions/11-decisions.md @@ -30,10 +30,3 @@ The `## Decision register` and decision-related findings in `05-architecture-and 4. Apply the generality gate to constants, thresholds, buffers, special cases, and narrowly placed fixes. 5. Use decision risk to focus code inspection; never let a declared decision suppress contradictory evidence. 6. Feed verified decision findings into the shared architecture chapter. - -## Test - -- Retrospective inference is never presented as original intent. -- Every non-general finding proves the class remains unresolved beyond the observed example. -- High-confidence declarations cannot override contradictory code evidence. -- The decision register contains no hidden chain-of-thought. diff --git a/plugins/aidd-dev/skills/04-audit/actions/12-automation.md b/plugins/aidd-dev/skills/04-audit/actions/12-automation.md index 199e9e570..4168d3a21 100644 --- a/plugins/aidd-dev/skills/04-audit/actions/12-automation.md +++ b/plugins/aidd-dev/skills/04-audit/actions/12-automation.md @@ -30,10 +30,3 @@ Confirmed findings from other pillars, active automation, rules, skills, memory, 4. Record recurrence, current human or token cost, proposed layer, expected leverage, false-positive risk, maintenance cost, and owner. 5. Prefer one class-level guardrail over multiple one-off fixes. 6. Rank at most five automation candidates. - -## Test - -- Every candidate links to a confirmed finding or repeated evidenced pattern. -- One-off preferences do not become permanent machinery. -- Proposed enforcement is placed at the strongest proportionate layer. -- Expected leverage and maintenance or false-positive risk are both stated.