Conversation
AuthenticationError/RateLimitError/NotFoundError/ValidationError/UpstreamError/TimeoutError lacked ClassVar http_status, so the error-envelope handler returned 500 for all of them despite their docstrings promising 401/429/404/422/502/504. Add code/http_status/user_message ClassVars (codes preserved to match the prior derive-from-name output) plus operator hints, and extend the taxonomy + handler tests to lock the six statuses. Retarget the fallback test to a genuinely undeclared subclass.
RegisteredSystem inherited extra=forbid from the SSHIntegrationInput write payload, so constructing the read shape from an ORM row carrying undeclared columns (team_id, private_key_secret_id, future columns) raised at response-serialization time -- a latent 500 waiting on the next systems-table migration. The read shape now sets extra=ignore; the write payload keeps forbid so agents cannot submit undeclared fields.
…tion (#61) ReasoningCaseState.observables and ReasoningTurnDecision.observables are dict[str, Any] persisted to the DB and passed as task kwargs, both of which json-encode them. A datetime/bytes/set passed Pydantic's dict[str, Any] check then crashed later at json.dumps (enqueue or DB write). A field_validator now fails fast at construction with the offending detail. Existing string-valued observables are unaffected.
_enrich_response constructed LLMResponse with classification/confidence/seal_id/pipeline_metadata, but the frozen+slots dataclass declared none of them, so any pipeline step that wrote a non-None value raised TypeError at the enrich boundary. Declare the four fields (default None). Repairs the enrich-response tests in test_pipeline/test_sanitize/test_gate/test_validate that already assert this behavior.
… lock (#44) Short rejection markers (o1/o3/o4) were matched as bare substrings, so proto1/audio1 and similar ids had temperature silently stripped. Match markers on alphanumeric boundaries via regex instead. Also remove _LLM_HEALTH_LOCK, an import-time asyncio.Lock() that was never acquired anywhere.
…#38) Sync call sites (proxy resolution, budget ceiling, worker bootstrap) could not resolve config without producing an un-awaited coroutine from the async get. get_sync mirrors get's resolution order (env > cache > DB > schema default) using the sync engine via session_scope. Cache access is lock-free (GIL-atomic dict ops; benign check-then-populate race). This is the foundation the #65 and #38 fixes consume.
_resolve_proxy is sync but called the async ConfigRegistry.get without await, so str(<coroutine> or '') returned the coroutine repr as the proxy URL and never reached the env fallback -- breaking every registry-configured provider client. Switch to get_sync (contract C3).
_resolve_ceiling (sync) called the async ConfigRegistry.get without await; the resulting coroutine is not None and int() of it raises, which the try/except swallowed -- so the per-run token ceiling always resolved to 0 (unlimited) and the budget was dead code. Switch to get_sync. Adds a reproduction test with an async-get registry stub that failed DID-NOT-RAISE before the fix.
Hoist in-function imports to module top (PLC0415) and rename local test doubles to conform to naming rules (N818 _FakeExc->_FakeError, N806 MockOAI->mock_oai) in the three pre-existing test files edited during Wave-1 fixes. No behavior change; failure set is unchanged (pre-existing stale-mock cluster verified via revert-compare).
Embedding encode (knowledge tool), pg_dump (backup_database), and the offline-installer subprocess calls now run on a worker thread instead of stalling the loop. EPSSKEVIntelTool.forward becomes async and routes the blocking EPSS/KEV HTTP through run_blocking_io, mirroring NVDIntelTool; the two intel-service callsites now await it (they previously discarded the sync result on the loop). database.py uses asyncio.to_thread directly to avoid a storage-init import cycle. New tests assert the blocking callables execute off the main thread.
…ce (#63) UnitOfWork.__aexit__ now raises UnitOfWorkNotCommittedError when a block exits cleanly with pending inserts/updates/deletes and no commit, converting the silent rollback-on-close data loss into a named, test-visible failure (C4). refresh_target_source in the VR router added a re-keyed audit-mcp handle via session.add but never committed, so the new index id was rolled back on close; it now commits. Full non-e2e suite shows zero new failures from the backstop (verified by revert-compare on the highest-volume and write-heavy suites).
…ng (#55) The severity sort table and the system_summary/report_count/fleet_severity_summary dashboards used the CVSS vocabulary (critical/high/medium/low) while findings store patching-urgency criticality (Immediate/High/Moderate/Planned). Severity sorting was a no-op and every tier except High reported zero. All three now source the vocabulary from CRITICALITY_KEYS. scoring_metadata raised RuntimeError when any finding was scored via fallback (a valid canonical mode); it now counts fallback and aggregates to fallback or mixed, and unknown modes are logged and counted as fallback rather than crashing the run summary.
…NVD limiter (#55) GHSA advisories now emit a match only when the installed version falls inside the advisory vulnerable_version_range (falling back to first_patched_version when the range is unparseable), instead of flagging every package that shares a name. The enrichment planner honors cve_cache_ttl_hours via _cve_cache_is_fresh instead of a keep-forever check, so stale intel is refreshed. The NVD rate limiter holds the global lock across the sleep and stamps the post-sleep proceed time, so concurrent threads serialize instead of stamping ~now and firing together.
Pure, deterministic metric layer for the eval harness: expected calibration error (10-bucket), reliability-diagram calibration curve, per-outcome-kind precision/recall (zero-support reported as None, not 0.0), case-count-weighted faithfulness, byte-for-byte determinism, and the EvalReport.beats promotion gate. The gate requires strictly lower ECE plus at-least-equal faithfulness with no per-kind precision/recall regression beyond tolerance, so a recall-only win never beats the baseline (RFC-08 amendment 2). No database, clock, or model dependency; the record-replay harness and storage land with the #62 backbone.
New platform helper redact_command_line masks the value after a known secret marker (-p, password=, token=, authorization: bearer, --api-key, etc.) up to the next whitespace. The SSH service now routes captured stderr and the leaked command text in its exit-code and idle-timeout error messages through it, so credentials passed on a remote command line no longer surface in exception text or logs. The config-key classification catalog and is_secret column land with #50.
Scheduled report emails carry security findings; STARTTLS was invoked with no SSL context, so smtplib used an unverified stdlib context and the server certificate was never checked. The send path now passes ssl.create_default_context(), enabling hostname and certificate verification.
…ata (#62) Legacy unit tests build an in-memory SQLite engine and create_all the whole metadata, which aborted on Postgres-only JSONB/TSVECTOR columns and the to_tsvector generated column, failing every SQLite-based test at setup. Root conftest adds test-only @compiles shims (JSONB->JSON, TSVECTOR->TEXT on sqlite) and a connect-time deterministic to_tsvector passthrough; production models stay pure Postgres. Also updates test_87 to the real patching-urgency criticality vocabulary (Immediate/High/Moderate/Planned) that findings store, correcting stale assertions surfaced once create_all succeeds, and isolates the NVD limiter test from ambient DB-URL pollution. Full non-e2e suite: +87 passing, zero new failures (verified by node-id diff vs baseline). Full Alembic-driven Postgres fixture migration remains the eventual #62 target.
The writeup artefact grouper read phantom attributes (family/type/data) via getattr defaults, so every artefact fell into the 'unknown' family with empty data and writeups rendered blank. Grouping now reads the real columns (artifact_family / artifact_type / data_json), extracted into a pure _group_artefacts helper with unit coverage.
…#43) The vulnerability report PDF rendered untrusted CVE data (cve_id, rationale, package names) through a Jinja environment with no autoescape, and the nvd_url flowed straight into an href, so HTML and javascript: injection survived into the rasterized document. The environment now enables select_autoescape and a safe_url filter neutralizes any non-http(s)/mailto scheme to about:blank; a restrictive CSP meta is added for standalone HTML viewing. Extracted _build_report_env and _safe_report_url with unit coverage.
) Adds the C1 request-scope helpers the tenant-isolation cluster wires in: require_team_context / get_team_context_or_admin, team_scoped_session and team_scoped_uow (bind TeamContext so the do_orm_execute listener filters every SELECT), and owned_or_404 which loads a single row via session.exec(select) -- not session.get, whose identity-map fast path bypasses the listener (the #57 IDOR class) -- and returns 404 (never a 403 existence oracle) for a missing or cross-team row. Postgres-backed tests prove the listener actually filters by team, admin sees all teams, and owned_or_404 enforces ownership. Team-scoped columns already exist via TeamScopedMixin, so no migration is required; wiring these into every route/UoW site is #36/#53/#57.
build_module_factory returned the raw create_module, so the validation call plus every load_builtin_modules pass reconstructed the module -- create_module side effects (platform_task registration, thread pools, metric counters) fired multiple times at startup. Validation now keeps the constructed instance and the factory returns that same instance on every call, so a module is built exactly once. Modules are stateless outside the registered runtime, so the process-lifetime singleton is safe.
…ation (#56) The malware workspace model declared uq_workspace_team_slug, colliding with the identical name on the vr workspace model. Postgres constraint names are schema-unique, so create_all (which uses model names, unlike migrations) failed with DuplicateTable whenever both modules loaded, erroring every test_db test that imported both. Migration 068 already created malware_workspaces with uq_malware_workspace_team_slug; the model now matches it. No new migration needed. Unblocks the cross-module test_db path (e.g. test_vr_disclosures).
The Fuzzilli scraper walked an untrusted fuzz output dir with is_file() (which follows symlinks) and read_bytes(), so a symlink planted in crashes/ was dereferenced and its target's bytes leaked into the crash record's payload_preview. Discovery now lstat-gates every entry (rejecting symlinks and non-regular files), refuses files above a hard size cap, and reads reproducers with O_NOFOLLOW to close the lstat-to-open TOCTOU on POSIX. _count_crash_files shares the same filter.
The investigation narrative is LLM output over untrusted case data and was persisted verbatim into the durable payload, so script/js/handler/iframe patterns survived into any future markdown renderer or evidence export. The persist path now runs title, body, and each chapter outline through sanitize_output via a pure _build_narrative_payload helper, and records per-field sanitizer_counts for the evidence lineage.
The config router is readable by any authenticated principal, so secret-classed values (keys containing api_key/secret/password/hmac_key/token/...) were returned verbatim to non-admins on GET /config, /config/{ns}, and /config/{ns}/{key}. A new is_secret_config_key classifier in the registry drives value redaction to [REDACTED] for non-admin callers; admins still receive plaintext. The schema-driven is_secret column and audit-write redaction remain follow-on #50 scope.
MalwareTargetTagIndexRecord declared its unique constraint as uq_target_tag_source, colliding with VRTargetTagIndexRecord's same-named constraint under metadata.create_all (db_init on a fresh DB, and the test harness) since Postgres constraint names are schema-unique. Migration 068 already creates uq_malware_target_tag_source, so the model was drifted from its own migration; the model name now matches it. No migration needed.
…#57) GET/PATCH/DELETE /malware/targets/{id} and GET .../stages loaded the row with session.get -- which bypasses the do_orm_execute team filter -- and checked only for None, so a caller from one team could read, mutate, or archive another team's target by id. The four handlers now bind TeamContext.from_auth(auth) to the UnitOfWork and resolve the row through owned_or_404, which selects via the team-scope listener and returns 404 (never a 403 existence oracle) for a cross-team or missing id. Admin identities (team_id None) still see any row.
… routes (#57) Beyond the four target single-resource handlers, ~14 more routes loaded a resource by a request-supplied id with session.get and only a None check, no team guard: every investigation lifecycle route (get, patch, delete, pause, resume, finalize, reset, evidence-graph, post-message), get_observation, ack_message, the target sub-resource analysis handlers (resume/retrigger), and the target-reference validation in create_project/create_investigation/create_observation/run_playbook. Each now binds TeamContext to the UnitOfWork and resolves via owned_or_404, yielding 404 across team boundaries. MalwareInvestigationMessageRecord is not team-scoped, so ack_message enforces ownership through its parent investigation. Tests exercise the real route endpoints for cross-team get/delete on targets, investigations, and observations.
ManagedSystemRecord is team-scoped, but the platform systems and tags routers loaded rows with session.get -- bypassing the do_orm_execute team filter -- and checked only for None, so any authenticated principal could read, update, delete, or enumerate another team's SSH systems by id. get_system_heartbeat also served a shared cached probe result before any ownership check. All eight systems handlers plus the two tags handlers now bind TeamContext to the session and resolve via a listener-filtered select (list_systems also filters its count explicitly, since the listener does not rewrite aggregate selects); heartbeat verifies ownership before the cache lookup. Both create paths now stamp team_id so non-admin-created systems remain visible to their team. Admin sessions (team_id None) bypass filtering unchanged.
The http_fetch tool checked SSRF only on the initial URL and let httpx follow redirects automatically, so an external 301 Location: http://169.254.169.254/... walked straight into cloud IMDS (and a redirect to file:// or an SSH port was equally unguarded). The tool now follows redirects manually, re-running the policy check on every hop before its socket opens, and enforces a scheme (http/https) and port (80/443/8080/8443) allow-list plus an expanded blocked-network list (adds CGNAT, 0.0.0.0/8, and IPv4-mapped IPv6 unwrapping). The redirect chain is capped at three hops. The shared build_http_client used by provider/health clients and SSH host-key reject-by-default remain follow-on #42 scope.
Remove vr's cross-module reach into the vulnerability runtime for CVE
intel. vr's cve_intel_resolver called
platform.runtime.require_module("vulnerability") then
getattr(vuln_runtime, "intel") -- a module-to-module coupling that
welded vr to vulnerability (vr could not ship without it, and
vulnerability could not restructure its intel service without breaking
vr).
Invert the direction through the platform:
- New platform/services/intel_service.py declares IntelServiceProtocol
(the fetch_cve_intel method contract; the knowledge return stays
module-owned and is consumed structurally).
- PlatformRuntime gains an intel_service slot.
- The builder collects it via _collect_intel_service, which reads each
module runtime's provides_intel_service() (platform-introspects-module
is legal; module-reaches-peer is not). First non-None wins; absence is
normal and yields None.
- VulnerabilityRuntime publishes its IntelService through
provides_intel_service().
- vr reads platform.runtime.intel_service and never names the provider.
Behavior-preserving: the same IntelService instance resolves the same
CVEs; when no module publishes one (worker started --modules vr only)
vr degrades through its existing status=error fallback exactly as
before. cve_intel_resolver.py now contains no require_module and no
getattr(vuln_runtime), satisfying the RFC-05 (e) acceptance criterion.
Tests: _collect_intel_service (published / absent / method-missing /
none-returning / empty) and the resolver reading the platform slot
(found + provider-absent). Full honesty audit stays green.
The three platform MCP bridge tools (AuditMcpBridgeTool, IDABridgeTool,
AndroidMcpBridgeTool) hard-coded a vr-prefixed tool name as a class
attribute (name = "vr.audit_mcp_bridge" etc.) and read their base URL
from a hard-coded "vr" config namespace. Both weld a platform-owned
bridge to one module.
Derive both from a constructor module_id (keyword-only, default "vr"):
- self.name = f"{module_id}.<suffix>_bridge" replaces the class literal.
- _resolve_base_url reads ConfigRegistry().get(self.module_id, <key>).
The default "vr" keeps every existing caller byte-identical: the tool
names stay "vr.audit_mcp_bridge" / "vr.ida_bridge" /
"vr.android_mcp_bridge" and the config namespace stays "vr", so there
are no callsite changes and no behavior change. A module that wants its
own namespace and tool-name prefix passes module_id explicitly (a
deliberate follow-up, not forced here).
Adds honesty rule 45 (module_prefix_in_platform_tool_name): a
class-level or self.name string literal under platform/mcp/bridges/ that
starts with a known module id is a finding; the f-string form is clean.
Rule tests (class-level, annotated, self.name fire paths; f-string,
non-bridge, non-prefixed negatives) plus a bridge test (default
preserves vr names, explicit module_id derives the name). Full honesty
audit stays green.
platform/events/domain_events.py carried a vulnerability-domain event vocabulary -- ScanStarted, ScanCompleted, FindingUpserted, FindingResolved and their five payload models -- that was never emitted or subscribed anywhere. The only references were the re-exports in events/__init__.py and two ReportService docstrings that claimed emission the method bodies never performed. Delete the four dead event classes and five dead payloads, drop them from events/__init__.py, and correct the ReportService docstrings to describe what the methods actually do (persist via PersistContract, no event emission). The platform keeps only generic infrastructure events: system lifecycle, config change, assessment lifecycle, and LLM-call accounting. Adds honesty rule 46 (platform_owns_event_vocabulary): an event class under platform/events/ whose class name or event_type literal carries a module-domain token (scan, finding, investigation, or a module id) is a finding. A module that needs to publish a workflow or entity event declares it in its own package; the platform does not host domain vocabulary. Rule tests cover class-name and event_type fire paths plus infrastructure-event, assessment, and non-events-file negatives. No new speculative event classes were added -- nothing emits them yet, so adding them would only recreate dead vocabulary. Full honesty audit stays green.
Reconciliation, reenqueue, and reaping of platform-owned task state are platform responsibilities, not custom module code. This moves the last module-side raw SQL against taskrecord / workflow_state_cursor into the platform and makes the zombie/cursor reaper a single platform-owned sweep across every track. - New platform reaper cursor_reaper.reap_zombie_tasks_and_cursors runs the four coupled maintenance statements (cancel stale-running tasks; purge orphan / terminal / __succeeded__ cursors) in one transaction. It now reaps EVERY track, not just vr -- a stale-running task is a zombie regardless of which module owns it. It is wired into the platform worker reaper cron next to sweep_orphan_crashed_cursors, with env-driven thresholds (PLATFORM_REAPER_ZOMBIE_HEARTBEAT_MIN default 10, PLATFORM_REAPER_CURSOR_BATCH_CAP default 5000 -- the historical vr values). - vr/masvs/parent_reconciler drops its private reaper step and function; it keeps only the masvs-domain parent/child orchestration. The two now-dead vr config fields (zombie_task_heartbeat_min, cursor_cleanup_batch) are removed. - investigation_lifecycle.purge_investigation_cursors centralizes the per-investigation cursor wipe; the malware reset handler and the platform reenqueue path both call it (reenqueue de-duplicated onto it). - pdf_report's active-runtime query moves to platform/tasks/runtime_stats.active_task_runtime_seconds, removing the module's psycopg import and raw taskrecord SQL. Adds honesty rule 47 (raw_sql_platform_tables): a module string constant with a FROM|INTO|UPDATE|JOIN clause against taskrecord / workflow_state_cursor is a finding, so this cannot regress. Rule tests cover DELETE/SELECT/concatenated fire paths plus module-owned-table, prose, and platform-file negatives. Full honesty audit stays green. The reaper trigger relocation and the all-track extension change worker cron behavior and need a live worker-cron smoke at merge.
The platform contract hardcoded a closed Literal of nine module-domain strategy families and a table of four domain profiles naming forensics and vulnerability domains. Both are module vocabulary the platform should not own. - ReasoningStrategyFamily changes from a Literal to str; strategy families are runtime-validated by a platform StrategyRegistry (seeded with the platform-owned "generic" family only). Adds ReasoningStrategyDeclaration. - New StrategyRegistry + DomainProfileRegistry in services/reasoning.py. _DOMAIN_PROFILES is removed; CyberReasoningEngine.resolve_domain_profile reads the DomainProfileRegistry, keeping the config-override tier and the graceful generic fallback for unregistered domains unchanged. - ModuleProtocol gains reasoning_strategies() and reasoning_domain_profiles() (default empty). The platform builder collects them from every module at load into the registries. - vr declares vulnerability_research / web_pentest / mobile_reverse; forensics declares filesystem_triage / persistence_hunt / memory_forensics / network_forensics / malware_static + the forensics profile. Profile values are reproduced verbatim from the former platform table, so resolution is behavior-preserving; malware carries no profile and uses generic as before. Tests cover the registries (seed/register/resolve/raise/clear), the module declarations, and that resolve_domain_profile returns the legacy values for registered domains and generic for the rest. Full honesty audit green. The Literal-to-str change moves strategy-family validation from parse time to the registry, and profile resolution now depends on module registration at load; a reasoning smoke on a live vr/forensics investigation is warranted at merge.
Two vr test modules failed collection because they imported symbols that RFC-03 moved to the platform: - test_vr_mcp_adapters imported _parse_command from vr.agents.tool_executor; the parser is now platform.agents.tool_execution.parse_command, re-exported as parse_command from the vr module. - test_vr_branch_manager imported the case-state merge helpers (_decode, _encode, _has_contract, _merge_case_states, merge_hypotheses, merge_rejected) from vr.agents.branch_manager; they now live in platform.agents.branch_pool. Redirect both imports. Restores pytest collection of the full suite; 77 tests pass. No production code change.
Contributor
Author
RFC-05 platform boundary settlement -- COMPLETE (
|
…dule
The shared API and platform layers resolved vulnerability-domain data by
naming the module: require("vulnerability") across dashboard, search,
systems, topology, tags, executive, findings-workflow, and the scheduled
risk-PDF task, plus a hardcoded TRACK_VULNERABILITY constant and a
finding-state vocabulary defined in the api schema layer.
- ModuleRegistry.first_with/all_with resolve a module by capability. The
14 require("vulnerability") sites now resolve by the method they call
and degrade to a graceful skip or 503 when no registered module
answers, so the boundary layer names no module.
- ModuleProtocol.scan_submission_track() lets a module declare the ARQ
track a POST /analyze scan runs on. scans.py resolves it instead of
importing TRACK_VULNERABILITY, which is removed from api/constants.
- The generic finding-triage lifecycle (new, investigating, mitigated,
verified, closed) moves from api/schemas/endpoints.py into a platform
contract at platform/contracts/finding_states.py. findings-workflow
reads the platform base and merges module workflow_definitions on top.
The FindingTransitionRequest state validator moves out of the Pydantic
model into the handler, which validates against the resolved machine.
- honesty rule 48 platform_names_module flags a boundary-guarded file
(api/, platform/, storage/) that names a feature module in a
.require(...) / .require_module(...) call.
require("<module>") and from aila.modules.<x> in platform+api: 0.
TRACK_VULNERABILITY: 0.
…a platform claim Two workers could each read the same accepted outcome as dispatchable and both run its handler, double-shipping the artifact (finding, child investigation, report). The malware dispatcher had no state gate at all, widening its window. - platform/services/outcome_dispatch.py: claim_outcome_for_dispatch opens a unit of work, selects the outcome FOR UPDATE, runs an optional domain guard, and flips PENDING -> CLAIMED in the same transaction. A concurrent dispatcher's FOR UPDATE blocks until the claim commits, then observes CLAIMED and backs off, so exactly one caller proceeds. - OutcomeDispatchStatus gains CLAIMED. The column is varchar(16), so no migration is required. - vr and malware outcome dispatchers claim before running any handler. vr supplies its state == approved gate as the claim guard (behavior preserved); malware supplies a kind guard that refuses the claim for outcomes with no handler so they stay pending. - Stale CLAIMED rows left by a dispatcher that crashed post-claim are reset by the RFC-07 resilience reaper (follow-on). Tests: 5 claim tests including a concurrent-race single-winner assertion; 23 vr and 21 malware existing dispatcher tests remain green.
…RFC-03 Phase 4b prep) The two module ToolExecutors have zero unit coverage of execute(); RFC-03 Phase 4b will lift ~750 shared lines into a platform ToolExecutorBase. This adds a golden-master net for the VR pre-dispatch gating so that extraction can be proven behavior-preserving on the shared paths: - malformed command -> error result, empty server/tool, bridge NOT called - tool without a server prefix -> error result, bridge NOT called - unknown tool with no adapter -> error result, bridge NOT called Each asserts the ToolExecutionResult contract and that no fake bridge forward() was awaited. The divergent hard-block, circuit-breaker, and success+observation paths need deep message-history seeding and remain covered by verbatim lift plus live smoke at merge.
…ing (RFC-03 Phase 4b prep) Golden-master coverage of the malware ToolExecutor.execute() gating so the Phase 4b platform extraction is provably behavior-preserving on the shared and malware-divergent paths: - malformed command -> error result, bridge NOT called - _AGENT_ALLOWED_SERVERS allowlist rejects a non-allowed server (audit_mcp) before the adapter lookup, bridge NOT called (the malware-divergent path Phase 4b turns into an allowed_servers constructor input) - unknown ida_headless tool with no adapter -> error result, bridge NOT called
Both module researchers carried a byte-identical _cached_read_prompt + _load_prompt pair differing only in the fallback base filename and the error class. This lifts the single file-backed resolution path into a platform PromptRegistry so no module reimplements it, and gives RFC-09's later steps (DB overrides, immutable versions, release aliases, per-investigation pins) one place to grow. - platform/prompts/registry.py: PromptRegistry(prompt_dir, fallback_base) .load(strategy_family, persona_voice) resolves system_<strategy-leaf>.md (falling back to the module base filename), prepending persona_<voice>.md when present. Behavior-identical to the prior module logic; file-backed only, no database and no versioning at this step. - vr and malware researchers delegate _load_prompt to a module-level registry, translating PromptNotFoundError into their own error type. The duplicated _cached_read_prompt and its now-unused functools import are removed from both. Tests: 8 unit tests (strategy-specific base, fallback, dotted-leaf extraction, missing-prompt error, persona prepend, case-insensitive persona filename, missing persona, malware fallback name). Verified both modules resolve their real base prompts unchanged (vr 33728 bytes, malware 130247 bytes).
…tent hash Each LLM cost record now carries the sha256 of the resolved system prompt template that produced the call, so cost (and later quality) is attributable to the exact prompt content. This is the observability foundation the later RFC-09 steps (DB overrides, versions, aliases) build on. - correlation.py: CorrelationContext gains prompt_content_hash; correlation_scope accepts it; current_prompt_content_hash() reads it. Threaded through the same ContextVar that already carries the investigation/branch/turn join keys, so no call signatures change. - cost_record.py: LLMCostRecord gains a nullable, indexed prompt_content_hash column. Migration 086 adds it (IF NOT EXISTS guarded); create_all keeps tests and fresh installs in sync. - cost.py: persist_cost_record reads current_prompt_content_hash() and tags the record. - vr and malware researchers compute sha256 of the resolved system prompt and pass it into correlation_scope for the turn. The semver prompt_version column is intentionally deferred to step 4, where immutable versions actually exist; content_hash is the stable identity now. Tests: 2 unit tests (tagged from correlation; None without a scope).
The DB-backed versioning layer RFC-09 needs for deploy/rollback without a code release: immutable content-hashed prompt versions, mutable release aliases (candidate/staging/production), and an append-only alias-change audit log. - version_models.py: prompt_versions (unique per key+version and key+content_hash), prompt_aliases (one pointer per key+alias), prompt_alias_changes (audit). Registered on SQLModel.metadata via db_models re-import. Migration 087 creates them (IF NOT EXISTS guarded); create_all keeps tests and fresh installs in sync. - version_store.py: PromptVersionStore.register (immutable, content-hash-deduplicated, monotonic 1.0.N versions), resolve (by explicit version or by alias pointer, None -> caller falls back to the file base prompt), set_alias (audited pointer flip = deploy/rollback, raises PromptVersionNotFoundError for an unknown version). This is the store layer. Wiring the file-backed PromptRegistry to resolve through it and pinning the resolved version per investigation at setup (steps 2-3) are the follow-on increments; both need the sync _load_prompt path to become async, so they land as a separate focused change. Tests: 5 unit tests (content-hash idempotent register, monotonic versioning, resolve by version/alias/miss, alias flip + audit log, missing-version guard).
Phase 6b (5247edc) added OutcomeDispatchStatus.CLAIMED on the backend and the outcome API serializes it, so a response can carry "claimed" while a dispatch handler runs. Both module frontend OutcomeDispatchStatus unions omitted the value, so the Record<OutcomeDispatchStatus, ...> color and label maps had no key for it. On the malware outcome detail page the maps were indexed with no fallback, rendering a blank pip and empty label for a claimed row; TypeScript could not catch it because the value arrives via an untyped API cast. - vr + malware types.ts: add "claimed" to the OutcomeDispatchStatus union. This makes the type checker enforce that every dispatch-status map covers it. - vr + malware InvestigationDetailPage dispatchColor: claimed -> "info" (transient, non-terminal). - malware OutcomeDetailPage DISPATCH_COLOR: claimed -> var(--status-running); DISPATCH_LABEL: claimed -> "Dispatching". Add a defensive fallback at the index site so any unmapped status degrades gracefully. type-check clean for @aila/vr-frontend and @aila/malware-frontend.
The platform zombie reaper (reap_zombie_tasks_and_cursors) cancelled any running task whose COALESCE(heartbeat_at, started_at) was older than the threshold, which treats a NULL heartbeat as evidence of death. That is wrong for single-shot tool tasks: run_function_ranking and deep_audit run for many minutes with heartbeat_at NULL while healthily awaiting one HTTP response, so a >10-minute audit run was cancelled mid-flight. This is the exact false positive that _sweep_orphan_running_tasks already guards with reap_null_heartbeat=False; the reaper reintroduced it for every track when the reaping moved to the platform cron. A zombie is a task that reported a heartbeat and then went silent. The cancel predicate is now `heartbeat_at IS NOT NULL AND heartbeat_at < threshold`. A task that never beat is left alone; its lifecycle is bounded by the ARQ job timeout. The change is track-agnostic (no module naming), so it keeps the platform boundary the reaping move established. Tests: tests/platform/test_cursor_reaper_zombie.py -- stale-beat reaped, null-heartbeat exempt (the fix), fresh-beat exempt.
Phase 6b closed the dispatch double-ship TOCTOU by flipping an outcome to CLAIMED under FOR UPDATE, but a dispatcher that wins the claim then crashes before writing the terminal DISPATCHED/FAILED status stranded the row at CLAIMED forever: every retry saw CLAIMED and skipped, so the outcome never dispatched. That traded double-ship safety for a new lost-dispatch failure mode with no recovery path. Add claimed_at to the outcome record. The claim now stamps it and treats a CLAIMED row as a live claim only while claimed_at is within a reclaim window (900s, far longer than any real dispatch). A CLAIMED row older than the window is a stranded claim and the next dispatch attempt reclaims it, so a crashed dispatch self-heals without a separate reaper. DISPATCHED stays terminal and is never reclaimed. The claim also splits the skip reasons that used to collapse into one: DISPATCHED -> "already_dispatched", a fresh CLAIMED -> "claim_in_progress" (both surfaced in logs/results only; no caller branches on the string). - OutcomeRecordBase: nullable claimed_at column; migration 088 alters vr_investigation_outcomes and malware_investigation_outcomes (create_all keeps tests and fresh installs in sync). - claim_outcome_for_dispatch: reclaim-window check + claimed_at stamp. Tests: 3 new (stale claim reclaimed, fresh claim held, dispatched terminal); existing claim + 26 module dispatcher tests still green.
PromptVersionStore.register derived the next version suffix from the count of surviving rows (len(prior)), so archiving an old version made a later register emit a suffix that collides with an existing row and violates the key+version unique constraint. The next suffix is now one past the highest suffix ever issued for the key, not the row count, so decommissioning an old version never reuses (or collides with) an already-issued number. Test: register three, archive the oldest, register a fourth -- it is 1.0.3, not the count-based 1.0.2 that would collide with the survivor.
…step 1) Step 1 tags each LLM call with the sha256 of the resolved system prompt so cost and evidence are attributable to the exact prompt content. 086 added that tag to llm_cost_records; the audit seal -- the HMAC chain-of-custody record -- was left untagged, so the seal could not be tied to a prompt version. Tag it from the same correlation ContextVar the cost writer reads. - AuditSealRecord: nullable indexed prompt_content_hash column. - seal step: reads current_prompt_content_hash() and stamps the record. - migration 089 alters auditsealrecord (create_all keeps tests and fresh installs in sync). Tests: seal carries the hash when a correlation scope is set; NULL for a call outside an agent turn.
RFC-05 Phase 6 intended the task-progress stream-key format to stay private to ProgressStream, with callers routing through a public method. That half never shipped: platform/workflows/log.py and, worse, the forensics module emitter read ProgressStream._KEY_FMT directly to build a custom XADD, a module reaching into a platform class's private attribute. Add ProgressStream.stream_key(task_id), the single public accessor for the key. emit / catchup / stream_events and both external XADD sites now route through it, so _KEY_FMT has exactly one reader. The forensics module no longer touches any platform private (it still uses the public get_redis service). Behavior is unchanged -- the key value is identical. - whitelist stream_key against do_nothing_wrapper: it is the deliberate encapsulation point; inlining it reinstates the private-attr coupling. - fix a stale assertion in the same test file (progress.__all__ already carried MAX_STREAM_LIFETIME_S). Tests: stream_key accessor unit test; existing emit/catchup/stream key assertions still green.
Contributor
Author
Design-fidelity audit of shipped RFC work + remediationAudited the code shipped on Class-A / real defects -- fixed this pass
All six carry tests and pass compile + ruff + honesty + targeted pytest. Alembic head is now Deliberate divergences -- reconciled, not revertedThese differ from the RFC text on purpose; the outcome the RFC targeted is achieved and the alternative is sound:
Deferred -- tracked, with rationale
|
…dule purge_arq_jobs_for_investigation resolves the Redis URL as param -> env -> ConfigRegistry. The ConfigRegistry branch imported it from aila.platform.services.config_registry, which does not exist (the class is aila.storage.registry.ConfigRegistry, as every other caller imports it). The ImportError was caught by a silent `except`, so redis_url fell to None and the purge returned a zero result without touching Redis whenever AILA_PLATFORM_REDIS_URL was not exported (start.sh does not export it). Every terminal / pause / cap-sweep purge routed through this platform primitive silently no-op'd, leaving stale queued ARQ jobs. The workflow STATUS_LOCKED guard prevented the stale jobs from causing data corruption, so the bug was invisible. Import ConfigRegistry from aila.storage.registry and log the fallback failure instead of swallowing it. Tests: url resolves through ConfigRegistry when env is unset and reaches the Redis client; an empty config yields a clean zero result with no client.
…ache ResolverAgent._attempt_resolution issued a direct llm_client.chat with no request key. resolve() is driven by the forensics resolution workflow state under the ARQ-backed DurableStateMachine, so a worker crash + ARQ redelivery re-paid the model for the same question -- the exact double-pay RFC-03 Phase 2 closed at the vr/malware agent sites. This module agents/ call was never adopted. Route the call through platform idempotent_llm_call (method="chat") keyed on project_id + the full messages, so a redelivered resolution replays the cached answer instead of calling the model again. Tests: an identical resolution calls the model once (second served from cache); a distinct question calls it again.
Contributor
Author
Design-fidelity audit, round 2: RFC-01, RFC-02, RFC-04, RFC-03 phases 1-5Extended the audit to the remaining shipped RFC surfaces. Four independent read-only reviews plus direct verification. RFC-01 (platform data-model bases): faithfulZero findings. Every base column has a matching Alembic migration; cross-module constraint names are collision-free (all derived from Class-A defects fixed this round
Class-A deferred, with detailed root cause -- pause/resume cursor keying (RFC-02)
This is NOT a one-line fix and a partial fix breaks a working feature: the loop reads the cursor via RFC-02 was otherwise faithful: fail-closed/open policy (queue defer, verify re-raise, pipeline fail-mode-aware, SSE display-only), JSON-serializable task/execute kwargs, merge-vs-add discipline, the state set, and the Class-C incompleteness -- reconcile (RFC-04, RFC-03)
Deferred tooling
|
… base RFC-03 Phase 4b, helper-extraction step. The vr and malware tool executors carried byte-identical copies of nine helpers -- tool-result persistence, the repeat-failure / error-class / malformed circuit-breaker counters, the read-tool resolver, and the observable-merge -- differing only in the module-specific message / branch record types. New platform/agents/tool_executor.py::ToolExecutorHelpersBase owns those nine helpers, parameterized on self._message_model / self._branch_model (and _MALFORMED_TOOL_RUN_MARKER / _MAX_OBSERVABLES as shared constants). Both module ToolExecutors now subclass it, set the two record types in __init__, and drop their local copies. The divergent parts stay module side: the execute() dispatch loop, the survey-streak hint text, pivot-history parsing, VR index-correction, and malware observation recording. Behavior-preserving: the lifted helpers differed only by the record type that is now injected. The execute() merged dispatch itself (domain divergent) remains module-side pending its own change. Verified: compile + ruff + honesty clean; the vr and malware ToolExecutor characterization nets (6), outcome dispatcher tests (26), and the full agents test directory pass; both executors resolve the inherited helpers at runtime.
The ToolExecutor characterization nets covered only the pre-dispatch gating paths. The nine helpers just lifted to ToolExecutorHelpersBase had no direct test, so the extraction rested on structural checks alone. Add golden-master tests that exercise the lifted helpers through both module injections: - _count_prior_failures: pairs a tool_call with a same-args error and counts repeats; a different-args pair does not count (vr + malware). - _count_total_malformed: counts malformed-marker errors across the window, ignoring a non-malformed error (vr). - _persist_result_and_observables: one call writes the result message AND merges the observables delta into the branch case_state (vr + malware). - _apply_observables_delta: merges a delta and caps at _MAX_OBSERVABLES while always keeping _directive.* keys (vr, pure helper). These pin the circuit-breaker and result-persistence behavior against the injected record types, strengthening the net for the deferred execute() merged-dispatch step. Agents suite: 12 pass (was 6).
The prompt version store shipped without a consumer -- nothing resolved
through it, so a registered version could never influence a turn. Wire it
into both researchers' _load_prompt: resolve the "production" alias for a
{module}/{strategy}/{persona} key first, and fall back to the file registry
when no version is deployed. The file is the baseline; the store is the
deploy override, so a store fault degrades to the file rather than blocking
a turn. An operator can now register a new prompt body and flip the
production alias to deploy a prompt change without a code release, which is
the point of RFC-09 versioning.
_load_prompt becomes async (its single caller per module is already async);
the file-loading unit tests are converted to async and take the test_db
fixture so the store resolve is deterministic.
Behavior-preserving with an empty store: resolve returns None, the file is
loaded exactly as before.
Tests: registered+aliased production version overrides the file; empty
store falls back to the file (vr). 103 researcher/prompt/agent tests pass.
The version store and the turn-time resolve path were in place, but there
was no operator way to get a version into the store or flip a release
alias, so the deploy mechanism could not be exercised without direct store
calls. Add a god-tier admin router that closes the loop:
POST /admin/prompts/versions register an immutable version
GET /admin/prompts/versions?key= list registered versions
PUT /admin/prompts/aliases point an alias at a version (deploy)
GET /admin/prompts/aliases?key= list alias pointers
An operator now registers a new prompt body and flips the "production"
alias; the researchers resolve that alias on the next turn (already wired),
so a prompt change deploys without a code release -- the RFC-09 goal. All
endpoints require god-tier admin (team_id=None) since prompt versions are
platform-wide; every endpoint is rate-limited. Unknown-version alias flips
return 404.
Adds PromptVersionStore.list_versions / list_aliases for the read surface.
Tests: register + content-hash dedup + list; alias deploy + rollback;
unknown-version 404; reader-role 403.
Contributor
Author
Session verification report -- audit remediation, RFC-03 Phase 4b, RFC-09Full gate + test run for the work landed on Verification (all green)
Backend test chunks: platform (49) + agents/researchers/prompts/progress (128) + api/dispatchers/seals (41). Chunked deliberately -- the full non-e2e suite is run in slices here. Alembic is a single linear head What shippedRFC-09 prompt registry + versioning -- COMPLETE and operator-usable end to end
Design-fidelity audit -- two rounds across RFC-01/02/03/04/05/09, eight real defects fixed
RFC-03 Phase 4b -- shared ToolExecutor helper base
Not in this session (needs a live smoke or a focused greenfield session)RFC-03 4b |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release 0.3.0 -- security, correctness, and reliability hardening
Rolls up the work landed on
devsince 0.2.1. Full detail inCHANGELOG.mdunder[0.3.0].Highlights
Behavior change requiring caller action
Not breaking
Verification
Opening for review; not for auto-merge.