Evidence Core v1: provider-native drift adjudication baseline - #1
Open
EdwinJdevops wants to merge 24 commits into
Open
EdwinJdevops wants to merge 24 commits into
EdwinJdevops wants to merge 24 commits into
Conversation
- fix all current CLI Ruff findings so the CLI test job can execute pytest - pin CLI CI pytest/ruff versions to the repository-verified toolchain - make the wrong-ExternalId STS probe use the same role session name and duration as the successful AssumeRole path - assert the negative-control request shape in tests so unrelated trust-policy conditions cannot masquerade as ExternalId enforcement This is the safety baseline before replacing the legacy drift detector with the evidence-core architecture.
Add Evidence Bundle v1 and a fail-closed Terraform/OpenTofu plan analyzer built around provider-native resource_drift semantics. Key invariants: - preserve absolute Terraform/OpenTofu resource addresses verbatim - omit all raw before/after values from evidence output - retain only changed JSON-pointer paths and sensitivity paths - reject errored plans and unsupported major JSON formats - skip non-managed entries explicitly rather than inventing drift semantics Document the architecture decision, rejected alternatives, security boundary, migration sequence and release gate in ADR 0001. This does not replace the legacy scan path yet; replacement is gated on real provider-generated adversarial fixtures.
- scope scan lookup by authenticated organization at the SQL query boundary - scope serialized findings to both the authorized scan and its workspace - add an API regression test covering owner access, cross-tenant 404, unauthenticated 401, and deliberately inconsistent cross-workspace finding data - fix the evidence-model Ruff import-format failure found by CI - tighten verify-role wording so it states exactly what the negative ExternalId control proves This closes the scan-read isolation P0 without relying on UUID secrecy as an authorization boundary.
Add a provider-backed contract harness that applies hashicorp/local 2.9.0 resources in a nested module, including count, for_each and sensitive data, mutates three resources outside IaC, then generates a real refreshed plan and feeds its resource_drift to DriftGuard. The verifier requires: - exact preservation of provider-native absolute addresses - correct count and for_each indexes - no false drift on untouched sibling instances - retention of sensitive-path metadata - no sentinel raw values in serialized Evidence Bundle output CI runs the contract independently against Terraform 1.16.2 and OpenTofu 1.12.6. Raw show -json output remains in an ephemeral temp directory and is never printed or uploaded. GitHub Actions are pinned to immutable commit SHAs and workflow permissions are explicitly read-only.
Add `python -m backend.evidence <plan.json>` as the first local-first analysis surface. It reads raw Terraform/OpenTofu plan JSON only on the local machine and emits or writes the redacted Evidence Bundle; tests assert sensitive sentinel values never appear in stdout, output files, or adjudication errors. Document the real provider-native validation from CI run 34838882930, including exact Terraform/OpenTofu versions, fixture topology, observed resource_drift semantics, redaction boundary, proven invariants and explicit non-claims. The packaged `driftguard analyze` command is intentionally deferred until the evidence contract stabilizes, avoiding premature shared-package restructuring.
Owner
Author
Validation checkpoint — provider-native contract establishedCommit Provider-generated contract results:
The full experiment and non-claims are recorded in Current HEAD |
Move Sequence to collections.abc and remove an unused IaCEngine import. These are the two exact Ruff failures from CI run 34839136526; no lint rules are weakened or suppressed.
Owner
Author
Foundation gate complete at
|
EdwinJdevops
marked this pull request as ready for review
September 14, 2026 11:40
Design review against the official Terraform/OpenTofu JSON contract found two evidence-model hazards: - lists, sets and tuples all lower to JSON arrays, so element indexes are not generically trustworthy without provider schema; array changes now collapse to the parent JSON pointer - deposed change objects require address + deposed key for unique identity; Evidence v1 now preserves deposed and previous-address metadata Also reject state JSON as the wrong input type, preserve provider-native applyable/complete flags and after_unknown paths, and add regression tests for each invariant. ADR 0001 is updated with the exact rationale and failure semantics.
- add workspace-scoped deterministic SHA-256 incident identity for Evidence Core - add incident, occurrence, and exactly-once reconciliation tables without mutating the legacy drift_findings schema - resolve incidents only from complete plans; incomplete/deferred plans cannot manufacture recovery - make scan replay idempotent and fail closed when replayed evidence differs - preserve redaction: lifecycle stores identity and path metadata, never raw before/after values - reserve a stable full-fingerprint remediation branch for future PR deduplication - register ORM models deterministically for create_all - add lifecycle regression tests and ADR 0002 The legacy collector-based scan path and its GitHub PR behavior remain unchanged in this commit.
- add one workspace-scoped EvidenceCursor watermark - record stale/ambiguous scans without applying lifecycle transitions - prevent late workers from falsely resolving or reopening newer incident state - require an explicit timezone-aware observation timestamp - validate all loaded incident states before mutation - preserve exactly-once replay semantics including observation time - fix the lifecycle-test Ruff import-format failure - document ordering and stale-scan semantics in ADR 0002
- add Alembic 1.20 migration history with legacy baseline and Evidence lifecycle revision - normalize database URLs through one shared function - add strict bootstrap classification for fresh, recognized legacy, versioned, and ambiguous schemas - preserve legacy data during migration and reject unknown unversioned layouts - add migration regression tests including alembic metadata parity checks - document production cutover boundary in ADR 0003 Runtime production scan behavior remains unchanged; API startup cutover is intentionally deferred until migration CI is green.
- persist sanitized submission provenance and source-execution idempotency - add migration 0003 for evidence submission records - reject conflicting retries and preserve one scan per submission key - expose tenant-scoped evidence ingestion behind an explicit feature flag - reject raw/unknown plan change-value fields at validation boundary - reconcile lifecycle atomically with submission persistence - add HTTP regressions for disabled mode, tenant isolation, replay safety, conflict handling, and redaction - document ingestion trust boundary in ADR 0004 Legacy collector-based scanning remains unchanged and provider-native ingestion is disabled by default.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The legacy detector cannot be the long-term production truth engine. It compares Terraform state against partial handwritten AWS collectors, which can create false modified/deleted findings and loses complete Terraform instance identity.
This PR starts the redesign without switching production traffic prematurely.
Commit 1 — safety baseline
Commit 2 — Evidence Core v1
EvidenceBundleresource_driftas provider-native drift semanticsbefore/aftervalues from persisted evidenceExplicitly not done in this PR
Required before merge/production cutover
count,for_each, sensitive values, deletion and provider failure.This PR should remain draft until those review gates are explicitly addressed.