feat(compliance): retention + GDPR erasure/residency + SOC 2 posture (Step 6.10) - #162
Merged
Merged
Conversation
…(Step 6.10)
Phase-6 capstone — adds the three compliance pieces on top of the controls
the platform already ships (audit / ACL / PII / BYOK / SSO / quotas).
- New rag-compliance package (deps rag-core + rag-observability only):
RetentionEnforcer drives tenant-scoped purge_* (purge = per-class
retention windows; erase_tenant = GDPR right-to-erasure); compliance_posture
+ residency_ok are pure.
- Retention is a *capability on the existing stores*, not a new SPI:
non-abstract purge_before / purge_tenant (default no-op) on FeedbackStore +
ProvenanceStore, with dry_run in the SPI so a preview counts-without-deleting
uniformly (ProvenanceStore has no list). The audit hash chain is never
purged in place (it would break tamper-evidence) — audit retention is the
6.6b WORM export; audit_days is advisory.
- POST /v1/compliance/erase erases the calling tenant's data (scope from the
principal, never the body); dry-run by default, real delete needs
dry_run=false AND confirm=true.
- Per-tenant data_region (resolved onto TenantSettings via the 6.1 resolver)
enforced at ingest against cfg.compliance.region → ResidencyViolationError
(403) + compliance.residency_violation. Opt-in, single-deployment assertion.
- GET /v1/status/compliance serves a CompliancePosture (controls + region +
retention) computed from the build_app flags — the live backing for the
SOC 2 / GDPR control-mapping docs (each criterion → a control → whether it's
on in this deployment, so the mapping is checkable).
New core types DataClass / RetentionPolicy / ErasureResult / CompliancePosture
+ ComplianceError (400) / ResidencyViolationError (403); cfg.compliance +
tenants[].data_region / retention_days; PII-free compliance.* events; ragctl
compliance report / demo. dist schemas/openapi/rag.schema regenerated.
~30 tests; all gates green: ruff, mypy --strict (325 files), RAG001, full
pytest (excl integration), schema/openapi/config-drift, policy-coverage,
log-schema/event-registry/PII.
Docs: docs/compliance/{soc2-control-mapping,gdpr-mapping}.md,
docs/reference/compliance.md, docs/architecture/compliance.md,
docs/adr/ADR-0042-compliance-posture.md, docs/README.md. TRACKER: 6.10 done,
Phase 6 complete (74/84).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Step 6.10 — Compliance posture (the Phase-6 capstone): SOC 2 Type II control mapping, GDPR data-residency config, and data-retention policies. New
rag-compliancepackage.The platform already ships the controls an auditor maps to (audit / ACL / PII / BYOK / SSO / quotas / breakers / guard); 6.10 adds the three missing pieces:
RetentionEnforcer(rag-compliance) drives tenant-scopedpurge_*:purgeapplies per-class retention windows,erase_tenantis the right-to-erasure primitive. Retention is a capability on the existing stores, not a new SPI — non-abstractpurge_before/purge_tenant(default no-op) onFeedbackStore+ProvenanceStore, withdry_runin the SPI so a preview counts without deleting uniformly (ProvenanceStorehas nolist). The audit hash chain is never purged in place (it would break tamper-evidence) — audit retention is the 6.6b WORM export;audit_daysis advisory.POST /v1/compliance/eraseerases the calling tenant's data (scope from the principal, never the body); dry-run by default, real delete needsdry_run=falseandconfirm=true.data_region(resolved ontoTenantSettingsvia the 6.1 resolver) enforced at ingest againstcfg.compliance.region→ResidencyViolationError(403) +compliance.residency_violation. Opt-in; a single-deployment assertion, not multi-region routing.GET /v1/status/complianceserves aCompliancePosture(controls + region + retention) computed from the build flags; the mapping docs map each Trust Service Criterion / GDPR article to a control, and the posture reports whether it's on in this deployment — so the mapping is checkable, not just prose.New core types
DataClass/RetentionPolicy/ErasureResult/CompliancePosture+ComplianceError/ResidencyViolationError;cfg.compliance+tenants[].data_region/retention_days; PII-freecompliance.*events;ragctl compliance report/demo. Phase 6 closes (74/84).Deferred: subject-level (vs tenant-level) chunk/document erasure (needs an index delete-by-document path), an admin retention-sweep endpoint (today the sweep is
ragctl/cron; erasure is the API), multi-region routing, automated audit-evidence bundles.Documentation
docs/compliance/soc2-control-mapping.md— TSC → feature, with the posture key for each controldocs/compliance/gdpr-mapping.md— GDPR article-by-articledocs/reference/compliance.md+docs/architecture/compliance.mddocs/adr/ADR-0042-compliance-posture.md; docs/README + TRACKER (Phase 6 ✅)Test plan
RetentionEnforcerpurge / erase / dry-run / no-window / no-stores;compliance_posture;residency_okpurge_before/purge_tenant(age, tenant-scope, dry-run, isolation) on Feedback + Provenance/v1/status/complianceposture;/v1/compliance/erase(dry-run default, two-flag delete, 401); residency-violation ingest 403; residency off-by-defaultragctl compliance report/demo🤖 Generated with Claude Code