Skip to content

12-roadmap.md

Latest commit

 

History

History
285 lines (195 loc) · 56.8 KB

12-roadmap.md

File metadata and controls

285 lines (195 loc) · 56.8 KB
Raw

12 — Roadmap

Version History & Milestones

v1.0 — General Availability ✅ released

Theme: Core platform — everything needed for a team to govern database access end-to-end.

Scope

Proxy & Execution

  • Full SQL proxy with bundled drivers for PostgreSQL 13+, MySQL 8+, MariaDB, Oracle, and Microsoft SQL Server — plus admin-uploaded custom JDBC driver JARs for any other relational engine
  • JSqlParser-based query validation and classification
  • Per-datasource HikariCP connection pool management
  • Schema / table allow-listing enforcement at AST level
  • Row cap enforcement via JDBC setMaxRows()
  • DDL blocking by default (opt-in via can_ddl permission)

SQL Editor

  • CodeMirror 6 editor with SQL dialect-aware highlighting
  • Schema autocomplete (tables, columns)
  • SQL formatter (Ctrl+Shift+F)
  • Real-time AI hint panel with gutter annotations

Review Workflows

  • Configurable review plans (per datasource)
  • Multi-stage sequential approval chains
  • AI review → human approval pipeline
  • Auto-approve reads option
  • Approval timeout / auto-reject
  • Cancel pending query (submitter)

AI Query Analysis

  • OpenAI API adapter (GPT-4o)
  • Anthropic API adapter (Claude Sonnet)
  • Ollama adapter (self-hosted LLMs)
  • Risk scoring (0–100), risk level (LOW/MEDIUM/HIGH/CRITICAL)
  • Missing index detection
  • Anti-pattern flags (SELECT *, no WHERE, no LIMIT)
  • Syntax improvement suggestions

Access Control

  • 4-role RBAC (ADMIN, REVIEWER, ANALYST, READONLY)
  • Per-user datasource permissions with granular flags
  • Time-limited access grants (expires_at)
  • Schema/table scoping per permission record

Audit

  • Append-only metadata audit log
  • Full event coverage (see audit action types in data model doc)
  • Searchable / filterable audit log UI

Notifications

  • Email (SMTP / JavaMail + Thymeleaf templates)
  • Slack (Incoming Webhooks + Block Kit)
  • Webhooks (HMAC-SHA256 signed, with retry)

Deployment

  • Docker Compose (single command startup)
  • Helm 3 chart for Kubernetes
  • Horizontal Pod Autoscaler support
  • Spring Boot Actuator health / readiness probes

Authentication

  • JWT RS256 with refresh token rotation
  • Redis-backed token revocation
  • SAML 2.0 SP-initiated and IdP-initiated SSO
  • Auto-provisioning of users from SAML assertions
  • SAML attribute → role mapping
  • OAuth 2.0 / OIDC sign-in with built-in templates for Google, GitHub, Microsoft, and GitLab; additional providers configurable via DB-driven oauth2_config rows

v1.1 ✅ released

Theme: Productivity and operations polish.

  • Dynamic JDBC driver loading — drivers resolved from Maven Central on demand instead of bundled in the fat JAR (AF-10)
  • Datasource creation wizard — visual type-selection step (Postgres, MySQL, MariaDB, Oracle, MSSQL) with logos and prefilled defaults (AF-11)
  • Query scheduling — submit an approved query to execute at a future datetime (AF-345)
  • Bulk approval UI — reviewers can approve/reject multiple queries in one action (AF-346)
  • AI analysis history dashboard — trend charts: average risk score over time, most flagged query types, most active users (AF-347)
  • Schema explorer ER view — basic entity-relationship diagram rendered from introspected schema (AF-348)
  • Enhanced schema exploration — searchable object tree (filter across schemas/tables/columns) plus on-demand, RLS- and masking-aware sample-row previews per table, across every engine (AF-443)
  • Review plan templates — pre-built plans (e.g. "Strict — all writes need 2 approvals", "Lenient — reads auto-approved") (AF-349)
  • Audit log CSV export (AF-350)
  • User invitation flow — invite users by email instead of admin-created accounts (AF-276)

v1.2 ✅ released

Theme: Integrations and reviewer experience.

  • Reviewer Access — user groups, per-datasource reviewer scoping (users or groups), and SSO group → role/group mapping so teams only see the queues they own (AF-353)
  • Read replica routing — admin can configure a read replica endpoint; SELECT queries are automatically routed there, leaving the primary for writes (AF-360)
  • Query result diffing — for repeated runs of the same query, show a diff of rows_affected and execution time vs previous run (AF-361)
  • Slack bot approve/reject — reviewers can approve or reject directly from the Slack message using Slack Interactive Components (OAuth app, not just webhooks) (AF-362)
  • PagerDuty integration — built-in PagerDuty channel type for CRITICAL risk or REVIEW_TIMEOUT events (AF-363)
  • Query templates library — save frequently used queries as templates, share across team (AF-364)
  • Datasource health dashboard — connection pool stats, query volume, average execution time per datasource (AF-365)

v1.3 ✅ released

Theme: Fine-grained access control and data protection.

  • Just-in-time (JIT) time-bound access requests — users self-request temporary, scoped datasource access (e.g. write for 4 hours); granted on approval and auto-revoked on expiry by a clustered scheduler (AF-378)
  • Policy-as-code routing engine — attribute-based rules decide query routing (auto-approve, auto-reject, require N approvers, escalate) from query type, referenced tables, AI risk, role / group membership, and time-of-day (AF-379)
  • Context-aware routing conditions — routing policies also match on submission client context: source IP / CIDR, user-agent, time-since-last-approval, and CI/CD origin (API key or X-AccessFlow-CI header), failing closed on missing context so off-network queries escalate instead of auto-approving (AF-446)
  • Row-level security policies — per-user/group/datasource row predicates injected at the AST layer so users see only the rows they are authorised for (AF-380)
  • Dynamic data masking policies — per-column masking strategies (full, partial, hash, email, format-preserving) with role/group-based reveal conditions, extending today's static restricted_columns masking (AF-381)

v1.4 — AI enhancements ✅ released

Theme: Deeper, configurable AI integrations.

  • Custom / OpenAI-compatible AI provider — point the OpenAI adapter at any OpenAI-wire-compatible endpoint via an admin-supplied base URL (AF-330)
  • Hugging Face AI provider — built-in provider using the Hugging Face Inference Providers router, with meta-llama/Llama-3.3-70B-Instruct as the default model (AF-331)
  • Editable analyzer prompts — per-ai_config system-prompt override (validated to contain {{sql}}), with the default template served to the admin UI (AF-332)
  • Langfuse integration — per-org LLM-call tracing + managed analyzer prompts (AF-333)
  • Text-to-SQL generation — draft SQL from a natural-language prompt, still submitted through the full review pipeline (AF-335)
  • RAG knowledge base — per-AI-config retrieval-augmented generation: admins attach knowledge documents (in-app pgvector or external Qdrant) that are embedded and injected into risk analysis and text-to-SQL prompts (AF-336)

v2.0 ✅ released

Theme: Beyond relational databases — plus native client access and continuous governance.

  • MongoDB support — query governance for MongoDB (find, update, delete, aggregation pipeline review)
  • Couchbase support — SQL++ (N1QL) query governance through the second on-demand engine plugin (AF-412): classification onto the standard approval workflow, WHERE-splice row-level security, field masking, and scope/collection introspection
  • Redis read-access governance — audit and optionally require review for Redis GET/SCAN/keys operations
  • Text-to-query for NoSQL — extend text-to-SQL (AF-335) to every NoSQL engine via an engine-language-aware generation prompt: a natural-language prompt drafts the engine's native query (MongoDB shell/JSON, Cypher, CQL, Elasticsearch Query DSL, redis-cli), still submitted through the full review pipeline (AF-439)
  • Behavioral anomaly detection (UBA) — a clustered-safe scheduled job builds rolling per-(user, datasource) baselines from audit_log metadata (never query result data) and flags out-of-pattern activity (query volume, off-hours activity, distinct/new tables, query types, rows returned, error rate) via z-score detection with a robust IQR fallback. Each anomaly carries an optional AI natural-language explanation, raises the anomaly_detected routing condition to escalate the flagged user's next query, fans out a notification (incl. PagerDuty), and surfaces on the /admin/anomalies admin dashboard where it can be acknowledged or dismissed (AF-383)
  • Break-glass / emergency access — a distinct submission mode gated by a per-user/per-datasource can_break_glass permission (required for everyone, including admins; time-boxed via the grant's expires_at). It skips pre-approval and executes immediately through all the usual proxy guards (allow-list, masking, row-security, row caps), compensating with: instant fanout to all org admins (incl. PagerDuty), a prominent QUERY_BREAK_GLASS_EXECUTED audit row, and a mandatory retro-review (tracked in break_glass_events, surfaced on the /admin/break-glass log) that an admin — never the submitter — must acknowledge (BREAK_GLASS_REVIEWED). The executed query never re-opens; the state machine adds the PENDING_AI → APPROVED → EXECUTED break-glass path and the illegal-transition guard still rejects everything off-path (AF-385)
  • Multi-tenant organization management & per-org quotas — organizations are first-class, manageable tenants: a super-admin (platform_admin) manages them across the cluster from a dedicated UI (/admin/organizations) — create, edit, disable / enable — with per-org quotas (max_datasources, max_users, max_queries_per_day, enforced as 409 QUOTA_EXCEEDED) and a disabled-org kill-switch that blocks login and requests immediately. Cross-org isolation is hardened defense-in-depth (org always derived from the JWT, never the request body) (AF-456)
  • Data classification tagging — tag datasource tables/columns with one or more data classifications (PII, PCI, PHI, GDPR, FINANCIAL, SENSITIVE) in the schema explorer; tagging a column auto-applies a masking policy, the AI analyzer raises a query's risk score when it references a tagged object, and a derivation preview suggests a stricter review posture. Tags are audited and queryable org-wide for compliance reporting (AF-447)
  • Expanded MCP agent toolset — three more safe, read-mostly MCP tools for external AI agents: validate_sql (parse-only validation against the engine dialect — no AI cost, no execution — reporting parse errors and tables absent from the visible schema), get_column_samples (bounded sample rows through the AF-443 governed path, with row-level security + column masking applied), and get_audit_log (the caller's own audit entries, scoped to the caller + organisation) (AF-455)
  • CI/CD & IaC tooling — an official Terraform / OpenTofu provider (bablsoft/accessflow) and reusable GitHub Actions + GitLab CI template that manage datasources, review plans, routing / row-security / masking policies, AI configs, and notification channels declaratively over the REST API, with the same authoritative-upsert semantics as the bootstrap reconciler. Pipelines authenticate with a bootstrap-provisioned service-account API key (since v2.7 also issued from the Service accounts admin page — 22-service-accounts.md). The provider's source lives in terraform-provider/ and is released to a dedicated terraform-provider-accessflow repo that the OpenTofu / HashiCorp registries ingest (AF-452) — see docs/16-iac.md
  • Usage analytics & compliance reporting — pre-built compliance reports over a period, computed from the immutable query_snapshots forensic record: a classified-data-access report (executed queries joined to data-classification tags — which queries touched PII/PCI/PHI/GDPR/FINANCIAL/SENSITIVE objects) and a regulatory audit trail of DDL/DELETE operations with approver names. Reports export as digitally signed PDF/CSV (detached RSA signature over the bytes, verifiable offline via a published public key) whose SHA-256 is chained into the tamper-evident audit log (COMPLIANCE_REPORT_EXPORTED). A dedicated read-only AUDITOR role exposes the auditor dashboard (/admin/auditor) (AF-459)
  • OpenTelemetry tracing export & pre-built Grafana dashboards — first-class OTLP trace export of the proxy pipeline (parse → AI analyze → pool acquire → execute) to Tempo / Jaeger / Honeycomb via OTEL_EXPORTER_OTLP_ENDPOINT, with configurable sampling and MDC trace ids preserved. The backend exposes Prometheus metrics (proxy timers plus workflow business meters for query volume, approval SLAs, AI usage/cost, rejection rates, and pool stats), and the Helm chart ships two toggleable, Grafana-sidecar-discovered dashboards (AF-454)

v2.1 ✅ released

Theme: API access governance, data lifecycle, and reviewer productivity.

  • AI query-optimization & index recommendations — the analyzer returns concrete, dialect-aware optimization suggestions (index DDL + query rewrites) alongside the risk verdict; each has a one-click "Apply as draft" that pre-fills the editor and routes the suggested statement through the normal review pipeline, audited as submission_reason=AI_SUGGESTION (AF-451)
  • Query playground / dry-run sandbox — preview a query's impact before formal submission: a non-committing dry run returns the engine's execution plan (node type, rows, cost, filters) and a best-effort estimated row impact without mutating data, surfaced in the editor's right rail next to the AI panel. Runs through the same governance as a real execution (datasource access, allow-list, row-security) but creates no review. Dialect-aware across every engine that has a plan concept — relational EXPLAIN (PostgreSQL / MySQL / MariaDB / Oracle / SQL Server), MongoDB explain, Couchbase / Neo4j EXPLAIN, Elasticsearch/OpenSearch _validate/query?explain; engines without one degrade gracefully. Cloud warehouses additionally surface their native cost signal — the estimated bytes the query would scan (BigQuery's dry-run job totalBytesProcessed) — as an extra line in the dry-run panel (AF-634). SELECT dry-runs prefer the read replica when configured (AF-445)
  • Saved-query version history & diffing — every template save records an immutable version; view a side-by-side Git-style diff between any two revisions and restore a prior one, with history preserved (AF-442)
  • Real-time collaborative query editing & co-authoring — multiple authorized engineers co-edit a query that is in review (VS Code Live Share / Google Docs style): live presence, remote cursors, conflict-free CRDT merge (Yjs relayed over the existing /ws), and inline comment threads anchored to the SQL, persisted and audited. Edits re-enter the workflow through the normal submit path; the self-approval guard is unaffected (AF-441)
  • Immutable query snapshots & replay — every executed query writes a sanitized, exact-replay-capable snapshot (SQL + schema fingerprint + AI verdict + approval decisions). "Replay in test environment" re-runs that exact SQL against a chosen test datasource through the full review workflow — validating engine + referenced-table compatibility, never bypassing approval, and distinctly audited (trigger=replay) (AF-449)
  • Mobile-friendly approvals (PWA) with one-tap push — an installable PWA with an offline-capable review-queue shell, plus Web Push (VAPID) delivery as a per-user notification path: a reviewer away from their desk gets a push when a query needs approval and approves/rejects in one tap. The decision only commits after a step-up auth re-verification (password, or TOTP when 2FA is enrolled), and the self-approval guard is enforced server-side regardless of channel (AF-444)
  • Multi-model AI orchestration, voting & guardrails — an ai_config can run several models in parallel (the primary plus weighted {provider, model, weight} members, e.g. a fast local Ollama pre-score alongside a deep Claude analysis) and combine their verdicts by a configurable voting strategy (weighted average, highest risk, or majority); issues are merged and risk aggregated. Pre-call guardrails block configured prompt-injection / pattern regexes before any model is called, and the existing strict output-schema validation rejects malformed responses without failing the query. Per-model cost (tokens) and latency are persisted per analysis and charted on the admin dashboard (AF-450)
  • API Access Governance — extend query governance to outbound REST / SOAP / GraphQL / gRPC calls: a connector catalog with schema ingestion, a schema-guided request builder, the submit → AI risk → review → execute pipeline, response masking, break-glass, and text-to-API drafting (AF-500) — see docs/17-api-governance.md
  • Request chaining & grouping — bundle ordered query + API-call members into one grouped request with aggregated AI analysis and a unified approval (union of approvers, every member's plan satisfied); an ordered executor runs the members in sequence with no distributed rollback (AF-501)
  • Data Lifecycle Manager — retention policies, a right-to-erasure request workflow, and proxy-enforced pseudonymization, driven by clustered-safe scan / execution jobs, with compliance reporting over the forensic record (AF-499)
  • Configurable & request-based data erasure — erasure conditions, cron-scheduled and request-based execution, and peer review of erasure requests with review-timeout auto-reject (AF-519)
  • Personalized dashboard & query insights — a self-scoped home surfacing pending approvals, my-query trends, the AI-suggestion backlog, and anomaly alerts, plus an opt-in weekly digest / export (AF-498)
  • Group-based access grants — grant read / write / DDL / break-glass on datasources and API connectors to a whole user group (inherited by every member), with parity-matched per-member authoring in request groups (AF-530, AF-531)
  • Access recertification campaigns — recurring attestation campaigns over the existing access model: an admin schedules an org- or datasource-scoped campaign that snapshots current standing datasource_user_permissions grants into per-grant items at open time and notifies the eligible reviewers (multi-channel + a attestation.campaign_opened WebSocket event). Reviewers certify or revoke each item from a worklist that reuses the review-queue patterns (eligibility derived from datasource reviewers, self-review blocked, bulk-certify); a REVOKE routes through the existing permission-revoke service. Two clustered-safe scheduled jobs open campaigns on their cadence and close them at the due date, applying a configurable default (KEEP/REVOKE) to anything still pending. A completed campaign exports as CSV evidence (who reviewed what, decisions, timestamps) for SOC2 / ISO 27001 audits, and every transition lands in the tamper-evident audit log (ATTESTATION_CAMPAIGN_OPENED/CLOSED, ATTESTATION_ITEM_CERTIFIED/REVOKED) (AF-384)
  • Dynamic variables for API connectors — named per-connector expressions evaluated at execution time and substituted into headers, path, query and body via {{name}} placeholders, making vendor contracts that require a value computed per call governable: HMAC request signing, nonces, timestamps, correlation ids, idempotency keys, digests. Variables resolve after connector auth (including a freshly minted OAuth2 token), so an expression can sign the finished Authorization header — and {{request.*}} describes the request before substitution, so a body being signed still carries its own placeholder, exactly as the motivating vendor spec requires. Eight kinds (CONSTANT/UUID/TIMESTAMP/EPOCH_MILLIS/RANDOM_HEX/HASH/HMAC/ENCODE) over hex / base64 / base64url, resolved in topological order with cycles and dangling references rejected at save time. Evaluation is template substitution plus a fixed function set only — no scripting engine — and resolved values are never persisted, snapshotted or logged. Submitters holding a per-connector can_override_variables grant may supply per-request overrides for variables explicitly marked overridable (never a secret-bearing one), persisted and shown to reviewers so an approval covers exactly what will execute (AF-613)
  • Masking & data classification for API connectors — brings the datasource governance model (dynamic masking AF-381 + classification tagging AF-447) to the API Access Governance module (AF-500), adapted to non-tabular API responses. Connector-level masking policies target a response field four ways — a schema-bound field, a JSON path, an XML/XPath, or a regex — each with a masking strategy and role/group/user reveal scoping; ApiResponseMasker applies them (reusing ColumnMasker) before the response snapshot is stored, so the raw value never persists, and applied policy ids land in the execution audit. Data-classification tags (PII/PCI/PHI/GDPR/FINANCIAL/SENSITIVE) auto-derive a masking policy and raise the apigov AI analyzer's risk for calls to a classified operation, with a derivation preview (AF-518)

v2.2 ✅ released

Theme: Enterprise operations — secrets, throughput, resilience, and ticketing.

  • External secrets managers — datasource credentials resolved from HashiCorp Vault, AWS Secrets Manager, or Azure Key Vault at connection time via a secret reference (vault:… / aws:… / azure:…) stored in place of the encrypted password, with local AES-256-GCM as the fallback, per-resolve audit, and SDK-managed token refresh (AF-448)
  • High-volume proxy performance — opt-in Redis-backed SELECT result caching keyed over the security-rewritten query (invalidated on any proxied write to a referenced table), JDBC executeBatch() for homogeneous INSERT runs inside BEGIN…COMMIT envelopes, and multi-endpoint read-replica load balancing with per-node circuit-breaker health checks and primary failover (AF-457)
  • Backup / restore / DR tooling & offline AI fallback — Helm backup CronJob (pg_dump to a kept PVC with retention pruning + optional rclone upload to S3/GCS/…) and a one-shot restore Job that preserves the audit-role ownership split; post-restore audit HMAC chain verification via ACCESSFLOW_AUDIT_VERIFY_CHAIN_ON_STARTUP; an AI provider fallback pool (ai_config.fallback_priority) so analysis degrades to a local Ollama model when the primary provider is unreachable; documented DR runbook (AF-458)
  • ServiceNow & Jira ticketing integration — two new ticketing notification-channel types that auto-create a ServiceNow incident / Jira issue when a query is rejected, escalated by a routing policy (new QUERY_ESCALATED event), or times out awaiting review; tickets link back onto the query detail page, and signed inbound status webhooks sync the ticket state — optionally mapping a ticket resolution onto an approve/reject of the still-pending query (bi-directional sync) (AF-453)

v2.3 ✅ released

Theme: Data-aware governance, automation, and reviewer intelligence.

  • Automated sensitive-data discovery — an opt-in per-datasource scanner samples column data through the governed sampling path, detects sensitive values with local regex + checksum detectors (email, credit-card PAN with Luhn, US SSN, IBAN, phone) and optionally the org's bound AI analyzer (column names, types, and redacted samples only), and proposes classification tags in an admin worklist — confirming applies the AF-447 tag (deriving masking), dismissing suppresses permanently; scheduled cadence + on-demand "Scan now", fully audited (AF-623)
  • Approval-outcome prediction — a per-organization logistic regression, trained on the org's own historical human review decisions, predicts the probability a reviewer approves a query entering PENDING_REVIEW and shows it as a triage signal on the review queue and query detail. Hand-rolled and deterministic (no ML library, no LLM call, no RNG — the train/holdout split hashes the query id), featurized from data already collected: AI risk verdict, the AF-624 cost estimate, query type, transactional flag, time of day, and Laplace-smoothed per-submitter / per-datasource approval rates. Every auto path is excluded from training — routing AUTO_APPROVE/AUTO_REJECT (AF-379), grant-covered auto-approval, break-glass (AF-385), external-ticket decisions (AF-453) — since none carries reviewer judgment. A clustered-safe daily job retrains per org behind a quality gate (≥ 50 decided samples, ≥ 10 per class, holdout AUC ≥ 0.65); below it, predictions persist as skipped rows so the UI can say "not enough history yet". Advisory only and structurally so: only the read side consumes it, never routing, grant coverage, or any decision path (AF-645)
  • Recurring approved queries with result subscriptions — "run this query every Monday 08:00 until <date>" as a single reviewable request: the query, cadence (6-field cron in UTC or a fixed ISO-8601 interval), and mandatory expiry are approved once, then a clustered-safe RecurringQueryRunJob executes each occurrence through the full proxy pipeline with current policy state, recording every run as its own immutable child request (results, snapshot, audit). Kill-switches everywhere: the submitter or any reviewer cancels the series, and permission revocation / expiry, a deactivated datasource, or unparseable SQL halts it fail-closed with an audited reason. Occurrence results are delivered through the existing notification channels — email carries the rows as a capped results.csv attachment, chat channels a summary with a link (#627)
  • Least-privilege intelligence — unused-grant analytics & revocation recommendations — a clustered-safe GrantUsageAggregationJob folds executed-query and executed-API-call audit metadata into a per-standing-grant usage summary (last used, usage count, granted vs actually-exercised tables / operations) for both grant kinds — datasource_user_permissions (scope = allowed tables) and api_connector_user_permissions (scope = allowed operations) — and derives a revocation recommendation from it: NEVER_USED / STALE / OVER_SCOPED / ACTIVE / INSUFFICIENT_DATA, conservative by construction (recent use beats youth, and an unrestricted grant with no allow-list can never be reported over-scoped). Surfaced in three places: attestation campaign items and the reviewer worklist, now sorted staleness-first so never-used grants land on page one (AF-384); a standing over-provisioned access report at /admin/over-provisioned-access for admins and auditors (new ACCESS_USAGE_REPORT_VIEW permission, filterable by resource kind / recommendation / resource / user, with a CSV export audited as OVER_PROVISIONED_ACCESS_EXPORTED); and an opt-out GRANT_STALE notification that nudges org admins once per cooldown when a grant crosses the staleness threshold. The summaries are a derived cache refreshed each tick rather than a record — a grant created since the last run is backfilled over the observation window, and a revoked grant's row is deleted instead of lingering as permanently idle. Advisory only and structurally so: nothing is ever revoked automatically, and no decision path — routing, grant coverage, break-glass — reads a recommendation. Group-inherited grants (AF-530/531) are out of scope (#625)
  • Reviewer delegation & escalation — a reviewer sets an out-of-office window naming a delegate, who becomes an eligible approver everywhere the delegator was (query review, governed API requests, and grouped requests) with every decision recording both identities and the delegation that authorised it. Structurally safe by construction: the permission check runs before delegation is resolved so it can never grant a permission, the self-approval ban covers both parties, resolution is one hop only (no A→B→C chains), both parties' active flags are re-checked on every read, and the existing per-stage unique index keeps one human to one vote. Ships with an out-of-office card on the profile page, a "Delegated" tag on the review queue, on-behalf-of attribution in the approval timeline, and an admin oversight listing. Alongside it, idle-reviewer escalation and nudge reminders: per-review-plan escalation_after and nudge_interval (both optional, both off by default) drive three clustered-safe jobs that escalate a stalled request to the reviewers at its current stage plus org admins before the hard approval timeout auto-rejects it, and re-nudge those same reviewers on a cadence. Notify-only by construction — idleness never widens who may approve — with the escalation flagged on the request detail page and delivered through the existing notification channels (#622)
  • Result-export governance & DLP — per-datasource export policies with role/group/user targeting govern how query results leave AccessFlow (allow / watermark / row-cap / deny-when-classified, most-restrictive-wins, no admin bypass): a new signed CSV/PDF result-export endpoint and the recurring-email results attachment both enforce them. Classified detection reuses the AF-447 classification tags at export time; watermarked exports carry the exporter, UTC timestamp, and query-request id baked into the signed bytes (stripping the stamp invalidates the signature); every export writes a RESULT_EXPORTED audit row, and exports containing classified columns raise a SENSITIVE_RESULT_EXPORTED notification to org admins. The export button on the query detail page is decision-driven — disabled with the denial reason when policy forbids (#626)
  • SIEM & WORM audit streaming — admin-configurable external audit sinks stream audit_log rows to four destination types: SPLUNK_HEC (newline-stacked HEC envelopes), SYSLOG_CEF (RFC 5424 frames carrying CEF:0 over TCP or TLS with RFC 6587 octet-counting; TLS validates against the system truststore, no skip-verify), HTTPS_BATCH (JSON event arrays HMAC-signed with the existing webhook signature contract) and S3_OBJECT_LOCK (periodic signed JSONL segments under a WORM retention lock). Managed on /admin/audit-sinks behind a new AUDIT_SINK_MANAGE permission, with a clustered-safe drain job, per-sink delivery state and AUDIT_SINK_* audit actions. Streaming is strictly additive — the tamper-evident audit_log chain remains the record, and a failing sink can never block or alter a write (#628)
  • SCIM 2.0 provisioning — IdP-driven user & group lifecycle over /scim/v2 with per-org bearer tokens (Okta / Entra ID / Keycloak); deactivation in the IdP revokes sessions and JIT grants immediately (#621)

v2.4 ✅ released

Theme: Deployment approval governance.

  • Deployment approval governance — a third governed request surface alongside queries and API calls: gate CI/CD deployments behind the same submit → AI → review → approve machinery. Admins define deployment pipelines (GitHub Actions, GitLab CI, Azure Pipelines, Jenkins, CircleCI, Bitbucket Pipelines, or generic) with ordered per-environment policies — required approvals, review-plan overrides, break-glass opt-in — and per-user/per-group can_trigger / can_break_glass grants resolved as a most-permissive union. A CI job triggers a request over the REST API with an existing AccessFlow API key, idempotent on the CI run id (202 on create, 200 on replay); the AI analyzer scores the release metadata; routing policies (environment / provider / version glob / risk / time window) may auto-approve, auto-reject or change the approval count; and reviewers decide in the UI, where the submitter — the API key's owning user — can never approve their own deployment. The pipeline then blocks on a fail-closed gate, GET /api/v1/deployment-gate, whose one pure releasable function defaults to not releasable: it opens only for an APPROVED request with no active freeze window and a scheduled_for that has passed, and any error, unknown tuple or under-permissioned poll answers releasable: false (404, never 403). Freeze windows are one-off or weekly recurring, scoped org-wide / per-pipeline / per-environment, HOLD (withhold releasability) or REJECT (auto-reject at submission), and fail closed to HOLD when a definition is unevaluable. Audited break-glass deploys need both a can_break_glass grant and an environment opt-in — no admin bypass — and carry a mandatory retro-review. After the deploy the pipeline confirms execution and reports the outcome (SUCCEEDED / FAILED / ROLLED_BACK); a rollback on a review-required environment opens a follow-up review the submitter can never acknowledge. Eight new tables, native GitHub Action / GitLab CI / Azure Pipelines wrappers plus a generic-curl path, five notification event types, an audit row for every transition, and a full web UI — deployment list and detail, reviewer queue with a rollback worklist, and pipeline administration with a copy-paste CI setup panel (AF-682) — see docs/18-deployment-governance.md

v2.5 ✅ released

Theme: Deployment version tracking and drift.

  • Multi-environment version tracking & drift — carry the AF-682 epic past the approval gate into what is actually running where: per-environment version tags and current-version state so an environment knows the release it last executed, rollback semantics that restore the prior recorded version rather than merely reopening a review, and a version-inventory / drift API that answers "which environments are behind, and by how much" across every pipeline in the org. Surfaced in the web UI as a version matrix across environments, a per-environment history timeline, and drift badges on the deployment list and detail pages (#741, #742, #743 — epic AF-682)
  • Release update notifications & public changelog — a self-hosted install learns that a newer stable release exists: the backend polls the static https://accessflow.io/version.json manifest at most once a day (lazy, off the request path, fail-soft, no telemetry — one unauthenticated GET), compares it to its own build version with a real semver comparison (pre-release and -SNAPSHOT builds never nag), and GET /api/v1/system/update-status surfaces the result to every signed-in user as a warn-toned version chip under the sidebar logo that deep-links to the release's entry on the new public changelog (https://accessflow.io/changelog/#vX-Y-Z). The prep-gh-release skill writes the changelog entry and the manifest when a milestone ships, so the page cannot go stale; ACCESSFLOW_UPDATES_ENABLED=false switches the check off for air-gapped installs (#836)

v2.6 ✅ released

Theme: Answering from what AccessFlow already knows.

  • Automatic query suggestions — an analyst opening the editor no longer starts from a blank buffer: a third right-rail tab offers whole draft queries mined from the organisation's own approved history on that datasource, each shown with the evidence behind it (how often it was approved, by how many people, how recently). Ranking is a deterministic local heuristic — frequency × recency × overlap with the analyst's own recent tables — so there is no provider call, no cost, and the rail keeps working with AI switched off. A scheduled per-datasource pass groups approved queries by canonical form, parses each shape once for its referenced tables, and sweeps whatever it no longer computes; break-glass, recurring occurrences and recurring-series parents never enter the corpus, because none of them is a query an analyst authored and put through the approval path. Every suggestion is filtered at read time against the viewer's own effective permission — capability plus table allow-list — so it can never hint at data they cannot reach, and a shape whose tables could not be resolved is dropped rather than stored. Strictly advisory: applying one only fills the editor, and the draft is then analysed, routed and reviewed like any other query, carrying submission_reason=HISTORY_SUGGESTION in the audit trail, kept distinct from the AI optimization suggestions of AF-451 so the two provenances stay tellable apart (#776)

  • Effective-access explainer — an admin can finally ask why without submitting a probe query in production. A read-only trace endpoint replays one hypothetical request through the very evaluators that decide real ones — the datasource and quota gates, the merged effective permission with each contributing grant named, every routing policy in priority order with the ones that nearly matched, the grant-covered fast path, the review plan, who could review it, and the row-security and masking policies that would apply — and returns an ordered, per-stage answer with the terminal status. Nothing is written, executed, published or sent to an AI provider, and the approximations a hypothetical request cannot avoid are named in the response rather than smoothed over. Alongside it, the reverse index the product never had: who can write to this table, one row per user, naming every source that grants it — the direct row, each inherited group permission, an active JIT grant, break-glass, and the QUERY_ADMIN holders who bypass the per-datasource gate entirely and therefore appear on no permission screen at all. Table matching delegates to the enforcement gate's own rules, so the report cannot drift from what actually happens (#859)

  • Decision traces for API calls and deployments — the explainer, carried onto the other two governed request kinds. Each of apigov and deploygov gets the AF-859 treatment: the review state machine is split into a pure evaluator that decides and records why, and an applier that persists and publishes, so a read-only trace endpoint can replay the very code production runs instead of a second copy of the rules. POST /admin/api-call-simulations walks the connector gate, the read/write classification and where it came from, the schema catalog, the effective connector permission, every routing policy in priority order, the review requirement, who could review it, and the masking rules that would rewrite the response — never contacting the governed third-party API. POST /admin/deployment-simulations is the sharper of the two, because a release is blocked by the gate rather than by the decision: it reports the trigger grant, the freeze window, routing, the environment's own policy, the approvers, the deferred-release moment and the gate's own verdict, computed by the very function the CI job blocks on — and it accepts an optional at, so "would a release this Friday evening be held?" is answerable without waiting for Friday evening. Stages are per-kind rather than one widened enum; only the trace vocabulary is shared, moved into core.api for the purpose. Both are read-only by construction, gated by the permission that already governs their kind, and audited as ACCESS_SIMULATION_RUN (#967)

  • Deterministic SQL review rules — the third judgement on a query, next to the AI verdict and routing policies: a named, deterministic rule catalog whose violations are reported one by one, carry a stable rule id, and appear in the editor as the author types rather than after a submit round-trip. Fourteen statement-safety rules derived from the parsed statement alone — missing WHERE on UPDATE / DELETE, an always-true WHERE that would defeat those two, SELECT *, unbounded or full-sort reads, cross joins, leading-wildcard LIKE, DROP / TRUNCATE / any DDL, banned functions, protected tables by glob, and DML outside a transaction — each run at an admin-set severity of OFF / WARN / BLOCK, configured per environment: every datasource can be tagged DEVELOPMENT / TEST / STAGING / PRODUCTION, one ruleset binds per environment, and an organisation-wide default catches the rest. BLOCK escalates, it never rejects — a blocking finding suppresses every path that would have approved the query without a person (routing auto-approve, a pre-approving grant, the review plan's own fast path) and sends it to a reviewer, while a routing auto-reject still rejects. Evaluation happens synchronously at submission, before the AI is asked, so the findings bind even when AI analysis is off or fails; they are stored as rule id plus arguments and rendered in each reader's language on the query detail, the reviewer queue, the break-glass retro-review and the request-group detail. Break-glass runs record findings but are never gated by them; engine-plugin datasources report not applicable rather than failing closed. Managed at /admin/sql-review, audited on every ruleset change and on every suppression (#860) — see docs/19-sql-review.md

  • Privileged-access report — the standing, org-wide answer to the question an auditor opens the product to ask: who can reach data without appearing in any permission table? Two paths qualify and neither showed on any admin screen: holders of QUERY_ADMIN — the system ADMIN role or a custom role carrying it — skip the per-datasource gate outright and so hold zero permission rows, and can_break_glass grants, direct or inherited through a group, were visible only as a column on one grant. The effective-access explainer answers this per table; the over-provisioned report cannot see either path because a bypass has no grant to report on. This report lists every such identity once, naming the role that carries the bypass, each break-glass datasource with its expiry, and the evidence — how many queries the user submitted and when last, and how many of those were break-glass runs — read straight from the request history rather than the per-grant usage fold, which by construction drops what a bypass produces. Readable by the same admins and auditors as the explainer, audited on every read, and advisory only: nothing revokes on its strength (#968)

  • In-app documentation help assistant — a user who does not know AccessFlow can finally ask it. A launcher in the app shell opens a chat panel that answers "how do I submit a query for review?", "what does break-glass actually do?", "how do I connect Snowflake?" from AccessFlow's own documentation, bundled with the running build so an install never describes a version it is not running, and cites the sections it used as server-resolved links. An admin enables it and binds it to an ai_config exactly as every other AI-backed feature; the corpus is embedded per organization with that configuration's embedding model into the same vector store as the RAG knowledge base, kept invisible to it by a metadata discriminator, and re-indexed on upgrade by content hash. Where retrieval cannot be configured — no embedding provider, an Anthropic-only install, pgvector absent — a bundled quick-reference orientation block answers instead, uncited. The guarantee that keeps a free-text LLM endpoint out of the governance path: it is a documentation reader, not a data surface — no tools, no actions, no query, result, audit row, schema or datasource ever in the prompt; only, optionally, the current screen's label and the caller's permission names. Conversations are private, retention-pruned, rate-limited per user and charged against the organization's monthly AI token budget; remote corpus refresh from gh-pages is opt-in and off by default, and ACCESSFLOW_HELP_CORPUS_OFFLINE pins an air-gapped install to what it shipped with (#899 — sub-issues #900–#908)

  • Policy simulator — dry-run a draft routing policy, row-security directive or masking rule against the organisation's own historical query traffic before saving it. One read-only endpoint per policy kind, each gated by the permission that already governs that policy; the window is replayed twice — once under the current policies, once with the draft applied — and the report is the difference: how many past queries would change outcome, which users would be affected, and which query shapes a row predicate would newly filter, deny or reject. Row-security shapes are classified offline by every engine's own classifyRowSecurity, so nothing is ever executed against a customer database, and an engine that cannot classify a shape offline reports it as unclassifiable, never as safe. Surfaced as a Simulate button on every routing, masking and row-security form (#630)

  • Onboarding by governance domain — the first-run wizard asks which of the three governed request kinds the organisation intends to use: database access is always on, outbound API calls and CI/CD deployments are opt-in. The answer is persisted on the organisation and drives the dashboard checklist ("create your first API connector", "create your first deployment pipeline") (#898), and from there the whole app: the sidebar hides a domain's navigation group until it is switched on, the dashboard gains deployment-governance widgets — pending deployment reviews, my recent deployments, an environment version / drift tile — and only offers each domain's widgets when that domain is governed; visibility only, never entitlement — no route or permission changes and every deep link keeps working — and an org admin can switch a domain on later (#926)

  • Discovery hardening — the sensitive-data scanner now walks nested document and map values on the NoSQL engines (MongoDB documents, Elasticsearch _source, DynamoDB items, Neo4j property maps), reporting findings under a dot-path pseudo-column so a confirmed tag lands on exactly the nested field the masker already understands (#658); a PENDING finding that a scan of its table stops re-detecting is retired as STALE and audited, never a finding on a table the run did not sample (#659); and the "a scan is already running" guard is a cluster-wide distributed lock rather than a per-node set, so an on-demand scan on one replica and the scheduled scan on another can no longer sample the same datasource twice (#660)

  • Warehouse engine follow-ups — the Snowflake connector accepts a passphrase-protected PKCS#8 private key for key-pair authentication, which Snowflake's own tooling generates by default, with the passphrase stored as a second secret alongside the PEM (#632); the Databricks connector falls back to the Statement Execution API's EXTERNAL_LINKS disposition when an inline result exceeds the ~25 MiB cap, streaming the presigned chunks up to the proxy's own row and byte caps and never persisting the presigned URLs (#633)

  • Voyage AI embeddings — a first-class embedding-only provider in the RAG and help-assistant configuration: prefilled endpoint and model, Voyage's input_type sent on every call, and an impossible vector dimension refused at configuration time rather than at ingest. Rejected as a chat or orchestration provider by construction (#918)

  • JIT grant provenance — a permission row materialised from an approved just-in-time access request now records the originating request as a foreign key, so the effective-access explainer can say with certainty whether queries under that grant also skip review, instead of correlating on requester, datasource and time (#969)

v2.7 ✅ released

Theme: Identities for machines, finer data-access controls, and governed schema changes.

  • Schema change governance — the missing unit between a single DDL query and a deployment: a schema change set, an ordered list of schema statements authored once under a deployment pipeline and promoted along its environment ladder (dev → staging → prod), with the guarantee that an environment only ever receives a change that already succeeded in every lower environment bound to a database. Deployment environments gain an optional bound datasource and a real, unique sort_order, which is what makes the ladder meaningful. Every statement is parsed with the target engine's own parser before it can be saved: SELECT / INSERT / UPDATE / DELETE are refused, a BEGIN … COMMIT envelope or multi-statement text is refused, and a deterministic SQL review BLOCK refuses the save outright — the one place a blocking rule rejects, since a set being authored has no reviewer to escalate to. The statement list freezes on first promotion and carries a SHA-256 checksum, so "applied to staging" still means the same text when prod is asked. Promotion is fail-closed: it needs can_ddl on the target datasource for the promoting user, with no admin exemption; it is refused in a frozen environment, HOLD included; and an environment that requires review is refused outright when the target datasource's review plan could not enforce one. A promotion runs as an ordered request group — aggregated AI analysis, union-of-approvers review, the scheduled run job as the durable execution trigger — and projects the group's status back onto itself. There is no rollback at all: each statement runs on its own, so a failure part-way leaves the environment PARTIALLY_APPLIED and a new change set is the remedy. An opt-in, per-pipeline schema drift job introspects each environment on a schedule and diffs it against the previous environment, a designated reference environment, or the snapshot taken right after the last successful promotion, opening findings an admin acknowledges; it compares schemas, tables, columns and foreign keys only — never indexes, constraints, defaults, views or type precision — and never writes. Notifications on review, apply, failure and newly opened drift; audited end to end; managed at /schema-change-sets and /schema-drift behind SCHEMA_CHANGE_MANAGE (#870 — sub-issues #877–#884) — see docs/20-schema-change-governance.md

  • Service accounts — CI pipelines, Terraform runs and AI agents get identities of their own instead of borrowing a person's API key. A service account is a user with principal_type=SERVICE_ACCOUNT that can never sign in interactively (password, SAML, OAuth and password reset all refuse it); its only credential is an API key issued from the admin Service accounts page — shown once, rotated with a grace window so running jobs are not cut off, revoked instantly — or declared in bootstrap config, where the declared key can be neither revoked nor rotated. An account created from the UI or API defaults to the Read-only role (a bootstrap-declared one defaults to ADMIN, so its spec should always name a role), carries an optional MCP tool allow-list enforced on every tool call, and per-minute / per-day request limits; every API-key request, a person's included, is rate-limited per identity with 429 + Retry-After, which every CI wrapper and the Terraform provider retry. An agent can name the human it acts for (X-AccessFlow-On-Behalf-Of, only after that human consents): its permissions stay its own, the human is stamped on the submission and every audit row, and that human becomes a second submitter identity for the self-approval ban — closing the approval-laundering path. Service accounts are badged wherever people are listed, and the review-plan editor warns when a role-based approver rule would include one (#867 — sub-issues #868–#876, #1129–#1131) — see docs/22-service-accounts.md

  • Finer data-access controls — the access grant learns to say no as precisely as it says yes. Table and schema deny-lists sit alongside the allow-lists and always beat them, tables created later included (#939); a grant's denied_columns refuses any query that references a column — through SELECT * too — before it runs, instead of masking it (#935); denied_shapes refuses joins, set operations, subqueries, CTEs, grouping, aggregates or window functions anywhere in a statement, and a matching query_shape routing condition can escalate them instead (#940). Per-table row limits cap what one table may return, for everyone or for chosen roles, groups or users, only ever lowering the datasource and grant caps (#934), and the per-user / per-group row_limit_override is now enforced everywhere, the smallest value winning (#933). Masking gains configurable strategies — keep-first-N, constant, nullify, regex replace, numeric bucketing and date generalization join partial, hash, email- and format-preserving — parameterised, validated on save, and failing closed to a full mask (#944). Schema introspection is scoped to the caller's allow-list, so a grant on two tables no longer discloses every other table's name and columns (#936)

  • Cost and volume budgets — on BigQuery, Snowflake and Databricks a query whose pre-flight bytes-scanned estimate exceeds the datasource or grant cap is refused before it runs and re-checked right before execution, with an explicit choice for queries that have no estimate (#941). Per-user data-volume budgets bound how many rows or bytes each person may read from a datasource over a rolling window: a result is capped to the remaining allowance, and once it is used up new reads are refused or sent to review; every delivered read counts — scheduled, recurring, grouped, previews and break-glass (counted, never blocked) — users see their allowance in the editor, and estimated_bytes_scanned / data_budget_used_percent are routing conditions (#942)

  • External policy decision hook — for rules that do not decompose into routing conditions, an organisation's own policy service (an OPA sidecar, an in-house API) is consulted over HMAC-signed HTTPS for a query no routing policy matched and no earlier cap, budget or AI-failure path already decided. It may allow, escalate, require approvals (never below the plan minimum) or reject — never approve — and any failure (timeout, error status, redirect, bad signature, unknown answer, open circuit) sends the query to human review. The SQL text is sent only on opt-in and the endpoint is SSRF-guarded; managed at /admin/decision-hooks or with the Terraform accessflow_decision_hook resource (#945)

  • Effective-permission explorer and decision-trace viewers — the forward question, what can this user actually do on this datasource right now?, answered in one read-only, audited view: merged capabilities, allowed and denied objects, masked columns, row-security predicates and the effective row cap, each naming the grant, group or policy it came from (#946). The query, API-call and deployment decision traces get their UI, alongside the reverse index that answers who can reach a table (#1066), and recertification evidence records each grant's effective row limit (#1084)

  • Execution provenance — an executed query's snapshot keeps the statement as it actually ran, row-security and soft-delete rewrites spliced in and bound values left as ?, shown on the query page as a Submitted / Effective / Diff toggle (#937); every request and audit row can name the calling application — trusted from the API key, marked untrusted from the X-AccessFlow-Application header — and the audit log and query list filter on it (#938)

  • Scheduled job monitor — a platform-admin /admin/jobs page lists every registered scheduled job with its cadence, lock, last status, duration and consecutive failures, read from a registry derived from the running application rather than a curated list, with a per-job execution history kept for a configurable retention (#923)

  • Structured-DDL SQL review rules — four more built-in rules on JSqlParser 5.4 for the schema-change shapes that hurt in production, such as adding a NOT NULL column without a default or creating an index without CONCURRENTLY, each applying only to the engines where it means something and defaulting to WARN (#1079, #1080)

Backlog / Unscheduled

Theme: Candidate features not yet scheduled into a milestone.

  • Native database wire-protocol gateway — connect existing SQL clients (psql, DBeaver, DataGrip) and BI tools (Metabase, Tableau, Superset) through AccessFlow over the native PostgreSQL wire protocol, with every statement still flowing through the proxy's validation, masking, row-security, and audit path (AF-382)

  • Aggregate disclosure guard — constrain queries that return few rows but reveal a great deal about individuals (COUNT(*) … WHERE national_id = …, GROUP BY over a one-person group). Detection first: a classification_referenced routing condition that sends aggregates over classified data to a reviewer; minimum-group-size suppression deferred. Design, non-goals and open questions in docs/21-aggregate-disclosure.md (#943)

  • Plugin API for custom AI analyzers — allow teams to plug in their own analysis logic via a defined Java SPI or HTTP callback

Contribution Path

PostgreSQL, MySQL, MariaDB, Oracle, and Microsoft SQL Server ship as bundled adapters; any other JDBC-compatible engine (SQLite, CockroachDB, Snowflake-via-JDBC, vendor forks, etc.) can already be added today by uploading a custom driver JAR through the admin UI (db_type=CUSTOM + custom_jdbc_driver — see docs/03-data-model.md). Community contributions promoting an engine to a first-class bundled adapter are welcome after v2.0; the accessflow-proxy module defines a DatabaseAdapter SPI that third-party adapters can implement. See CONTRIBUTING.md for the adapter development guide.