feat(policy): enforce static view policy on Iceberg/SQL graph sources

[bplatz]

Summary

View policy never reached an Iceberg or SQL graph source. The R2RML scan emits string-IRI bindings with no index behind them, so the flake filter never ran and every policy shape fell open: f:onProperty / f:onClass / f:onSubject denies, default-allow: false, even an unknown identity returned every row, while the same policy on a native ledger returned none.

This PR enforces static targeting on virtual sources with native parity, gives a source somewhere to keep its policies and class hierarchy, and makes the one thing a virtual source cannot do (f:query) visible instead of silent.

Enforcement (fluree-db-query/src/r2rml/policy.rs)

The scan operator builds a lane-local policy gate when a non-root policy is active. It re-indexes the request's view restrictions against class→property stats derived from the mapping (a virtual source has no index stats) and also selects class policies by the row's classes, then evaluates each read triple with the engine's own evaluator.

• Subject classes come from rr:class and constant rdf:type maps, or are materialized per row when a map derives rdf:type from a column (the type columns are added to the projection).
• Decisions are memoized per (triples map, predicate) unless a policy targets subjects, and a map whose required predicates are all hidden is skipped before its table is read.
• f:query policies evaluate as "no rows" and deny their targets. There is no graph to run them against.
• The fused aggregate fold declines under a policy so counts go through the gate.

Covered shapes: fixed predicates, same-subject stars, constant objects, class scans, projected rdf:type, wildcards, aggregates, and GRAPH blocks in dataset mode.

Model ledger for virtual sources

A source can name a model ledger (--model, HTTP/config model) whose default graph becomes its f:policySource and f:schemaSource through the existing cross-ledger resolver. That supplies stored policies, rdfs:subClassOf / rdfs:subPropertyOf expansion of targets, and identity f:policyClass lookup (done in the model ledger for virtual sources, which have no ledger of their own). The schema-bundle translation no longer drops axioms whose namespaces the genesis data snapshot never registered; on a native ledger those axioms are inert.

The model is validated at registration: it must be an existing native ledger, and policies in it that use f:query are reported as model_warnings.

Ergonomics

• policy_enforcement.unevaluable_policies on the tracked response lists the f:query ids the source could not evaluate; the server logs a warning once per policy, and skipped triples maps log at debug.
• --default-allow / default_allow on a source keeps a model-less source readable under authentication, the counterpart of a native ledger's f:defaultAllow. Without it, an authenticated server now returns nothing for a source with no model and no matching policy, which is the correct fail-closed outcome but a visible change on upgrade.
• fluree model access enable --connected warns when a registered source is governed by the model.
• Ledger-info's source block reports model and default-allow.

Verification

it_iceberg_policy checks every static shape and pattern form against a native twin of the same fixture as the oracle, plus fail-closed f:query, column-derived classes, dataset mode, the model-ledger flow, registration validation, the tracked-response field, and the default-allow knob. Existing R2RML, Iceberg, SQL, policy, cross-ledger and reasoning suites pass.

Not in this PR

• Explain output does not yet mark a policy-gated scan.
• The impersonation gate cannot test policies against a virtual source.
• Hidden columns in a partially hidden star are still read; results are unaffected.
• The graph-scoped builder still ignores opts.policy for native ledgers, so a native ledger read through proxy mode is unfiltered. Pre-existing and out of scope here; this PR narrows it rather than widening it. Follow-up: graph-scoped query builder ignores opts.policy for native ledgers #1766

[aaj3f]

This is a genuinely important fix, @bplatz, and we talked about it a bit in-office. No disagreements with design or with decisions (i.e. how to narrow the scope of possibilities for what can be policy-enforced reasonably on remote Iceberg/SQL data). I'll provide Claude's review below:

---

Praise for the following: — the same allow_view_flake_async the native path uses, running as a lane-local gate inside the R2RML scan with the view restrictions re-indexed against mapping-derived stats, rather than a second policy implementation for virtual sources. I verified it rather than reading it: with --features iceberg,native all 13 it_iceberg_policy tests pass on this head, and forcing the gate to always-allow makes static_policies_match_native_twin fail at :197 with the native twin denying what the virtual source returned — so the parity oracle has teeth. f:query failing closed and landing on the tracked response as unevaluable_policies (plus the once-per-source warning) is the right way to make the one thing a virtual source can't do visible instead of silent, and catching the proxy-mode builder in commit 3 — where a governed source would have been read unfiltered — is the kind of entry-point audit that usually gets missed.

Two things I'd fold in now. First, a performance nit on the gate: allows allocates the (tm.iri, pred) memo key as two Strings per row per predicate before checking the cache, and on the static path the per-row loop is redundant anyway (same answer for every row in the window) — hoisting the static decision out of the row loop makes the common case allocation-free. Second, the pre-existing gap your commit body already names — the graph-scoped builder ignores opts.policy for native ledgers, which I confirmed — is the same class of bypass this PR closes for virtual sources, so it deserves a scoped fix soon; I'm not asking for it here because whether proxy mode should wrap policy for native ledgers is a scope call, not a review thread. Also worth a line in the title or first paragraph: the fail-closed behavior for model-less sources under authentication is a visible upgrade change and should reach the release notes.

Adherence to repo commitments:

• Patterns/abstractions: ✔ reuses PolicyContext / allow_view_flake_async and the cross-ledger f:policySource/f:schemaSource resolver; no parallel evaluator; SPARQL/JSON-LD parity untouched (scan-level).
• Performance (speed first, memory second): ✔ no-policy path byte-for-byte unchanged (policy_gate: None); under a policy: memoized per (map, predicate), fully hidden maps skipped before their table is read; ⚠️ two avoidable String allocations per row on the static path — see the inline note; fold and top-k decline under policy is correctness over speed and documented.
• Testing: ✔ 13 integration tests against a native-twin oracle in a declared [[test]] (required-features = ["iceberg","native"], so they run under CI's --all-features); mutation-verified; three regression tests in commit 3 the author verified red-then-green.
• Conventions: ✔ thorough multi-line commits; docs across six pages; wasm32 --no-default-features clippy fixed in-branch; conventional titles.

Verified locally at branch HEAD c221efdd0: cargo test -p fluree-db-api --features iceberg,native --test it_iceberg_policy → 13/13; cargo test -p fluree-db-core tracking → green; mutation (gate → always allow) → parity test FAILED as expected, restored clean; entry-point audit of server routes, SQL bridge, Cypher, and the two query builders; Cargo.toml [[test]] wiring confirmed.

Approving so you can merge when ready — maybe worth the allocation hoist first, and let's get the native-ledger builder gap on the board.

[aaj3f]

Should-fix (performance, fold in now). allows builds the memo key (tm.iri.clone(), pred.to_string()) before it checks static_cache, so on the static path every row pays two String allocations per required predicate even on a cache hit.

And on that path the answer is the same for every row in the window, so the per-row loop itself is redundant: filter_rows (:179-224) already knows required and whether the decision is static (!self.per_subject && row_classes.is_none()).

I'd decide each required predicate once per call before the row loop and only walk rows for the predicate_var and per-subject cases; the cache can then be keyed on borrowed (&str, &str) or on required's index.

It isn't a regression — nothing enforced anything here before — but it's a per-row allocation in the scan operator, which is the thing we grade hardest on, and the fix is local. If you agree it's right I'd rather see it here than in the backlog.

[bplatz]

Agreed, and folded in as 37a0e7a.

You were right that the per-row loop is the bigger half. On the static path — no subject-targeted restriction, no column-derived classes — every input to the decision is fixed for the whole window, so filter_rows now settles required and the projected rdf:type once before it walks anything. A denial returns the empty window; with no predicate variable in the pattern the rows pass straight through. Only a bound predicate variable, a subject-targeted policy, or column-derived classes put the gate back on the per-row path, and that path no longer copies the row's bound predicate into a String either.

static_cache is now nested (map IRI → predicate IRI → allowed) so a probe borrows both keys and only a miss allocates.

Behavior is unchanged by construction: passing no subject on the hoisted path is exactly what allows already substituted for a non-subject-targeted set, and row_classes.is_none() is precisely the condition under which every row previously took the static branch.

Verified the lane is not vacuous the way you did the gate: forcing the hoisted branch to return its window unfiltered makes wildcard_scan_hides_denied_predicates fail, then restores clean. All 13 it_iceberg_policy tests pass, as does the full fluree-db-query suite. Worth noting for the record that the two native-twin parity tests stay green under that mutation — their denials are caught earlier by the map-level tm_can_yield skip, so the wildcard test is the one pinning filter_rows' static deny path.

[aaj3f]

Note (pre-existing, author-acknowledged; needs a scope decision). This hunk is right: a governed source through the graph-scoped builder now wraps policy when the request carries policy inputs, and a failed lookup propagates instead of reading as "ungoverned".

The commit body also says the graph-scoped builder still ignores opts.policy for native ledgers, and I confirmed it: load_view → load_graph_db_at with no wrap, and neither query_with_options nor query_tracked_with_r2rml_options applies policy from the input. So a native ledger read through fluree.graph(id).query() — the server's proxy-mode route at routes/query.rs:2234 / :2276 — with opts.identity set is unfiltered.

Not introduced here, and this PR narrows rather than widens it, but it's the same class of bypass this PR fixes for virtual sources, so it deserves its own fix soon. I'm not asking for it in this PR only because whether that route should wrap policy for native ledgers (or whether proxy mode is ever policy-bearing) is a call above a review thread.

[bplatz]

Already on the board: #1766, filed before this review landed, labeled bug / P1 / area:query. Your read of the path matches what's in the issue — load_view hands back load_graph_db_at's result unwrapped for native ledgers, and neither of the two execute methods applies policy from the input, so the proxy-mode JSON-LD route reads unfiltered with an identity set. Proxy mode being a real deployment shape (peer role plus proxy storage access) is why it's P1 rather than lower.

I've added it to this PR's "Not in this PR" section as Follow-up: #1766 so the deferral direction is explicit rather than inferred, per the repo's linking convention. Agreed it's a scope call and not a review thread — leaving it for its own PR.

[aaj3f]

Praise. unevaluable_policies with #[serde(default, skip_serializing_if = "Vec::is_empty")] keeps the wire shape additive for every consumer that already parses policy_enforcement (solo's contracts included), and the dataset merge deduping by id (:389-401) means a source governed twice by the same model reports each policy once. Nice.

[aaj3f]

Praise. Treating the gate as a residual filter for the scan-side prune — and, in commit 3, declining top-k under it because a denied row can set the k-th bound and prune files whose visible rows belong in the true answer — is the correctness-preserving-fallback pattern done right. Worth keeping the comment.