feat(plugin): add opt-in Rampart PII redaction

[afourniernv]

Overview

Add pii_rampart as an opt-in, in-process native Rust plugin. It runs the pinned nationaldesignstudio/rampart ONNX model through Tract and sanitizes copied observability events without modifying provider requests, tool arguments, provider responses, or tool results.

This update merges current main into the branch without rebasing and replaces the plugin's bespoke ABI-v3 FFI adapter with Relay's typed ABI-v4 async plugin SDK. It is a compatibility catch-up, not a new middleware surface or a change to application-path behavior.

Rampart remains outside the Relay CLI, language bindings, and default host artifacts. The existing deterministic pii_redaction component remains unchanged and autoloaded as before. Cross-platform packaging and release publication remain in the separate follow-up PR #737.

• I confirm this contribution is my own work, or I have the right to submit it under this project's license.
• I searched existing issues and open pull requests, and this does not duplicate existing work.

Details

• Replace approximately 500 lines of handwritten ABI-v3 runtime, completion, and unsafe FFI plumbing with typed ABI-v4 registrations for mark, scope, tool, and LLM observability sanitizers.
• Use one native executor thread for async callback dispatch while keeping CPU inference on Rampart's dedicated bounded Rayon pool.
• Preserve executor as plugin-host configuration and reject invalid or unknown executor fields before the remaining configuration reaches Rampart.
• Accept invocation-scoped request codec capabilities supplied by the ABI-v4 host. Runtime or opaque response codecs still fail closed because there is no verified response projection capability.
• Bound normalized support to the explicitly qualified OpenAI Chat, OpenAI Responses, Anthropic Messages, and Gemini payload shapes. Explicit normalized OCI fails closed; OCI remains supported through raw trajectory_context sanitization.
• Preserve application non-interference: only queued copies of observability events are sanitized. Provider and tool payloads are never rewritten or blocked.
• Add native coverage for typed runtime request capabilities, OCI raw request/response payloads, concurrent tool activity, explicit subscriber flush, payload non-mutation, and fail-closed projection boundaries.
• Reject normalized Gemini multi-candidate responses so an unchecked candidate cannot leak through copied telemetry.
• Serialize native plugin tests because Relay's native registry and subscriber state are process-global.
• Align the manifest, schema, license metadata, dependency surface, and README with current Relay conventions.

Normalized projection is deliberately shape-bounded. The tests qualify specific payload structures; they do not claim lossless coverage for every multipart, repeated-item, or multi-candidate provider response. Broader response round-trip parity is pre-existing follow-up work rather than part of this ABI-v4 catch-up.

Validation on current head 261f2502:

• just test-rust — passed the complete Relay Rust suite
• cargo test -p nemo-relay-pii-redaction --features rampart — 208 passed
• cargo test --manifest-path plugins/pii-rampart/Cargo.toml — 5 unit tests and 1 schema-default test passed; 2 full-model tests intentionally ignored by default
• Both ignored full-model native tests explicitly invoked together against the rebuilt release dylib — passed
• cargo clippy -p nemo-relay-pii-redaction --features rampart --all-targets -- -D warnings — passed
• cargo clippy --manifest-path plugins/pii-rampart/Cargo.toml --all-targets -- -D warnings — passed
• Changed-file pre-commit suite — formatting, Clippy, Cargo check, dependency audit, attribution, schema, type, and link checks passed

The visible require-nvskills-ci failure is not a Rampart test failure. The reusable check classifies this repository's plugins/ directory as an NVSkills path, but NVSkills validation cannot run for this forked PR and only a maintainer or administrator can request it. The branch itself is green under Relay's local test and check commands.

Where should the reviewer start?

Start with plugins/pii-rampart/src/lib.rs for the typed ABI-v4 registration and codec bridge. Then review plugins/pii-rampart/tests/native_plugin.rs for the application non-mutation, OCI, runtime-codec, concurrency, and flush contract. The normalized payload boundaries are implemented in crates/pii-redaction/src/rampart/sanitizer.rs.

The main decision is intentionally narrow: remove obsolete custom FFI machinery while preserving the plugin's observability-only behavior. This PR does not add a new middleware API, does not change provider/tool execution, does not package the model, and does not absorb PR #737.

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

• Relates to [Enhancement]: Add worker-backed local-model providers for PII redaction #550
• Relates to [Enhancement]: Expose safe typed asynchronous callbacks in the Rust native-plugin SDK #716
• Relates to [Enhancement]: Decouple observability sanitization from managed execution #749

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

Walkthrough

Changes

Rampart PII support adds an ONNX detector, structured prefilter, bounded asynchronous sanitizer, plugin registration, CLI editor integration, and Node, Go, and Python APIs. Host initialization paths register the component and expose configuration validation helpers.

Rampart PII plugin

Layer / File(s)	Summary
Detector and plugin contract crates/pii-redaction/src/rampart/*, crates/pii-redaction/Cargo.toml	Adds configuration validation, tokenization, structured PII prefiltering, verified model loading, and ONNX inference.
Sanitizer callbacks crates/pii-redaction/src/rampart/sanitizer.rs, crates/pii-redaction/src/builtin.rs	Adds bounded executor admission, codec-aware redaction, fail-closed behavior, and asynchronous sanitizer callbacks.
Host and CLI integration crates/ffi/src/api/plugin.rs, crates/python/src/lib.rs, crates/node/src/api/mod.rs, crates/cli/src/server/mod.rs, crates/cli/src/plugins/*	Registers Rampart during host initialization and adds CLI editing, persistence, summaries, schema handling, and registration errors.
Language APIs crates/node/pii_rampart.*, go/nemo_relay/pii_rampart*, python/nemo_relay/pii_rampart.*, python/nemo_relay/__init__.*	Adds configuration types, component builders, metadata constants, validation helpers, package exports, and type declarations.
Validation and supporting updates crates/node/tests/pii_rampart_tests.mjs, go/nemo_relay/pii_rampart*_test.go, python/tests/test_pii_rampart_plugin.py, crates/cli/tests/coverage/shared/plugins_tests.rs, Cargo.toml, ATTRIBUTIONS-Rust.md, .gitattributes	Adds cross-language and CLI coverage, enables the Rampart feature, refreshes attribution metadata, and marks generated files.

Estimated code review effort: 5 (Critical) | ~120 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Host
  participant PluginRegistry
  participant RampartDetector
  participant RampartSanitizer
  participant Guardrail
  Host->>PluginRegistry: register Rampart PII component
  PluginRegistry->>RampartDetector: verify and load model artifacts
  PluginRegistry->>RampartSanitizer: create sanitizer
  PluginRegistry->>Guardrail: register surface callbacks
  Guardrail->>RampartSanitizer: sanitize selected payload
  RampartSanitizer->>RampartDetector: detect text batch
  RampartDetector-->>RampartSanitizer: return scored spans
  RampartSanitizer-->>Guardrail: return redacted payload

Loading

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 35.23% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title follows Conventional Commits format, uses an allowed type and lowercase scope, summarizes the feature, and is 46 characters long.
Description check	✅ Passed	The description includes all required template sections, completed confirmations, detailed changes, reviewer guidance, related issues, and validation results.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 11

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

crates/worker/src/lib.rs (1)

676-705: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Merge push_registration into push_contract_registration to remove duplication.

Both functions build an identical Registration except for the contract field. Python's SDK already unifies this via a single _push_registration(..., *, contract: str = "") (see python/plugin/src/nemo_relay_plugin/_api.py lines 1288-1310); the Rust SDK diverging into two near-duplicate helpers risks future drift (e.g., a new field added to one but not the other).

♻️ Proposed refactor

-    fn push_registration(
-        &mut self,
-        name: &str,
-        surface: RegistrationSurface,
-        priority: i32,
-        break_chain: bool,
-    ) {
-        self.handlers.registrations.push(Registration {
-            local_name: name.into(),
-            surface: surface as i32,
-            priority,
-            break_chain,
-            contract: String::new(),
-        });
-    }
-
-    fn push_contract_registration(
-        &mut self,
-        name: &str,
-        surface: RegistrationSurface,
-        contract: &str,
-    ) {
-        self.handlers.registrations.push(Registration {
-            local_name: name.into(),
-            surface: surface as i32,
-            priority: 0,
-            break_chain: false,
-            contract: contract.into(),
-        });
-    }
+    fn push_registration(
+        &mut self,
+        name: &str,
+        surface: RegistrationSurface,
+        priority: i32,
+        break_chain: bool,
+    ) {
+        self.push_contract_registration(name, surface, priority, break_chain, "");
+    }
+
+    fn push_contract_registration(
+        &mut self,
+        name: &str,
+        surface: RegistrationSurface,
+        priority: i32,
+        break_chain: bool,
+        contract: &str,
+    ) {
+        self.handlers.registrations.push(Registration {
+            local_name: name.into(),
+            surface: surface as i32,
+            priority,
+            break_chain,
+            contract: contract.into(),
+        });
+    }

Then update the call in register_worker_inference to self.push_contract_registration(name, RegistrationSurface::WorkerInference, 0, false, contract);.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/worker/src/lib.rs` around lines 676 - 705, Merge push_registration
into push_contract_registration by giving push_contract_registration priority
and break_chain parameters, with contract supplied as the final argument and
defaulting to an empty string where appropriate. Remove the duplicate helper,
update all callers including register_worker_inference to pass the unified
arguments, and preserve existing Registration field values.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/pii-redaction/src/component.rs`:
- Around line 960-962: Update is_valid_json_pointer_pattern to validate wildcard
segments according to JsonPointerPattern::matches: reject any path segment that
contains '*' unless the entire segment is exactly "*". Preserve existing JSON
Pointer validation for all other segments and continue accepting standalone
wildcard segments.

In `@crates/pii-redaction/src/local.rs`:
- Around line 732-740: The empty-paths branch in llm_sanitize_request_callback
must sanitize request headers as well as request.content, preserving the
intended pointer prefixes for the headers and content roots. Reuse the existing
request/header sanitization behavior used by sanitize_raw_request or the builtin
flow, and ensure the broad-coverage path returns a request with sanitized header
values.

In `@crates/pii-redaction/tests/unit/component_tests.rs`:
- Around line 1916-1921: Update the table-driven assertion in the
validate_plugin_config test loop to include failure context identifying the
current config/field/message case and the produced diagnostics. Preserve the
existing matching condition while supplying a descriptive assertion message so
failures reveal which case failed and the actual report contents.

In `@crates/pii-redaction/tests/worker_detection_tests.rs`:
- Around line 54-58: Remove the duplicate “/message” selector from either
target_paths or target_path_patterns in the test configuration, keeping it in
only one collection so the test expresses a single intent.
- Around line 268-285: Update the fail-closed assertion in the worker exit test
around the event emitted by “worker-pii-exit” to use a message value that the
healthy fixture worker does not redact, while retaining the expected redaction
for “unselected” if applicable. Ensure the assertion can only pass when the
crashed batch is handled fail-closed, rather than matching normal “PRIVATE”
detection behavior.

In `@crates/pii-redaction/workers/rampart/README.md`:
- Around line 134-136: Update the Runtime Bounds section in the README by adding
a complete introductory sentence before the existing bullet list; leave the
documented limits unchanged and ensure the lead-in grammatically introduces the
list.

In `@docs/build-plugins/dynamic-plugins/grpc-worker/python/about.mdx`:
- Line 168: Rename the “Register worker inference” heading to “Register Worker
Inference” in docs/build-plugins/dynamic-plugins/grpc-worker/python/about.mdx at
lines 168-168 and docs/build-plugins/dynamic-plugins/grpc-worker/rust/about.mdx
at lines 71-71, preserving the existing heading structure.

In `@docs/configure-plugins/pii-redaction/configuration.mdx`:
- Around line 424-431: The omission guidance following the “Path Semantics”
section is outdated. Update the paragraph describing manual LLM calls with
normalized target_paths and no active or fallback codec to state that payloads
are sanitized using the configured raw paths and emitted, matching the
early-return behavior in builtin and local redaction flows; preserve the
documented fail-closed contract.
- Around line 329-331: Add a complete introductory sentence immediately before
the TOML code block following the sanitizer registration-rejection paragraph,
clearly describing what the configuration example demonstrates. Keep the
existing TOML content unchanged.

In `@go/nemo_relay/pii_redaction/pii_redaction_test.go`:
- Around line 47-62: Extend the validation condition in the NewComponentSpec
test to assert that spec.Config.Local.Backend matches the configured backend
value from the test setup. Keep the existing configuration assertions unchanged
and include the backend check alongside the other Local fields.

In `@python/plugin/README.md`:
- Around line 108-125: Update the Worker Inference example to establish that ctx
is a PluginContext available inside WorkerPlugin.register, either by showing the
enclosing register method or explicitly stating that scope. Keep the
register_worker_inference usage and handler behavior unchanged.

---

Outside diff comments:
In `@crates/worker/src/lib.rs`:
- Around line 676-705: Merge push_registration into push_contract_registration
by giving push_contract_registration priority and break_chain parameters, with
contract supplied as the final argument and defaulting to an empty string where
appropriate. Remove the duplicate helper, update all callers including
register_worker_inference to pass the unified arguments, and preserve existing
Registration field values.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: b6f7fc1b-5518-49b8-af16-dd1759b6fa64

📥 Commits

Reviewing files that changed from the base of the PR and between 42c7655 and 7541bec.

📒 Files selected for processing (60)

• crates/cli/src/server/mod.rs
• crates/core/src/lib.rs
• crates/core/src/plugin.rs
• crates/core/src/plugin/dynamic/host.rs
• crates/core/src/plugin/dynamic/worker.rs
• crates/core/src/plugin/worker_inference.rs
• crates/core/tests/fixtures/worker_plugin/src/main.rs
• crates/core/tests/integration/worker_plugin_tests.rs
• crates/core/tests/unit/dynamic_worker_tests.rs
• crates/core/tests/unit/plugin_tests.rs
• crates/core/tests/unit/worker_inference_tests.rs
• crates/node/pii_redaction.d.ts
• crates/node/pii_redaction.js
• crates/node/tests/pii_redaction_tests.mjs
• crates/pii-redaction/Cargo.toml
• crates/pii-redaction/README.md
• crates/pii-redaction/src/builtin.rs
• crates/pii-redaction/src/component.rs
• crates/pii-redaction/src/local.rs
• crates/pii-redaction/tests/unit/component_tests.rs
• crates/pii-redaction/tests/unit/local_tests.rs
• crates/pii-redaction/tests/worker_detection_tests.rs
• crates/pii-redaction/workers/rampart/MANIFEST.in
• crates/pii-redaction/workers/rampart/README.md
• crates/pii-redaction/workers/rampart/THIRD_PARTY_NOTICES.md
• crates/pii-redaction/workers/rampart/config.schema.json
• crates/pii-redaction/workers/rampart/nemo_relay_pii_rampart/__init__.py
• crates/pii-redaction/workers/rampart/nemo_relay_pii_rampart/detector.py
• crates/pii-redaction/workers/rampart/nemo_relay_pii_rampart/prefetch.py
• crates/pii-redaction/workers/rampart/nemo_relay_pii_rampart/py.typed
• crates/pii-redaction/workers/rampart/nemo_relay_pii_rampart/worker.py
• crates/pii-redaction/workers/rampart/pyproject.toml
• crates/pii-redaction/workers/rampart/relay-plugin.toml
• crates/pii-redaction/workers/rampart/tests/test_detector.py
• crates/pii-redaction/workers/rampart/tests/test_worker.py
• crates/worker-proto/README.md
• crates/worker-proto/proto/nemo/relay/worker/v1/plugin_worker.proto
• crates/worker-proto/tests/proto_tests.rs
• crates/worker/README.md
• crates/worker/src/lib.rs
• crates/worker/tests/worker_sdk_tests.rs
• docs/about-nemo-relay/release-notes/index.mdx
• docs/build-plugins/dynamic-plugins/grpc-worker/grpc-worker-protocol.mdx
• docs/build-plugins/dynamic-plugins/grpc-worker/python/about.mdx
• docs/build-plugins/dynamic-plugins/grpc-worker/rust/about.mdx
• docs/configure-plugins/pii-redaction/about.mdx
• docs/configure-plugins/pii-redaction/configuration.mdx
• go/nemo_relay/pii_redaction.go
• go/nemo_relay/pii_redaction/pii_redaction.go
• go/nemo_relay/pii_redaction/pii_redaction_test.go
• go/nemo_relay/pii_redaction_test.go
• justfile
• python/nemo_relay/pii_redaction.py
• python/nemo_relay/pii_redaction.pyi
• python/plugin/README.md
• python/plugin/src/nemo_relay_plugin/__init__.py
• python/plugin/src/nemo_relay_plugin/_api.py
• python/tests/plugin/test_public_api_docstrings.py
• python/tests/plugin/test_worker_sdk.py
• python/tests/test_pii_redaction_plugin.py

[ericevans-nv]

You could refine local_model to resolve an integration-provided callback. The PII middleware would execute that callback with the selected text and detector settings, then receive the detected spans, labels, and confidence scores. The integration could implement the callback using any model, runtime, or transport it chooses, while the PII component continues to own field selection, detection validation, policy, and redaction.

[copy-pr-bot]

/ok to test c7c2f371d4a7f862d1062944219a678736672c3b

@afourniernv, there was an error processing your request: E2

See the following link for more information: https://docs.gha-runners.nvidia.com/cpr/e/2/

[afourniernv]

/ok to test 699e823

[afourniernv]

@willkill07 what u think?

[afourniernv]

/nvskills-ci

[afourniernv]

@willkill07 Review-readiness update for current head 261f2502:

• The July 28 requested-change reviews were against the earlier synchronous implementation. The current branch uses Relay's typed ABI-v4 async plugin SDK and Rampart's bounded inference executor; all review threads are resolved.
• just test-rust passes on the current head.
• The changed-file pre-commit suite passes, including formatting, Clippy, Cargo check, dependency audit, attribution, type, schema, and link checks.
• Both full-model native tests were explicitly invoked against the rebuilt plugin and pass.

The remaining red require-nvskills-ci check is external to Rampart. The reusable workflow classifies NeMo Relay's plugins/ directory as an NVSkills path, but this PR is from a fork. My /nvskills-ci request was rejected because that command requires maintain/admin permission, and the gate states that forked branches are unsupported. Could you clear or reroute that check, approve the current head for NVIDIA runner CI, and re-review when available?

Keeping this PR in draft until that external CI path and reviewer re-review are available.