[Abhishek Acharya (abhishekacharya200)]

fix(hooks): preserve nested metadata in native JSON

Description

TL;DR

Restore lossless native hook JSON during apm install and apm update without changing the immutable hook IR or supported adapter behavior. #3211 remains the sole implementation; selected, nonredundant regression tests from #3199 are incorporated with author credit.

Problem (WHY)

• The reported failure was Object of type mappingproxy is not JSON serializable during hook integration (#3167).
• _freeze recursively snapshots metadata, but shallow dict(...) copies and raw-entry passthrough left nested mappings frozen.
• [!] Rendering needs independent containers, not stringification or dropped metadata. The regression proof follows the validation-loop guidance: "do the work, run a validator (a script, a reference checklist, or a self-check), fix any issues, and repeat until validation passes."

Approach (WHAT)

• Keep one recursive _thaw beside _freeze in hook_contract.py.
• Route handler, binding, nested raw-entry and flat raw-entry rendering through that owner.
• Preserve mappings, arrays and scalars as independent native JSON values.
• Keep existing matcher, command, timeout conversion and provenance behavior.
• Adopt fix(hooks): serialize nested hook metadata in native hook files #3199's additional test scenarios, not its duplicate renderer-local helper.

Implementation (HOW)

File	Change
src/apm_cli/hook_contract.py	Own recursive mapping-to-dict and sequence-to-list materialization.
src/apm_cli/integration/hook_native_formats.py	Use that owner at all four rendering boundaries.
scripts/architecture_linter/checks/mutation_hook_contract.py	Register static protection against shallow metadata copies.
tests/integration/test_architecture_contract_guards.py	Cover five rendering cases, Gemini compatibility, mutation isolation, static guards and four real native-file writes.
tests/unit/integration/test_hook_native_metadata.py	Adopt binding/provenance, raw-array and source/IR isolation scenarios from #3199.
tests/integration/test_hook_nested_metadata_install.py	Adapt #3199's CLI regression to a local package, normal approval, install/update and a no-execution sentinel.
docs/src/content/docs/producer/author-primitives/hooks-and-commands.md	Document preserved nested metadata.
packages/apm-guide/.apm/skills/apm-usage/package-authoring.md	Keep packaged hook guidance consistent.
CHANGELOG.md	Retain one fix entry and credit both contributors.

Diagrams

The dashed stage materializes independent native output; the source and immutable IR remain unchanged.

flowchart LR
    subgraph Source["Source"]
        S["Nested hook JSON"]
    end
    subgraph Contract["Canonical contract"]
        I["Immutable HookDocument"]
        T["hook_contract._thaw"]
    end
    subgraph Native["Existing native adapters"]
        O["Claude / Codex / Gemini / Antigravity JSON"]
    end
    S --> I
    I --> T
    T --> O
    classDef new stroke-dasharray: 5 5
    class T new

Loading

Trade-offs

• Retain fix(hooks): render nested hook metadata as plain JSON #3211's canonical owner rather than importing a second equivalent _thaw from fix(hooks): serialize nested hook metadata in native hook files #3199; combining both full implementations would shadow the canonical import.
• The CLI regression uses local fixtures and normal apm approve, not fix(hooks): serialize nested hook metadata in native hook files #3199's test-only --no-policy flag. No production policy bypass was present in fix(hooks): serialize nested hook metadata in native hook files #3199.
• The update fixture removes its local lock and generated configs before update --yes, exercising materialization rather than an unchanged-ref no-op. --yes accepts the update plan; hook consent remains explicit.
• No new hook schemas, events, targets, trust rules, runtime execution or release commitment.

Benefits

1. Five rendering cases preserve nested JSON values without default=str.
2. Mutating handler, binding or raw output cannot mutate source metadata or the frozen IR.
3. Four actual native-file writes and a normal-policy CLI lifecycle cover the reported failure.

Issue and approved scope

Issue: #3167

Human scope-approval comment: #3167 (comment)

This PR completes that bounded scope. The maintainer subsequently selected #3211 as the sole landing vehicle, with useful #3199 tests retained and credited. No action closing or merging #3199 is part of this PR.

Fixes #3167

Note

Original Abhishek Acharya (@abhishekacharya200) commits are preserved. The selected tests from Mohammed Alkindi (@MohammedAlkindi)'s #3199 commit b36ed0ff2a60ee1974172d4d40bf21760ada7f7a are credited in commit c26ea4976b8591673e2067a92f525d0894c28beb using the original commit's author identity.

Type of change

• Bug fix
• New feature
• Documentation
• Maintenance / refactor

Testing

• Tested locally
• All existing tests pass
• Added tests for new functionality (if applicable)

The full local repository suite was not run. The hook consumer suite and full registry mutation matrix passed at exact head 247d486801b73a29a70c0380533f89a43bc10bcb, including main 65b3b9f4172feaae5be9118e9a8b672f7a527481.

Validation

uv run --frozen --extra dev pytest tests/integration/test_architecture_contract_guards.py tests/unit/integration/test_hook_native_metadata.py tests/integration/test_hook_nested_metadata_install.py tests/integration/test_architecture_owner_rule_mutations.py -q --basetemp=../pr-3211-artifacts/consolidation/restart-65b3/pytest-exact --junitxml=../pr-3211-artifacts/consolidation/restart-65b3/tests-exact.xml

506 passed in 1360.91s (0:22:40)

This includes all 46 retained hook contract/metadata/CLI cases and the full registry matrix, including test_matrix_covers_every_registry_guard_exactly_once and the registry-delegation-build-id-verification mutation case repaired on main by #3251.

Exact local lint and consumer-path evidence

Command	Result
uv run --frozen --extra dev ruff check src/ tests/ scripts/lint_architecture_boundaries.py scripts/architecture_linter/ scripts/windows_native_symlink_probe_entry.py	Passed
uv run --frozen --extra dev ruff format --check src/ tests/ scripts/lint_architecture_boundaries.py scripts/architecture_linter/ scripts/windows_native_symlink_probe_entry.py	Passed
uv run --frozen --extra dev python -m pylint --disable=all --enable=R0801 --min-similarity-lines=10 --fail-on=R0801 src/apm_cli/ scripts/lint_architecture_boundaries.py scripts/architecture_linter/ scripts/windows_native_symlink_probe_entry.py	Passed
bash scripts/lint-auth-signals.sh	Passed
bash scripts/lint-architecture-boundaries.sh	Passed
uv run --frozen python scripts/check_test_assertions.py	Passed
uv run --frozen python scripts/check_exact_test_duplicates.py	Passed

The CI-equivalent YAML I/O, 2100-line and portable-relative-path guards passed using equivalent Python scans on macOS. The owner-v2 schema and full semantic verifier passed with five exact-head consumer test IDs; no blocked-return shortcut is used.

The installed .venv/bin/apm independently passed unapproved install (no native hooks emitted), approve ../hookpkg, approved install, and update --yes after fixture lock/config removal. Claude/Codex JSON and original source bytes were checked; the hook-execution sentinel remained absent. No --no-policy or --force was used.

Historical mutation proof remains attributed to bc941e0a4cd55f2b610107fede64d29fb17012bd: replacing _thaw with identity and unregistering _nhc_native_metadata_thaw produced 28 failed, 1 passed, 17 deselected in 28.58s. Guards were restored byte-for-byte. The protected implementation, static guard and PR regression files are unchanged in this recovery; no new execution of that historical mutation is claimed.

Scenario Evidence

#	Scenario (user promise)	Principle(s)	Test(s) proving it	Type
1	Install/update hooks with nested metadata without crashing or executing hooks	DevX, Secure by default, Governed by policy	tests/integration/test_hook_nested_metadata_install.py::test_install_update_write_nested_hook_metadata (regression-trap for #3167)	integration
2	Keep metadata, matcher, command and timeout across native targets	Multi-harness support	tests/integration/test_architecture_contract_guards.py::test_integrator_writes_nested_hook_metadata_to_native_json	integration
3	Preserve binding provenance and raw nested arrays	Multi-harness support	tests/unit/integration/test_hook_native_metadata.py	unit
4	Editing generated metadata never changes source or immutable IR	Secure by default	tests/unit/integration/test_hook_native_metadata.py::test_rendered_binding_handler_and_raw_output_are_independent_of_ir_and_source	unit

Post-release recovery and authorship

Normal merge 247d486801b73a29a70c0380533f89a43bc10bcb retains published parent 2b794ed5f7a80ef834cc925e20d2ba01b4b440ad and main 65b3b9f4172feaae5be9118e9a8b672f7a527481. All eight non-changelog PR files and their semantic diff are byte-identical to the previously reviewed head. Git auto-merged the changelog but misplaced the undelivered fix and restored obsolete duplicate release entries; the correction retains main's dated v0.34 release bytes exactly and places only the identical #3211 entry under Unreleased.

Original Abhishek Acharya (@abhishekacharya200) commits and Mohammed Alkindi (@MohammedAlkindi)'s selected-test credit in c26ea4976b8591673e2067a92f525d0894c28beb remain ancestors. No duplicate thaw implementation, trust bypass or unlanded #3153 change is imported. The genuine prior specialist/CEO ship_now carries forward on this proven faithful integration and refreshed exact-head evidence, not on a historical badge. No broad panel was rerun.

Retained budgets: 4/4 convergence iterations, 2/2 Copilot rounds and 0/3 CI recovery iterations. The two prior Copilot findings remain resolved by the original author. No budget was reset.

Technical CI

Observed standard workflows for exact head 247d486801b73a29a70c0380533f89a43bc10bcb:

Workflow	Observed result
Deploy Docs	completed / success
CI	completed / success
Spec conformance	completed / success
Merge Gate	completed / success
NOTICE Drift Check	completed / success
CodeQL	completed / success

Docs build passed; deployment is skipped for PRs. Main's former registry-matrix blocker is fixed and the full matrix now passes locally. The PR remains open and non-draft. No merge, auto-merge, queue, assignment, reviewer request, competing-PR closure or policy change was performed. The parent independently verifies and serializes any landing.

How to test

• Run the exact pytest command above; expect the shown pass count and no skipped matrix cases.
• Confirm the CLI regression requires normal approval, regenerates native JSON on update, and never creates its execution sentinel.
• Run bash scripts/lint-architecture-boundaries.sh; expect exit 0.

Spec conformance (OpenAPM v0.1)

If this PR changes behaviour that an OpenAPM v0.1 req-XXX covers,
confirm the three-step ritual in the
development guide:

• Spec edit: docs/src/content/docs/specs/openapm-v0.1.md updated
  (new/changed <a id="req-XXX"></a> anchor + prose + Appendix C
  row).
• Manifest edit: docs/src/content/docs/specs/manifests/openapm-v0.1.requirements.yml
  updated.
• Test edit: a @pytest.mark.req("req-XXX") test under
  tests/spec_conformance/ added or extended.
• CONFORMANCE.{md,json} regenerated via
  uv run --extra dev python -m tests.spec_conformance.gen_statement
  and committed.
• N/A -- this PR does not change OpenAPM-observable behaviour.

This restores existing supported metadata serialization; it introduces no normative schema or requirement.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

[Copilot]

🟡 Changes recommended

The canonical thawing path lacks its required static architecture guard, and the changelog entry omits the PR reference.

2 open findings

• Require architecture and mutation guards for canonical _thaw rendering · New
• Add pull request reference to the Unreleased changelog entry · New

What changed in this PR

Fixes nested hook metadata serialization while preserving immutable hook IR.

Changes:

• Adds recursive metadata thawing at native render boundaries.
• Adds regression coverage across supported renderers.
• Updates documentation and changelog.

File	Description
src/​apm_cli/​hook_contract.py	Adds recursive metadata thawing.
src/​apm_cli/​integration/​hook_native_formats.py	Thaws metadata before native rendering.
tests/​integration/​test_architecture_contract_guards.py	Tests serialization and IR independence.
docs/​src/​content/​docs/​producer/​author-primitives/​hooks-and-commands.md	Documents nested metadata preservation.
CHANGELOG.md	Records the bug fix.

🧠 Review effort: Balanced

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Abhishek Acharya (abhishekacharya200)]

@microsoft-github-policy-service agree

[Daniel Meppiel (danielmeppiel)]

APM Review Panel: ship_now carried forward

The prior inherited registry-matrix blocker is repaired on main by #3251. This bounded recovery now has passing exact-head evidence; #3211 remains the sole #3167 vehicle with selected #3199 tests and both contributors credited.

Review and faithful integration

• Genuine prior specialist and CEO review returned zero remaining findings and ship_now. That recommendation carries forward to 247d486801b73a29a70c0380533f89a43bc10bcb only after proving all eight non-changelog PR files and their semantic delta unchanged and rerunning the functional gates. No new persona or broad panel is claimed.
• Normal merge parents: published head 2b794ed5f7a80ef834cc925e20d2ba01b4b440ad and main 65b3b9f4172feaae5be9118e9a8b672f7a527481. Original Abhishek Acharya (@abhishekacharya200) commits and Mohammed Alkindi (@MohammedAlkindi)'s author trailer in c26ea4976b remain in history.
• Git auto-merged rather than reporting an unresolved file. The manual changelog correction preserves the dated v0.34 release bytes exactly, removes obsolete duplicate release text introduced by auto-merge, and keeps the undelivered fix(hooks): render nested hook metadata as plain JSON #3211 fix under Unreleased.
• The canonical recursive _thaw, all four rendering boundaries, immutable IR, consent behavior, regression traps, Starlight page and packaged guide are unchanged. No unlanded fix(hooks): type untyped flat command handlers for Claude #3153 change or duplicate fix(hooks): serialize nested hook metadata in native hook files #3199 implementation was imported.
• Both Copilot findings remain resolved in 832a0c8. Retained counters are 4/4 convergence, 2/2 Copilot and 0/3 CI recovery; no reset or additional fold iteration.

Current-head execution

• Functional and full registry matrix: 506 passed in 1360.91s (0:22:40). Includes the 46 retained hook cases, exact registry coverage and registry-delegation-build-id-verification.
• Installed CLI: unapproved install, normal approve, approved install and regenerated update passed. Native nested JSON and source bytes remain intact, and no hook executed.
• Lint and owners: all seven canonical gates, architecture lint, assertion/duplicate ratchets, owner-v2 schema and full semantic verification passed. Five executed exact-head consumer IDs cover the detected owner.
• Mutation provenance: the actual prior bc941e0a proof had 28 intended failures and one passing control. Protected files are unchanged; no rerun of that historical proof is claimed. Main's matrix mutations ran freshly as part of the current suite.
• Docs/body: preserved truthful Starlight and packaged guidance; this body and advisory replace the stale wind-down/blocker account.

Exact-head standard technical workflows

Workflow	Observed result
Deploy Docs	completed / success
CI	completed / success
Spec conformance	completed / success
Merge Gate	completed / success
NOTICE Drift Check	completed / success
CodeQL	completed / success

Docs deployment was skipped for the PR; the build passed. Workflow execution approval was restricted to these reviewed-head standard workflows, byte-identical to trusted main.

Disposition

Current code/test recommendation is ship_now on proven faithful integration; platform admission and any landing remain the parent's independent responsibility. The PR is open and non-draft. No merge, auto-merge, queue, new PR, reviewer request, assignment, original/competing closure, automation or policy change was made.

Push: git push --force-with-lease=fix-hook-metadata-json-rendering:2b794ed5f7a80ef834cc925e20d2ba01b4b440ad https://github.com/abhishekacharya200/apm.git HEAD:fix-hook-metadata-json-rendering (fast-forward history, explicit concurrency lease).

---

Generated by autopilot-pr-merge-worker. This comment is AI-generated and may contain errors.