feat(sdk): read the off-spec tdf_spec_version manifest field

[pflynn-virtru]

Proposed Changes

schemaVersion is the correct name for the TDF spec version field, and is what this SDK writes. tdf_spec_version is not a former spelling that was later renamed — it is an error that leaked into some specification drafts and some older OpenTDF documentation, and into the writers built from them. This PR makes the Go reader tolerate that error so the affected files stay usable. It does not put the name on a deprecation path, because it was never correct.

• Read: add Manifest.UnmarshalJSON. When schemaVersion is absent, fall back to tdf_spec_version at the manifest root, then to tdf_spec_version under payload. Both placements are real — writers disagree, and this SDK's own bundled schemas (sdk/schema/manifest*.schema.json) reproduce the mistake under payload while never declaring the root schemaVersion it actually emits.
• Precedence: schemaVersion always wins when both names are present; between the two off-spec locations, root wins over payload. No error on conflict.
• Write: unchanged. The default marshaller still emits schemaVersion only, so the error is never propagated — including by code that re-emits a manifest it just read. A regression test locks this in.
• Tolerance: an off-spec value type is ignored rather than fatal. The lax schema permits tdf_spec_version: null, and reporting malformed manifests is schema validation's job, not the decoder's.
• Experimental: sdk/experimental/tdf/manifest.go is a deliberate duplicate of sdk/manifest.go, so it gets the same decoder. That package has no manifest read path today, so this changes no behavior there — it just keeps the two from drifting.

The JSON schemas are deliberately untouched. Neither sets additionalProperties: false, so both spellings already validate; correcting the schema/spec discrepancy is separate work, not this PR.

The spec version no longer affects verification

Reading a second name for the field exposed a deeper problem, so this PR fixes it too.

The reader used to pick the integrity digest encoding from the field's presence: isLegacyTDF := r.manifest.TDFVersion == "" at sdk/tdf.go:965, :1070, :1478, :1633. That only worked because our writer sets useHex and excludeVersionFromManifest from one boolean (sdk/tdf_config.go:310), so "field present" meant "raw digests" — for files we wrote. Nothing authenticates the field, and writers that decoupled the two (including every writer that emitted the off-spec name) break the correspondence. Reading tdf_spec_version made it worse: a hex-digest container carrying that field would have been verified as raw and rejected.

digestMatchesRecorded replaces the flag. It recomputes the raw digest and compares against the manifest value in both spellings — the same move #3597 makes for the policy binding, where the encodings are distinguished by length "so no version signal is needed". Accepting both weakens nothing: hex is an invertible encoding of the same HMAC, so forging either form still requires the payload key.

The assertion path can't use length — the hash is concatenated into a larger buffer before signing, leaving no trace of its encoding — so it builds both candidates and accepts either, sound on the same grounds.

The writer is untouched and still honors WithTargetMode; the bool those helpers take is renamed useHex to say what it selects.

This also fixes a pre-existing bug on the canonical path: schemaVersion: "4.2.0" was treated as non-legacy despite being below the hex threshold.

Closes #4059.

Checklist

• I have added or updated unit tests
• I have added or updated integration tests — Test_OffSpecSpecVersionIsReadable decrypts real containers through CreateTDF/LoadTDF against the fake KAS backend
• I have added or updated documentation — not applicable; the behavior is documented on Manifest.UnmarshalJSON, and no public API changed

Testing Instructions

# new tests
$ cd sdk && go test -run 'TestManifest' -count=1 -v .
--- PASS: TestManifest_UnmarshalJSON_SpecVersion          (11 subtests)
--- PASS: TestManifest_Marshal_EmitsSchemaVersionOnly
--- PASS: TestManifest_RoundTripNormalizesSpecVersion     (2 subtests)
ok      github.com/opentdf/platform/sdk

$ go test -run 'TestTDF/Test_OffSpecSpecVersionIsReadable' -count=1 -v .
--- PASS: TestTDF/Test_OffSpecSpecVersionIsReadable
    tdf_spec_version_at_root
    tdf_spec_version_under_payload
    control:_no_spec_version_at_all_is_read_as_pre-4.3.0_and_fails
ok      github.com/opentdf/platform/sdk

# regression guards
$ go test -run 'TestTDF/Test_ValidateSchema' -count=1 .        # schema strict/lax behavior unchanged
ok      github.com/opentdf/platform/sdk
$ go test -run TestREADMECodeBlocks -count=1 .
ok      github.com/opentdf/platform/sdk

# full suites
$ cd sdk && go test ./... -race -count=1
ok      github.com/opentdf/platform/sdk                        8.448s
ok      github.com/opentdf/platform/sdk/experimental/tdf       3.214s
ok      ... (all packages pass)

$ gofmt -l <changed files>    # clean
$ go vet ./...                # clean

Two things could not be verified locally, both pre-existing and unrelated to this change — each reproduced identically on a stashed clean tree:

• make fmt / make lint abort before running: the installed golangci-lint 2.9.0 is built with go1.26 while the repo targets go1.27.1 (the Go language version (go1.26) used to build golangci-lint is lower than the targeted Go version (1.27.1)). Formatting was checked with gofmt -l and gofumpt -l instead, both clean.
• make test fails in lib/fixtures and parts of service (TestTokenManager_*, service/integration, service/pkg/server, service/rttests) — these need the docker compose Keycloak/Postgres stack, which was not running (dial tcp [::1]:8888: connect: connection refused).

Cross-SDK interop via opentdf/tests xtest has not been run yet; the java/js readers are the real counterparties here and it is worth a run before this leaves draft.

🤖 Generated with Claude Code

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[pflynn-virtru]

Superseded by #4060, which is the same work on a branch whose name carries no internal tracker reference. No review had happened here.