Host conformance: where the novasama host differs from TrUAPI

[decrypto21]

Spans: paritytech/truapi, paritytech/triangle-js-sdks (@novasamatech/host-api + host-container)

Summary:
the wire shapes - envelope, wire-id table, and most payload/error types - match between the generated @parity/truapi reference and the latest hand-written novasama host. Two divergences are still open and need code changes, and they point in different directions: (D1) on framework faults the current reference runtime (@parity/truapi-host) is silent while novasama correctly emits a terminal error frame - a fix on the reference side; (D2) novasama's PaymentTopUp encoding (error-variant order + key size) does not match the crate - a fix on the host-api side. A third gap is Android-only handler wiring. Transport, rate-limiting, and coin_payment are out of scope (below). This is not a "bump a version" cleanup.

Background

TrUAPI is the one wire language a product speaks to a host: fixed methods, fixed request/response/error shapes, a fixed numeric wire id each, all SCALE-encoded (positional binary - field and variant order is the contract). It is defined by the Rust crate rust/crates/truapi (the source of truth), from which the @parity/* code is generated.

Two things implement the host contract, and either can diverge:

• novasama @novasamatech/host-api + host-container - hand-written TypeScript. Its codecs are hand-maintained and can drift from the crate (D2).
• the generated reference runtime - today the TypeScript @parity/truapi-host; the Rust truapi-server → @parity/truapi-host-wasm is the successor but lives on an unmerged branch (PR-104). The current TS reference host has a behavioural gap the WASM runtime fixes (D1).

Every response frame:

┌───────────┬───────────┬────────────┬────────────────┬───────────┐
│ requestId │ action id │ version    │ Result tag     │ payload   │
│  (str)    │ (1 byte)  │ V1 = 0x00  │ Ok=00 / Err=01 │ per type  │
└───────────┴───────────┴────────────┴────────────────┴───────────┘

What conforms

• Envelope + wire-id table - [requestId str][action id u8][version V1=0x00][Result Ok=0x00/Err=0x01][payload], verified end-to-end through encodeWireMessage and the generated indexedTaggedUnion({V1:[0,...]}). The table has 55 non-coin_payment method-actions plus 9 coin_payment (ids ≈136-163).
• Payload/error types cross-checked are byte-identical except PaymentTopUp (D2): e.g. HostSignPayloadData (15 fields), HostGetUserIdResponse { primary_username }, Statement/SignedStatement with proof enums, chain types, genesis_hash as variable Vec<u8>.
• Error channel: there are 22 concrete per-method domain error enums (plus a shared GenericError struct, referenced by ~17 versioned wrappers). The generic CallError tier (Domain / Denied / Unsupported / MalformedFrame / HostFailure) is stripped from the wire by codegen - it derives no Encode/Decode and is unwrapped to its inner domain type D (truapi-codegen/src/ts.rs, skip list in rustdoc.rs). So the only typed error on the wire is the per-method domain enum. This matters for D1.

Coverage honesty: the differential cross-checked ≈18-19 distinct payload types (shared types inherit); full per-method input-vector coverage across all 55 is the CI's job (T3), not yet executed. "Byte-identical everywhere" is the target the CI enforces, not a fully-executed claim today.

Open divergences (each needs a code change)

Ordered by severity. D1 first: it is the one where the caller literally hangs.

D1 · Framework faults: the reference runtime is silent; novasama emits a frame → reference-side fix

Call a method the host doesn't implement, denies, or that throws:

• novasama host-container returns a terminal Result::Err frame - it folds the fault into the per-method domain enum's catch-all (…Err::Unknown { reason }) and answers. The caller gets an error.
• the current reference @parity/truapi-host (TS dispatcher) sends nothing: unregistered wire id → drop (default onUnknownFrame); handler throw/reject → swallow; subscription-start error → swallow. The caller hangs / times out.

Root cause: the generic CallError tier has no wire representation (stripped by codegen, above), and the crate's default handler bodies return CallError::unavailable() = HostFailure { reason }, which cannot be SCALE-encoded; required methods (e.g. navigate_to) have no default body at all. So the TS reference has no typed channel for a framework fault - it drops. The "Not implemented → domain Unknown" mapping is host-side behaviour (novasama does it; dotli does it), not something the crate provides.

The Rust truapi-server (WASM successor, on the PR-104 branch) does emit a terminal frame on handler Err - so landing it closes this on the reference side.

Close: land the WASM runtime (T1, emits terminal frames), and have the conformance CI assert a terminal typed frame on faults - i.e. assert novasama's behaviour, not "match the silent TS host" (T3).

D2 · PaymentTopUp encoding drift (variant order + key size) → host-api code change

Variant order. The crate's HostPaymentTopUpError (v01/payment.rs) declares InsufficientFunds(0), InvalidSource(1), Unknown(2), PartialPayment(3). Current host-api (0.8.3, payments.ts) declares PartialPayment before Unknown, so their discriminants are swapped:

                    host-api 0.8.3      crate (source of truth)
Unknown{reason}     0x03 …              0x02 …
PartialPayment{c}   0x02 …              0x03 …

(0.8.3 added PartialPayment in the wrong slot; it was never a later reorder.)

Key size. The crate's PaymentTopUpSource uses 64-byte sr25519 secret keys ([u8; 64]); host-api still encodes Bytes(32). Divergent.

Close (T2): in host-api, reorder PaymentTopUpError so Unknown precedes PartialPayment (matching the crate) and widen PaymentTopUpSource keys to 64-byte sr25519. This is a code change, not a version bump - re-derive any version specifics from a current checkout / the npm registry (the local triangle-js-sdks is 0.8.3 and still divergent; there is no 0.8.4 in it).

D3 · get_user_id is unregistered on Android → host wiring

The codec matches on both sides (HostGetUserIdResponse { primary_username } ↔ host-api UserIdentity { primaryUsername }). dotli, iOS, and desktop each register a real handler returning { primaryUsername }; only Android leaves it unregistered, so the container falls back to GetUserIdErr::Unknown { reason: "Not implemented" }. (This is a domain Unknown, not an Unsupported - there is no Unsupported wire variant.)

Close (T4): register the get_user_id handler on Android, or confirm the omission is intentional.

Known & out of scope

• Transport establishment - by design, not a task. The client and the host establish the byte channel differently (MessagePort vs raw window.postMessage, with different readiness handshakes across implementations). Same SCALE bytes, different carrier. A product selects the provider that matches its host; this is inherent to the client/host integration layer, and swapping it is expected on a migration. Out of scope for wire conformance. (The exact handshake mechanism differs between the base repo and the PR-104 sandbox layer; don't rely on a single "MessagePort vs postMessage" dichotomy.)
• Rate-limiting - integrator-owned on both sides, not a divergence. host-container ships createRateLimiter as a standalone opt-in util; createContainer never wires it. TrUAPI standardizes no rate-limiting (no method, RFC, or wire id), and the runtime ships none. Whether the runtime should ever offer it as a framework capability is an open question below, not a conformance gap.
• coin_payment - a shared future milestone, not a divergence. CoinPayment is part of the unified TrUApi trait (9 host methods, ids ≈136-163), so a fully-conformant host implements it eventually. But host-api has none (only the CoinPaymentPurseId = u32 selector), and on the reference side it is explicitly deferred: the PR-104 truapi-server carries an empty impl CoinPayment for ProductRuntimeHost {} ("outside this milestone"). Both sides unimplemented → nothing to close today.
• statement_store/subscribe decode error seen in testing - reproduces only against a checkout with a mismatched host-api install (0.7.9 resolved under a 0.8.2 pin). Statement types are byte-identical across the crate, @parity/truapi, and current host-api; a clean install resolves it. Not a protocol difference.

Tasks

#	Task	Closes	Owner (repo)
T1	Land + publish the WASM reference runtime (truapi-server → @parity/truapi-host-wasm): it emits terminal frames on framework faults (closes D1) and gives the CI a stable runtime reference. Currently on the PR-104 branch and unpublished; base truapi still ships the TS @parity/truapi-host.	D1	truapi
T2	host-api code change: reorder HostPaymentTopUpError so Unknown precedes PartialPayment (match the crate) + widen PaymentTopUpSource to 64-byte sr25519 keys. Re-derive version specifics from a current checkout / npm.	D2	triangle-js-sdks
T3	Conformance CI in triangle-js-sdks: per distinct type decode → re-encode + direct dual-encode (catches fixed-byte-size drift scale-ts skips on decode) + failure-path frames, vs the crate codecs (@parity/truapi, published 0.3.x). Assert a terminal typed frame on faults (novasama's behaviour). The codec half can run today; the runtime/behavioural half waits on T1.	D2 (+ D1 oracle)	triangle-js-sdks
T4	Register the get_user_id handler on Android (or confirm the omission is intentional).	D3	polkadot-app-android

Open questions

• Rate-limiting ownership - TrUAPI standardizes none; host-container offers createRateLimiter integrator-side

[filvecchiato]

Thanks for looking into this! regarding D2: it was solved here but I think I did not push it to main. I will raise a PR now to fix it.

[filvecchiato]

On further checking, on main it also follows the same order, and checking the ordering in triangle-js-sdk seems to be matching, am I looking in the wrong place?