No guide for building a language integration on Stack Encrypt — write one, with the wire format documented in vitaminc-aead-frame #1254

Background

Stack Encrypt (packages/stack-encrypt) is CipherStash's Rust encryption engine. Rust programs use it directly. Every other language reaches it through a binding: the FFI or WASI interface between the Rust engine and the language. A language SDK is built on top of that binding, and it's what that language's users work with (docs/sdk-design-principles.md). The Go SDK (languages/golang) is the first. A Node SDK to replace protect-ffi is planned, and we want third parties to be able to build integrations for other languages.

Problem

Nothing explains how to build a binding. The knowledge exists, but it is spread over code comments, ADRs and two repositories, and an author has to reconstruct it from the Go SDK:

• The engine's side (this repo): the dynamic feature and its API (dynamic::record, query_term, verify), the plan grammar, how an EQL type is reached by name, the reasons a call is refused, and ADR-0007 ("bindings enter through a plan, never a second executor").
• The wire (vitaminc-aead-frame, in cipherstash/vitaminc): the value frame, the ciphertext tree, the tag table, the depth and length limits, and the conformance corpus.
• The host's duties: where plaintext may and may not be copied, what must be wiped and when, how keys reach ZeroKMS (the Go guest's transport_send and token_get imports), how errors come back (se_last_error), and how the host and engine versions are kept in step.
• Which shape to build: a native library is the default, with WASI only where it is the reason (Go, edge runtimes, sandboxing).

Reviewing the Go SDK's value path took hours, and turned up the questions this guide should answer. Which codec does the Go host use? Is it vitaminc 0.6's frame? Which types are control data and which are plaintext? Where is plaintext held on each path? A third-party author has no maintainer to ask.

Proposal

1. Write the guide in packages/stack-encrypt, as the crate documentation for the dynamic module or a docs/ page it links to, so it ships with the crate on docs.rs. It covers:
   • binding versus language SDK;
   • native versus WASI, and how to choose;
   • the three operations (encrypt a record, query, decrypt) with the exact calls;
   • the plan grammar;
   • control messages versus plaintext (stack-encrypt: dynamic control messages (plan, options, query results) are parsed from an untyped FfiValue tree — give each its own type, and stop eql-bindings forcing dynamic #1252);
   • plaintext hygiene rules;
   • key retrieval and authentication;
   • errors;
   • versioning.
2. Use one integer as the worked example. A Go int32 stored as IntegerEq, traced through encrypt, query and decrypt with the real bytes at each step. That trace exists already, as a page made while reviewing feat(stack-encrypt,golang): give every Go integer width, dates, timestamps, decimals and 128-bit integers their own kinds #1246.
3. Put the wire format in vitaminc-aead-frame's documentation: the frame and ciphertext-tree byte layout, the tags, the limits, and how to run the conformance corpus from the crate (aead-frame: ship the conformance corpus in the crate and fill its gaps; remove the Go bindings once Stack has its own vitaminc#473). The stack-encrypt guide links to it instead of restating it.
4. Give a checklist for a new binding: the conformance corpus passes; the differential fuzzers run against the engine; the plaintext hygiene tests (as in tests/heap_hygiene.rs) pass; the version pins of host and engine are tied together.
5. Link it from the Stack Encrypt README, docs/sdk-design-principles.md and the Go SDK's README.

Relationship to other work

• stack-encrypt: dynamic control messages (plan, options, query results) are parsed from an untyped FfiValue tree — give each its own type, and stop eql-bindings forcing dynamic #1252: typed control messages. The guide should describe them as they will be, or be updated when that lands.
• Go SDK: bring the frame codec and value types in from vitaminc, so the Go host and Rust guest codecs are pinned together #1253: the Go codec moves into this repo. The Go SDK becomes the reference integration.
• aead-frame: ship the conformance corpus in the crate and fill its gaps; remove the Go bindings once Stack has its own vitaminc#473: the conformance corpus ships in vitaminc-aead-frame.