From eb2c8bd856391c789d45659f57e57bcffd8e3736 Mon Sep 17 00:00:00 2001 From: bdchatham Date: Thu, 3 Sep 2026 15:04:16 -0700 Subject: [PATCH] docs(config): specify the experimental configuration namespace A binary has one place to put a setting today, which is the declared key space, and every key there is a commitment: the schema counter versions that space and changing a key in it is a migration. Paying that per knob is not viable, so a knob either goes unwritten or the commitment happens by accident. This specifies a second namespace under a reserved experimental prefix, registered through one SDK verb. Values in it resolve and reach their readers like any other, and they sit outside the schema contract. Twenty-four criteria across five requirements, each in an EARS template with an ID a test names. The plan marks which of them the minimum version delivers and which belong to the migration chain, and the tasks cut it into six slices that ship on their own. An ADR records the decision the registry forced. A per-key marking was the other shape and it fails on the reader's side, because a reader asks for a key by name and an operator writes that name into a file, so neither sees a flag held in the registry. The registry also refuses a section name carrying a dot, which is why a package cannot register experimental.evm through the section verbs and why the registry owns the prefix and nests under it. Two risks are named rather than deferred. Regenerating sei.toml derives from the legacy files, which carry no experimental key, so a controller regenerating on every start discards every experimental value unless the carry-forward lands with the namespace. And registration is available before any migration chain exists, which is acceptable only while the namespace promises nothing. First specs/ and docs/adr/ in this repository. Co-Authored-By: Claude Opus 5 (1M context) --- ...perimental-configuration-is-a-namespace.md | 62 +++++ specs/experimental-config/plan.md | 127 +++++++++ specs/experimental-config/spec.md | 252 ++++++++++++++++++ specs/experimental-config/tasks.md | 94 +++++++ 4 files changed, 535 insertions(+) create mode 100644 docs/adr/0001-experimental-configuration-is-a-namespace.md create mode 100644 specs/experimental-config/plan.md create mode 100644 specs/experimental-config/spec.md create mode 100644 specs/experimental-config/tasks.md diff --git a/docs/adr/0001-experimental-configuration-is-a-namespace.md b/docs/adr/0001-experimental-configuration-is-a-namespace.md new file mode 100644 index 0000000000..f93ef8a087 --- /dev/null +++ b/docs/adr/0001-experimental-configuration-is-a-namespace.md @@ -0,0 +1,62 @@ +# 1. Experimental Configuration Is a Namespace, Not a Marking + +Date: 2026-09-03 + +## Status + +Accepted + +## Context + +A binary needs somewhere to offer a setting it will not stand behind. Every declared +key today is a commitment: `schema_version` versions the declared space, and changing +a key in it is a migration the binary performs. Work at the bleeding edge cannot pay +that per knob, so a knob either goes unwritten or the commitment is made by accident. + +Nodes a controller manages are the ones that need it. They take their configuration +from the declaration, so a knob the declaration does not carry cannot be handed to +them at all. + +Two shapes were considered. + +A **per-key marking** keeps one namespace and records instability in the registry. A +package registers `evm.new_knob` and flags it experimental. + +A **separate namespace** puts the setting at `experimental..`, registered +through a verb of its own. + +## Decision + +Experimental configuration is a separate namespace under a reserved `experimental` +prefix, registered through one SDK verb. Keys in it resolve and reach their readers +like any other key, and they sit outside the schema contract: a key MAY be renamed, +removed, or change meaning between releases, with no migration and no deprecation. + +The registry owns the prefix and nests under it on the caller's behalf. + +## Consequences + +A marking is invisible where it matters. A reader asks for a key by name, and an +operator writes that name into a file. Neither sees a flag held in the registry, so +somebody writes `evm.new_knob`, finds it gone a release later, and nothing in the file +they were looking at said it could be. A prefix is visible in the file itself. + +The registry refuses a section name carrying a dot, because a dotted name declares +keys inside another section's subtree, where two sections' defaults land in one map +and whichever renders last silently wins. A package therefore cannot register +`experimental.evm` through the section verbs. The dedicated verb is what makes nesting +safe, because the registry controls the prefix and holds one owner per name. + +Two namespaces mean two rules for the migration chain rather than one, and the chain +has to tell them apart. It reads a stability recorded on each section rather than +matching on a key's name, so a renamed prefix does not silently move a key into the +contract. + +The declared key space grows by keys that are not commitments. Anything counting +declared keys as the contract now overstates it, and the count of committed keys is +the one that means what it used to. + +A key under the prefix that no binary declares is a note rather than a refusal, +because a controller writing a file for a newer binary and rolling it to nodes that +have not upgraded is the ordinary case. The cost is that a misspelled experimental key +is reported and not refused. diff --git a/specs/experimental-config/plan.md b/specs/experimental-config/plan.md new file mode 100644 index 0000000000..bb3f46094b --- /dev/null +++ b/specs/experimental-config/plan.md @@ -0,0 +1,127 @@ +# Experimental Configuration Plan + +## Minimum Viable Version + +The smallest thing that delivers the point of the namespace is a package registering +a knob, a `sei.toml` writing it, a node running it, and an older binary meeting it +without complaint. That is EXP-1 through EXP-9, EXP-14 through EXP-18. + +Two requirements are deliberately outside it. EXP-10 and EXP-11 are the migration +chain's half of the contract, and EXP-12 and EXP-13 are rules a graduation follows. +Neither can be exercised before a chain exists, so the minimum version states the +stability on each section and stops there. That much is what the chain will read. + +## Interfaces + +The SDK verb, beside the four registration verbs the registry already offers: + +```go +// RegisterExperimental records a group of settings this binary offers without +// committing to them. +// +// owner is one segment and names the group, so its keys are experimental.. +// where each tag comes from the prototype. One package per owner. +func RegisterExperimental(owner string, prototype any, defaults func(Mode) any) +``` + +The stability a section carries, read by the migration chain rather than matched on a +name: + +```go +// Stability says whether a section's keys are inside the schema contract. +type Stability int + +const ( + // Committed keys are versioned by schema_version, and changing one is a migration. + Committed Stability = iota + // Experimental keys sit outside that contract and may change between releases. + Experimental +) + +// Stability reports whether this section's keys are inside the schema contract. +func (s Section) Stability() Stability +``` + +What the resolution reports about a key nothing declares, split so a caller can tell +the two apart: + +```go +type Resolved struct { + // ... + // UnknownInFile are keys the file carried that no section declares, sorted. + UnknownInFile []string + // UnknownExperimental are keys the file carried under the experimental prefix that + // no section declares, sorted. + // + // Separate from UnknownInFile because the two are different situations. A key + // outside the prefix that nothing declares is a mistake. A key under it is + // routinely a file written for a newer binary, which is the case a controller + // rolling a change forward produces on every node it has not reached yet. + UnknownExperimental []string +} +``` + +No new verb for reading. An experimental key reaches its reader through the delivery +its group declares, so a reader calls what it already calls. + +## Sequence + +**1. The namespace and the verb.** EXP-1 to EXP-6. The registry reserves the prefix, +refuses a section claiming it, refuses a duplicate owner, and derives keys from the +prototype. Nothing consumes it yet, so this lands with the registry's own tests. + +**2. Resolution and delivery.** EXP-7 to EXP-9. A registered group resolves like a +section and reaches its reader the same way. The separateness in EXP-9 is a property +to measure rather than build: the prefix cannot collide with a committed key because +EXP-1 reserves it. + +**3. The unknown key.** EXP-14 to EXP-16. `Resolved` splits the two kinds of unknown +key, the boot reports the experimental ones, and the check reports without failing. +This is the requirement that lets a controller write ahead of a rollout. + +**4. Carrying it forward.** EXP-17 and EXP-18. `seid config generate` reads the +existing `sei.toml` and re-emits its experimental section. + +**5. Stability on a section.** EXP-11. One accessor, so the chain has something to +read when it arrives. + +Steps 1 and 2 are one slice. Step 3 stands alone. Step 4 stands alone and is the one +with a data-loss failure, so it does not wait. + +## Risks + +**Regeneration drops the value.** `seid config generate` derives its file from +`app.toml` and `config.toml`, and neither carries an experimental key. A controller +regenerating on every start therefore discards every experimental value unless step 4 +lands with the namespace. The failure is silent, and the node reverts to the +registered default rather than refusing. + +**Registration is available before a migration exists.** A package can register a +group, the key can be written into files on real nodes, and no chain exists yet to +graduate it. That is acceptable while the namespace promises nothing, and it is the +reason EXP-13 is stated now rather than when the chain is built. + +**One owner per package is a convention the compiler cannot hold.** EXP-3 refuses a +duplicate owner at registration, which is a defect the registry reports rather than a +build failure, because registration happens during package initialisation. + +## Verification + +Each requirement is covered by a test naming its ID. Three are worth naming here +because they are the ones a passing suite could otherwise fake. + +**EXP-14 and EXP-15** need a `sei.toml` carrying a key under the prefix that no +registered group declares, driven through a real boot and a real check. Measured by +the check's exit status, not by its text. + +**EXP-17** needs a file with an experimental value, a regeneration over it, and the +value still there. The mutation is removing the carry-forward and confirming the value +is gone. + +**EXP-9** needs a committed key and an experimental key whose tags read alike, with +the committed one measured after the experimental one is written. + +## Out of Scope + +The migration chain. Graduation tooling. Any change to what a committed key resolves +to. diff --git a/specs/experimental-config/spec.md b/specs/experimental-config/spec.md new file mode 100644 index 0000000000..0679f4d0c0 --- /dev/null +++ b/specs/experimental-config/spec.md @@ -0,0 +1,252 @@ +# Feature Specification: Experimental Configuration + +**Feature Branch**: `plt-exp-config` + +**Created**: 2026-09-03 + +**Status**: Draft + +**Input**: sei.toml needs an `[experimental]` section. Its values are not a committed +part of the configuration API contract. Controller-managed nodes at the bleeding edge +need somewhere to put knobs that are not promises, and the config SDK has to let a +package register them. + +## Semantic Anchors + +Named once. Not restated below. + +| Anchor | Governs | Does not cover | +|---|---|---| +| EARS | acceptance criteria syntax | whether each template fits the behaviour | +| RFC 2119 | normative keywords | whether the obligation is the right one | +| SemVer | what a version number promises | this schema counter, which counts migrations | +| INVEST | whether a story is a real slice | whether the slice delivers value | + +## Glossary + +- **Committed key**: a declared key inside the schema contract. The schema counter + versions it. Changing it needs a migration. +- **Experimental key**: a declared key outside the schema contract, under the + reserved prefix. +- **Owner**: the name a registering package gives its group. One segment. One package + per name. +- **Graduation**: an experimental key becoming a committed key. +- **The reserved prefix**: `experimental`, the one top-level name a section cannot claim. + +## Boundary Context + +- **Sits within**: the configuration registry and the `sei.toml` reader. +- **Owns**: the reserved prefix, the SDK verb that registers into it, how those values + resolve and reach a reader, and what a binary does with a key under the prefix that + it does not declare. +- **Does not own**: the schema migration chain, which is decided separately. This spec + states only that the chain leaves the prefix alone, and records the stability the + chain reads. + +## User Scenarios & Testing *(mandatory)* + +### User Story 1 - Offer a Knob Without Promising It (Priority: P1) + +A package has a setting it wants to expose and cannot stand behind yet. Today its only +option is the declared key space, where every key is a commitment the schema counter +versions. + +**Why this priority**: without it the choice is to make a commitment by accident or to +ship no knob. Both cost more than the feature. + +**Independent Test**: register a group in a test package, write one of its keys into a +`sei.toml`, start a node, and read the value back through the reader that owns it. + +**Acceptance Scenarios**: + +1. A package registers a group. Its keys appear in the declared set under the reserved + prefix and the owner's name. +2. A `sei.toml` states one of those keys. The node runs the written value. +3. A `sei.toml` omits it. The node runs the registered default. + +### User Story 2 - Roll a File Ahead of the Binary (Priority: P1) + +A controller manages a fleet. It writes one `sei.toml` for a release that adds an +experimental knob, and rolls it to nodes that have not upgraded yet. + +**Why this priority**: this is the case the namespace exists for. A file that fails on +every node the rollout has not reached is unusable for a fleet. + +**Independent Test**: write a `sei.toml` naming a key under the prefix that this binary +does not declare, then start a node and run the check. The node starts. The check +passes and names the key. + +**Acceptance Scenarios**: + +1. A node meets a key under the prefix it does not declare. It reports the key and + applies every other value. +2. The check meets the same file. It reports the key and exits zero. +3. A file names an undeclared key outside the prefix. The check still fails. + +### User Story 3 - Keep the Value Across a Regeneration (Priority: P2) + +An operator regenerates `sei.toml` on a node that holds an experimental value. + +**Why this priority**: below the first two because a node has to be able to hold the +value before losing it matters. It ranks above the rest of the work because the failure +is silent: the value goes, the node reverts to a default, and nothing says so. + +**Independent Test**: place a `sei.toml` with an experimental value, regenerate over +it, and read the section back. + +**Acceptance Scenarios**: + +1. A regeneration over a file holding experimental values writes those values again. +2. A regeneration over a file holding none writes no experimental section. + +### Edge Cases + +- Two packages claim one owner name. The registry reports a defect and declares + neither group. +- A section registration claims the reserved prefix as its name. +- An owner name carries a dot, which would place its keys inside another group. +- A committed key and an experimental key carry matching tags. +- A `sei.toml` states a value under the prefix whose shape the reader cannot use. + +## Requirements *(mandatory)* + +### Requirement 1: The Reserved Prefix and Its Registration + +**Objective:** As a package author, I want one verb that registers a group of settings +this binary offers without committing to them, so that a new knob does not join the +schema contract. + +**Traces to:** User Story 1 + +#### Acceptance Criteria + +1. **EXP-1**: THE registry SHALL reserve `experimental` as a top-level prefix. +2. **EXP-2**: WHEN a section registration claims the reserved prefix as its name, THE + registry SHALL report a defect and SHALL derive no key from it. +3. **EXP-3**: THE registry SHALL offer one verb that registers an experimental group, + taking an owner, a prototype whose tags name the keys, and a function answering the + group's default for a kind of node. +4. **EXP-4**: WHEN a package registers an experimental group, THE registry SHALL derive + its keys from the prototype's tags and SHALL name each one + `experimental..`. +5. **EXP-5**: IF two registrations claim one owner, THEN THE registry SHALL report a + defect and SHALL declare neither group. +6. **EXP-6**: IF an owner name carries a dot, THEN THE registry SHALL report a defect. +7. **EXP-7**: THE registry SHALL make that verb the only way to declare a key under the + reserved prefix. + +### Requirement 2: Resolution and Delivery + +**Objective:** As a node, I want an experimental value to reach its reader the way every +other value does, so that a package reading one calls what it already calls. + +**Traces to:** User Story 1 + +#### Acceptance Criteria + +1. **EXP-8**: THE resolution SHALL answer every registered experimental key. +2. **EXP-9**: WHERE `sei.toml` states an experimental key, THE resolution SHALL answer + with the written value. +3. **EXP-10**: WHERE `sei.toml` omits a registered experimental key, THE resolution + SHALL answer with the registered default for the node's kind. +4. **EXP-11**: THE delivery SHALL hand an experimental value to its reader by the same + route its group declares. +5. **EXP-12**: THE resolution SHALL keep the two namespaces separate. A value under the + reserved prefix SHALL NOT change what a committed key answers. + +### Requirement 3: Meeting an Unknown Experimental Key + +**Objective:** As a fleet operator, I want one file to work on nodes at two releases, so +that a rollout does not have to reach every node before the file is usable. + +**Traces to:** User Story 2 + +#### Acceptance Criteria + +1. **EXP-13**: IF `sei.toml` states a key under the reserved prefix that this binary + does not declare, THEN THE boot SHALL report that key and SHALL apply every other + resolved value. +2. **EXP-14**: IF `sei.toml` states a key under the reserved prefix that this binary + does not declare, THEN THE check command SHALL report that key and SHALL exit zero. +3. **EXP-15**: IF `sei.toml` states a key outside the reserved prefix that this binary + does not declare, THEN THE check command SHALL exit non-zero. +4. **EXP-16**: THE resolution SHALL report the two kinds of undeclared key separately, + so a caller tells a file written for a newer binary from a mistake. + +### Requirement 4: Carrying the Section Through a Regeneration + +**Objective:** As an operator, I want a regeneration to keep the experimental values my +file holds, so that writing the file again does not revert a setting I chose. + +**Traces to:** User Story 3 + +#### Acceptance Criteria + +1. **EXP-17**: WHERE an existing `sei.toml` states experimental keys, THE generate + command SHALL write those keys into the file it produces. +2. **EXP-18**: THE generate command SHALL NOT write an experimental key the existing + file does not state. The legacy configuration files carry no experimental key, so + nothing else derives one. + +### Requirement 5: The Schema Contract Boundary + +**Objective:** As a migration author, I want to tell a committed section from an +experimental one, so that a rename never moves a key into the contract by accident. + +**Traces to:** User Story 1 + +#### Acceptance Criteria + +1. **EXP-19**: THE registry SHALL record, for each section, whether its keys sit inside + the schema contract. +2. **EXP-20**: THE registry SHALL report that stability without a caller matching on a + key's name. +3. **EXP-21**: THE schema counter SHALL NOT govern an experimental key. +4. **EXP-22**: A migration SHALL NOT read, write, rename, or remove a key under the + reserved prefix. +5. **EXP-23**: WHERE a key graduates, THE change SHALL be a migration that renames it + into a committed section and raises the schema counter. +6. **EXP-24**: A migration SHALL NOT move a committed key under the reserved prefix. + +### Key Entities + +- **Experimental group**: one package's set of experimental keys, named by its owner. + Holds a prototype and a default per kind of node, as a section does. +- **Stability**: whether a section's keys sit inside the schema contract. Two values. + +## Success Criteria *(mandatory)* + +- **SC-001**: A package registers a group and a node runs a written value from it, + measured through a real boot rather than through the registry alone. +- **SC-002**: A `sei.toml` naming an undeclared key under the reserved prefix starts a + node and passes the check, while the same file with an undeclared key outside the + prefix fails the check. +- **SC-003**: A regeneration over a file holding experimental values produces a file + holding the same values. Removing the carry-forward makes that measurement fail. +- **SC-004**: Every section reports its stability, and the committed set and the + experimental set together account for every registered section. +- **SC-005**: Every criterion above is named by a test. A criterion no test names is + not done. + +## Assumptions + +Nodes use this namespace rarely. The volume is a handful of keys, not a second key +space of comparable size. + +An experimental group answers per kind of node, for consistency with a section. A group +whose value does not vary answers the same for each kind. + +The reader that owns an experimental key holds its own fallback, as readers of committed +keys do. This spec does not change how a reader treats a value it cannot use. + +## Out of scope + +The schema migration chain. EXP-21 through EXP-24 state the boundary the chain honours, +and EXP-19 and EXP-20 give it something to read. Nothing here builds it. + +Graduation tooling. A graduation is a migration, written when a key graduates. + +Any change to what a committed key resolves to. + +Making the namespace safe to misspell. A key under the prefix that no binary declares +reaches nothing, and EXP-13 and EXP-14 require a report rather than a refusal. diff --git a/specs/experimental-config/tasks.md b/specs/experimental-config/tasks.md new file mode 100644 index 0000000000..17894d5dac --- /dev/null +++ b/specs/experimental-config/tasks.md @@ -0,0 +1,94 @@ +# Experimental Configuration Tasks + +Each task is a slice that ships on its own. The requirement IDs are what its tests +name. + +## T1 — Reserve the Prefix and Refuse a Section Claiming It + +**Covers** EXP-1. + +The registry refuses a section named `experimental`, the way it already refuses a name +carrying a dot. Reported as a defect rather than a panic, because registration runs +during package initialisation. + +**Done when** a registration claiming the name produces a defect naming the reserved +prefix, and no key is derived from it. + +## T2 — Register an Experimental Group + +**Covers** EXP-2, EXP-3, EXP-4, EXP-5, EXP-6. + +`RegisterExperimental(owner, prototype, defaults)`. Keys derive from the prototype's +tags and land at `experimental..`. A second registration claiming the same +owner is a defect. The section verbs cannot reach the prefix, which T1 already +guarantees. + +**Done when** a registered group's keys appear in the declared set under the right +names, a duplicate owner is a defect, and an owner carrying a dot is a defect. + +## T3 — Resolve and Deliver an Experimental Key + +**Covers** EXP-7, EXP-8, EXP-9. + +A registered group resolves like a section: the written value where the file states +one, the registered default where it does not. Delivery is the one the group declares. + +EXP-9 is measured rather than built. A committed key and an experimental key with +matching tags, and the committed one still answers its own value after the +experimental one is written. + +**Done when** a written experimental value reaches a reader through a real boot, an +absent one reads as the registered default, and the committed key beside it is +unmoved. + +## T4 — Report an Unknown Experimental Key Without Failing + +**Covers** EXP-14, EXP-15, EXP-16. + +`Resolved` gains `UnknownExperimental`, split from `UnknownInFile`. The boot reports +those keys and applies everything else. `seid config check` reports them and exits +zero. A key outside the prefix that nothing declares goes on failing. + +This is the requirement that lets a controller write a file for a newer binary and +roll it to nodes that have not upgraded. + +**Done when** a `sei.toml` carrying an undeclared experimental key boots, the check +exits zero and names the key, and the same file with an undeclared key outside the +prefix exits non-zero. + +## T5 — Carry the Section Through a Regeneration + +**Covers** EXP-17, EXP-18. + +`seid config generate` reads the existing `sei.toml` and re-emits its experimental +section. Nothing else can derive one, because the legacy configuration files carry no +experimental key. + +Ordered after T4 rather than T2 only because it needs a key to carry. It is the task +with a silent failure, so it does not slip. + +**Done when** a regeneration over a file holding an experimental value writes that +value again, and removing the carry-forward makes the test fail with the value gone. + +## T6 — Record Stability on a Section + +**Covers** EXP-11. + +One accessor on `Section`, so the migration chain can tell a committed section from an +experimental one without matching on the key's name. + +**Done when** every section registered through the section verbs reports committed, +every group registered through the experimental verb reports experimental, and a test +measures both sets against the registered set rather than against a written list. + +## Not Tasks Yet + +**EXP-10, EXP-12, EXP-13.** The migration chain's half of the contract, and the rules +a graduation follows. Nothing can exercise them until the chain exists. T6 is what +leaves the chain something to read. + +## Order + +T1 and T2 are one change. T3 follows them. T4 and T5 each stand alone and can go in +either order, except that T5 needs a key to carry, so it runs after T2 at the earliest. +T6 is independent of all of them.