Icarus cloud · Ready to rerun · Decision reopened
+Rerun the Dart client gauntlet with the missing control
+The first gate found a real Dartvex strictness gap, but it did not settle Dartvex versus convex_flutter. Its result-field leg regenerated the same fixture, whose return contract was null, instead of testing an actual result rename. A diagnostic control with an explicit return schema did catch the rename at analysis time. The fair next step is to make the server contract explicit, add a thin fail-closed Icarus wrapper, and then run the same runtime faults against both clients.
Decision now
+ ++ +No client winner yet. Keep the first gate as evidence that Dartvex 0.2.0 is not fail-closed by default. Reopen the runtime decision because neither client has completed the symmetric chaos test.
The goal is still less JSON plumbing and stronger generated Dart APIs. The first run shows that Dartvex can supply useful function and argument types, but Icarus must enforce complete public return validators and reject degraded output. That compensation is small enough to test before writing a generator or forking another package.
+What the first gate proved
+ +| Check | Observed | Meaning | Status |
|---|---|---|---|
| Function rename | Old method failed analysis with exit 3 | Generated function names protect callers | proved |
| Argument rename | Old named argument failed analysis with exit 3 | Generated arguments protect callers | proved |
| Result-field rename | Baseline fixture regenerated; returns stayed null | No result mutation was exercised | invalid leg |
| Missing return schema | Dartvex emitted Future<dynamic> | Unspecified server output cannot become a typed Dart result | real gap |
| Unknown validator | Warning, exit 0, field degraded to dynamic | Default generation is not fail-closed | real gap |
| Determinism | Second baseline generation produced no diff | Same input generated the same output | proved |
| Runtime chaos and profile | Skipped by the original stop rule | No correctness, latency, memory, reconnect, or auth comparison exists | not run |
Evidence: contract_gate.dart, baseline function spec, contract_gate.json, and the first result note.
+ +The exact fairness failure
+In contract_gate.dart, the result-field leg calls generate('folders_list_for_parent.json') again. The fixture contains "returns": null. The real folders:listForParent query also has no explicit returns: validator. The old caller therefore remained valid because there was no typed result contract to rename.
A diagnostic control supplied an explicit object return, generated the typed caller, then changed publicId to folderPublicId. Baseline analysis exited 0; analysis against the renamed result exited 3 with an undefined getter. That does not make Dartvex the winner. It proves the missing control can reverse the narrow conclusion about result-drift detection.
The neutral test boundary
+ +One Icarus contract · two replaceable client adapters · one verifier
+ +Icarus continues to own the outbox, clientId/opId identity, revision and conflict rules, canonical JSON, and .ica round-trip. A client package is transport and tooling, not the owner of those promises. This is also why adopting either package unchanged would be the wrong abstraction boundary.
Before touching server sync boundaries, read server_side_sync_boundaries_handoff.md.
+Fair rerun plan
+ +Phase 1: repair the contract gate
+-
+
- Add explicit
returns:validators to the stable public Convex functions used by the comparison. Match the real payload exactly; do not create a test-only fantasy type.
+ - Regenerate a scrubbed baseline fixture and add a separate result-renamed fixture where
publicIdbecomesfolderPublicId.
+ - Compile the unchanged caller against both. Baseline must exit 0. The renamed fixture must exit nonzero and identify the old getter. +
- Add an Icarus-owned wrapper around Dartvex generation. It fails if generation logs
Warning:, if a stable public module contains unexpecteddynamic, or if the generator exits nonzero.
+ - Keep the unknown-validator mutation. It must fail the wrapper with the function and field path. +
- Run generation twice and require a clean repository diff. +
+ +Boundary: do not fork Dartvex or write a replacement generator in this phase. First prove whether explicit server schemas plus a thin strict wrapper deliver the typed API we want.
Phase 2: make the runtime comparison symmetric
+Define one small typed Icarus transport interface for the operations used by the gauntlet. Implement it once with Dartvex and once with convex_flutter. Both adapters must receive the same serialized operations, tokens, reconnect schedule, timeouts, and deployment. Package-specific convenience APIs cannot change the workload.
Keep two scorecards. The tooling score covers generated coverage, compile-time mutation catches, determinism, warnings, diff size, and maintenance. The runtime score covers correctness, convergence, recovery, latency, CPU, memory, and platform builds. A tooling loss cannot masquerade as a runtime loss, and a fast runtime cannot excuse corrupted library state.
+ +Phase 3: run correctness before performance
+-
+
- Run 50 deterministic seeds × 1,000 operations for each adapter. +
- Use editors A and B plus a clean verifier C. Begin every seed from base-test-v43.ica. +
- Exercise offline edits, delayed and duplicated delivery, reconnect, subscription restart, delete/recreate, revision conflict, and bounded retries. +
- Exercise an expired or rejected access token, call the current Supabase Flutter
refreshSession()path, reconnect, and prove the queued op lands exactly once.
+ - Persist the runner ledger and reuse identical
clientId/opIdvalues for both adapters so a process restart does not give one client an easier test.
+ - After every seed, verifier C exports canonical state. Compare strategies, pages, folders, lineups, order, revisions, and round-trip output. Never compare timestamps or transport-only metadata that the product does not promise. +
Use disposable test accounts and publishable client credentials. Never put a Supabase secret or service_role key in Flutter, fixtures, committed output, or logs. The current Supabase Dart API documents refreshSession() as refreshing and returning a new session even when the current session is not expired; the fault injector should assert the session actually changes or is accepted before replaying the op.
Phase 4: profile only after both are correct
+Run at least 10 paired profile-build trials, alternating which adapter runs first. Report median and p95 remote convergence, reconnect-to-live time, peak RSS, steady-state CPU, transferred bytes, and build size on every supported desktop target. Record raw samples, tool versions, machine state, commit, and deployment identity.
+What settles the gauntlet
+ +| Condition | Dartvex consequence | convex_flutter consequence |
|---|---|---|
| Any dropped, duplicated, misordered, or silently conflicted library change | Immediate loss, regardless of speed | |
| Cannot recover queued work after auth refresh or reconnect | Immediate loss | |
Generated stable API contains unexpected dynamic | Loss unless the thin strict gate rejects it before commit | Not a generated-code criterion |
| Both complete all 50 seeds with canonical equality | Compare profile results, API clarity, adapter size, dependency health, and maintenance cost | |
| Runtime is tied within measurement noise | Wins if the generated boundary materially removes JSON plumbing | Wins if Dartvex still needs broad custom generation or fragile patches |
The expected best outcome is not “Dartvex untouched.” It is Dartvex plus a narrow Icarus strictness policy. If that produces complete, deterministic types and passes the same runtime gauntlet, Dartvex earns the win because it moves contract failures into analysis and removes hand-written JSON decoding. If the compensation grows into a package fork, a second generator, or recurring patches for common Convex validators, convex_flutter remains the more honest base.
Remote machine handoff
+ +Start here
+-
+
- Check out
t3code/convex-client-gauntletand pull the latest commit.
+ - Read this handoff, the first result note, and the server sync boundary handoff. +
- Reproduce the committed first gate before changing fixtures. +
- Implement Phase 1 as a distinct commit. Do not begin runtime work until every repaired contract check is green. +
- Implement the neutral interface and two adapters without changing local Hive models,
.ica, UI, outbox semantics, revision rules, or server payload semantics.
+ - Run correctness, then profile. Commit raw machine-readable results and a short human verdict. Leave the old result file intact as historical evidence. +
Current commands that exist
+git switch t3code/convex-client-gauntlet
+git pull --ff-only
+
+cd tool/convex_client_gauntlet
+fvm dart pub get
+fvm dart run bin/run.dart
+fvm dart test
+fvm dart analyze
+
+cd ../..
+npx tsc --noEmit
+npm run test:convex
+fvm flutter test
+fvm flutter analyze --no-fatal-infos
+fvm flutter build web --no-tree-shake-iconsThese reproduce the committed first gate and repository baseline. Add named contract-v2, runtime, and profile entry points as part of the rerun; document their exact commands beside the resulting artifacts rather than pretending they already exist.
+ +Required artifacts from the rerun
+-
+
- Explicit-return baseline and actual result-renamed fixtures. +
- A strict wrapper test proving warnings and unexpected
dynamicfail with a useful path.
+ - Generated-output snapshots or hashes proving determinism. +
- Per-seed runtime JSON for both adapters, including fault schedule and canonical verifier hash. +
- Paired profile samples with run order, machine, build mode, and package versions. +
- A final matrix that distinguishes compile-time safety, runtime correctness, performance, and maintenance. +
Acceptance checklist
+-
+
- The result rename mutates a return field, not the function name, argument, caller, or baseline fixture. +
- The stable public Convex functions in scope have explicit return validators that match real payloads. +
- Dartvex baseline generation is warning-free and contains no unexpected
dynamic.
+ - The unchanged caller fails analysis for function, argument, and result renames. +
- Unsupported validators fail before generated code can be committed. +
- Both adapters receive byte-for-byte equivalent operation traces and fault schedules. +
- All 50 seeds end with exact canonical equality and no unresolved op. +
- Auth refresh is performed with the client session only; no elevated credential appears anywhere. +
- All exported strategies and library backups still round-trip. +
- No winner is declared from the tooling gate alone. +
Appendix
+Raw fairness evidence · first gate and diagnostic control
+Committed first gate
+ result-field step: generate('folders_list_for_parent.json')
+ baseline fixture: "returns": null
+ runtime seeds: 0
+
+Diagnostic missing control
+ explicit typed baseline analysis: exit 0
+ publicId -> folderPublicId analysis: exit 3
+ failure: undefined getter on the unchanged caller
+
+Correct interpretation
+ Dartvex 0.2.0 is not fail-closed by default.
+ The original test did not compare runtime clients.
+ A complete result schema lets the generated caller catch result drift.Captured 2026-08-26 against branch base df9f1934bbc7ab71e144208036f385b8f73c77aa. The diagnostic fixture was intentionally not committed; the fair implementation must add a reviewed equivalent.
Generated 2026-08-26 · Icarus cloud client evaluation · base df9f193 · Dartvex 0.2.0 · convex_flutter 3.0.1 · Supabase Flutter auth guidance checked 2026-08-26 · revision v1
+