Live durability: do-block, frame rotation, writer wrapper

[tbeason]

Replaces #14. Lifts the live-capture durability features from stream_to_file-only to the subscription layer, so any user iterating records benefits — not just file-capture callers.

What this adds

1. Live do-block constructor

Mirrors Base.open(f, path). Guarantees close(client) on Ctrl-C, exceptions, or normal exit:

Live(dataset = "GLBX.MDP3") do client
    connect!(client)
    subscribe!(client; schema = Schema.TRADES, symbols = ["ES.FUT"], stype_in = SType.PARENT)
    start!(client)
    for rec in client
        # …
    end
end   # Ctrl-C here triggers a clean close(client)

Manual connect! → subscribe! → start! → close lifecycle keeps working unchanged.

2. open_dbn_writer + write_record! (public API)

Thin do-block wrapper over the internal RotatingDBNFile. Same kwargs as stream_to_file, plus a stable write_record!(writer, rec):

Live(dataset = "GLBX.MDP3") do client
    connect!(client); subscribe!(client; …); start!(client)
    open_dbn_writer(; base_dir = "./cap", dataset = "GLBX.MDP3",
                      schema = Schema.TRADES, symbols = ["ES.FUT"],
                      stype_in = SType.PARENT,
                      frame_seconds = 60.0) do writer
        for rec in client
            write_record!(writer, rec)
        end
    end
end

3. In-file zstd frame rotation (default frame_seconds = 60.0)

_rotate_frame_if_needed! closes the active zstd frame via TranscodingStreams.TOKEN_END + flush and starts a new one on the same file — the resulting .dbn.zst is a standards-compliant multi-frame file (any zstd reader concatenates transparently). A hard kill (SIGKILL, OOM, power loss) loses ≤ one frame instead of corrupting the whole file. Hooked into _session_monitor so quiet schemas reach disk.

Also

• Base.close(Live) hardened: flag-flip up-front (re-entry is a no-op); channel cleanup pass runs regardless of c.typed.
• SymbolMappingMsg now lands in the .dbn.zst — DatabentoBinaryEncoding ≥ 0.1.1 fixes the v1→v3 layout re-encode (hd.length is re-derived from metadata.version), so the previous workaround dropping the record on the floor is no longer needed.
• frame_seconds kwarg added to stream_to_file / stream_multi_to_files.
• README: canonical Live example switched to do-block; new "Writing records to disk yourself" section.
• Version 0.1.1 → 0.1.2.

Test plan

• All 1587 offline tests pass (1558 previous + 29 new).
• Live do-block cleans up on normal exit, exception, and InterruptException.
• Base.close(Live) re-entry is a no-op.
• open_dbn_writer closes on InterruptException mid-write.
• write_record! smoke + round-trip.
• Multi-frame .dbn.zst round-trips as one continuous record stream.
• SymbolMappingMsg now appears in the .dbn.zst and decodes via DBN.read_dbn (load-bearing for the upstream fix integration).
• CI green on all matrix entries.
• Manual Ctrl-C smoke against the real Live gateway with stream_to_file(... duration_s = 600) — verify resulting .dbn.zst opens cleanly via DBN.read_dbn_with_metadata.

Supersedes #14.

🤖 Generated with Claude Code

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 430dc8d231

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Require fixed DBN encoder before persisting SymbolMappingMsg

_handle_control! now writes SymbolMappingMsg to disk, but the package compat still allows DatabentoBinaryEncoding = "0.1" (including 0.1.0). The comment here explicitly depends on a fix in DatabentoBinaryEncoding ≥ 0.1.1; with 0.1.0 this path can emit inconsistent record lengths and corrupt the capture stream when a mapping record arrives. Please gate this write path on a safe encoder version (or tighten compat) so users on older 0.1.x don’t get malformed .dbn.zst output.

Useful? React with 👍 / 👎.

[tbeason]

Fixed in 45f63d0 — bumped [compat] DatabentoBinaryEncoding = "0.1.1" so the resolver can't pick up the pre-fix 0.1.0 with this write path enabled. Tightening compat over a runtime version gate keeps the failure mode at Pkg.resolve time (clear) rather than mid-stream record corruption (silent).