Skip to content

Fix #23: tolerate unset (0xFF) stype in v3 SymbolMappingMsg decode#24

Merged
tbeason merged 1 commit into
mainfrom
worktree-fix-issue-23-stype-unset
Jun 1, 2026
Merged

Fix #23: tolerate unset (0xFF) stype in v3 SymbolMappingMsg decode#24
tbeason merged 1 commit into
mainfrom
worktree-fix-issue-23-stype-unset

Conversation

@tbeason

@tbeason tbeason commented Jun 1, 2026

Copy link
Copy Markdown
Owner

Summary

Fixes #23 — the generic read_record path crashes on DBN v3 captures with
ArgumentError: invalid value for Enum SType: 255.

_read_symbol_mapping_v2 (src/decode.jl) did an unguarded SType.T(byte) on
the stype_in/stype_out fields of a v2+ SymbolMappingMsg. A DBN v3 live
gateway can emit an unset stype as 0xFF (the documented sentinel), so
decoding real v3 live captures threw on the first control record — typically
after exactly one data record — making v3 captures effectively undecodable
through the generic reader. (v1 files decode fine; this is v3-specific.)

Fix

  • Add an explicit SType.UNDEF = 255 member so the enum constructor is total
    on the 0xFF sentinel instead of throwing.
  • This keeps the non-nullable SymbolMappingMsg stype fields representable and
    round-trips losslessly through encode (UInt8(SType.UNDEF) == 0xFF), so no
    struct/API change is needed.
  • Also fixes the same latent crash on metadata stype_out (decode.jl:239).
    Metadata's nullable stype_in still decodes 0xFF to nothing, unchanged.

Why this slipped through (test-gap analysis)

The gap is structural, not an oversight. Every existing SymbolMappingMsg
test is a construct → encode → decode → compare round-trip built through the
typed Julia constructor with valid enum values. Such a round-trip can never
produce a 0xFF stype byte if the type system can't represent one — the encoder
only writes bytes for valid SType members, so the decoder never sees the
sentinel. The only way to hit SType.T(0xFF) is to decode bytes the encoder
didn't author (a real gateway capture).

Lesson: round-trip tests verify the encoder/decoder agree with each other;
they can't verify the decoder against wire values the encoder is incapable of
emitting. Those need raw-wire decode fixtures.

Tests

  • New test/test_issue23_unset_stype.jl:
    • asserts SType.T(0xFF) == SType.UNDEF and UInt8(SType.UNDEF) == 0xFF
      (unit-level guard; this threw pre-fix);
    • decodes a v3 (MIX schema) file with a SymbolMappingMsg whose stypes are
      unset, interleaved with a TradeMsg, via the generic read_record loop —
      the exact path from the issue's reproduction.
  • Full suite green locally: 3558/3558 passed.

Relationship to PR #22

Independent. PR #22 reworks the v3 InstrumentDefMsg layout; this touches the
SymbolMappingMsg decode and the SType enum, in regions far from #22's edits,
so the two merge cleanly in either order.

🤖 Generated with Claude Code

The generic read_record path did an unguarded `SType.T(byte)` on the
stype fields of a v2+ SymbolMappingMsg. A DBN v3 live gateway can emit an
unset stype as 0xFF (the documented sentinel), so decoding real v3 live
captures threw `ArgumentError: invalid value for Enum SType: 255` on the
first control record, making them undecodable through read_record.

Add an explicit `SType.UNDEF = 255` member so the enum constructor is
total on the 0xFF sentinel. This keeps the non-nullable SymbolMappingMsg
stype fields representable and round-trips losslessly through encode
(UInt8(SType.UNDEF) == 0xFF). Metadata's nullable stype_in still decodes
0xFF to `nothing`, unchanged.

Add a v3 decode/round-trip regression test (test_issue23_unset_stype.jl)
covering a SymbolMappingMsg with unset stype interleaved with a data
record. Existing SymbolMappingMsg tests only ever constructed records
with valid enum values, so the construct->encode->decode round-trip
suite was structurally blind to this wire sentinel.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@tbeason
tbeason merged commit ca9f5b3 into main Jun 1, 2026
8 checks passed
@tbeason
tbeason deleted the worktree-fix-issue-23-stype-unset branch June 1, 2026 16:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Generic read_record crashes on DBN v3 captures: invalid value for Enum SType: 255

1 participant