Fix #23: tolerate unset (0xFF) stype in v3 SymbolMappingMsg decode#24
Merged
Conversation
The generic read_record path did an unguarded `SType.T(byte)` on the stype fields of a v2+ SymbolMappingMsg. A DBN v3 live gateway can emit an unset stype as 0xFF (the documented sentinel), so decoding real v3 live captures threw `ArgumentError: invalid value for Enum SType: 255` on the first control record, making them undecodable through read_record. Add an explicit `SType.UNDEF = 255` member so the enum constructor is total on the 0xFF sentinel. This keeps the non-nullable SymbolMappingMsg stype fields representable and round-trips losslessly through encode (UInt8(SType.UNDEF) == 0xFF). Metadata's nullable stype_in still decodes 0xFF to `nothing`, unchanged. Add a v3 decode/round-trip regression test (test_issue23_unset_stype.jl) covering a SymbolMappingMsg with unset stype interleaved with a data record. Existing SymbolMappingMsg tests only ever constructed records with valid enum values, so the construct->encode->decode round-trip suite was structurally blind to this wire sentinel. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #23 — the generic
read_recordpath crashes on DBN v3 captures withArgumentError: invalid value for Enum SType: 255._read_symbol_mapping_v2(src/decode.jl) did an unguardedSType.T(byte)onthe
stype_in/stype_outfields of a v2+SymbolMappingMsg. A DBN v3 livegateway can emit an unset stype as
0xFF(the documented sentinel), sodecoding real v3 live captures threw on the first control record — typically
after exactly one data record — making v3 captures effectively undecodable
through the generic reader. (v1 files decode fine; this is v3-specific.)
Fix
SType.UNDEF = 255member so the enum constructor is totalon the
0xFFsentinel instead of throwing.SymbolMappingMsgstype fields representable andround-trips losslessly through encode (
UInt8(SType.UNDEF) == 0xFF), so nostruct/API change is needed.
stype_out(decode.jl:239).Metadata's nullable
stype_instill decodes0xFFtonothing, unchanged.Why this slipped through (test-gap analysis)
The gap is structural, not an oversight. Every existing
SymbolMappingMsgtest is a construct → encode → decode → compare round-trip built through the
typed Julia constructor with valid enum values. Such a round-trip can never
produce a
0xFFstype byte if the type system can't represent one — the encoderonly writes bytes for valid
STypemembers, so the decoder never sees thesentinel. The only way to hit
SType.T(0xFF)is to decode bytes the encoderdidn't author (a real gateway capture).
Lesson: round-trip tests verify the encoder/decoder agree with each other;
they can't verify the decoder against wire values the encoder is incapable of
emitting. Those need raw-wire decode fixtures.
Tests
test/test_issue23_unset_stype.jl:SType.T(0xFF) == SType.UNDEFandUInt8(SType.UNDEF) == 0xFF(unit-level guard; this threw pre-fix);
MIXschema) file with aSymbolMappingMsgwhose stypes areunset, interleaved with a
TradeMsg, via the genericread_recordloop —the exact path from the issue's reproduction.
Relationship to PR #22
Independent. PR #22 reworks the v3
InstrumentDefMsglayout; this touches theSymbolMappingMsgdecode and theSTypeenum, in regions far from #22's edits,so the two merge cleanly in either order.
🤖 Generated with Claude Code