Skip to content

Write 12 edge case regression tests and fix the bugs. - #2

Merged
koculu merged 2 commits into
mainfrom
storage-format-edge-cases
Oct 15, 2026
Merged

koculu merged 2 commits into
mainfrom
storage-format-edge-cases

Conversation

@koculu

@koculu koculu commented Oct 15, 2026

Copy link
Copy Markdown
Member

All 12 regression tests failed or showed the problem on the old code, and the full suite now passes (844 of 844, 4 m 42 s).

What each fix does

Finding Test that showed it on the old code Fix
Rollover can hide data (a new case I found while designing the fix): records with timestamps later than the housekeeping clock ended up past the new boundary After a rollover, a day's raw query returned 0 instead of 1 Each partition now tracks the latest time written to it, saved in a high-water file at the store root. A new boundary always comes after that day.
Rollover stalls after a clock that ran ahead created an empty future partition RollOverPartitions returned 0 The partition actually receiving events can now be split, provided its latest written time is known (PartitionedStore.cs).
Salts deleted during a backfill One anonymous visitor counted 3 times (raw and rollups) A salt for a past day now records its last use (salt-used: keys) and is destroyed one day after that use. Salts for today and yesterday keep the old date rule (VisitorHasher.cs).
Invalid UTF-16 Rollups counted 2 distinct values where raw counted 1 Text.Name and Text.Value replace unpaired surrogates with U+FFFD before hashing and storing (Text.cs).
Raw codec accepts control characters Encode didn't throw RawEventCodec.Encode rejects them; the strict decoder throws FormatException.
SessionState with no dimensions IndexOutOfRangeException when written, and the session was lost Missing positions are written as null, meaning "not collected".
Backfill journal size The journal was 177 MB for 123 MB of rows It is now a binary file: two alternating generations with a SHA-256 checksum, streamed to disk (journal-a.bin / journal-b.bin).

Existing tests changes

  • PublicStorageCodecTests: only the escaping change above. Every assertion is the same.
  • EngineTests.cs:663: Updated a comment that described the old salt lifetime. The assertions are unchanged.

@koculu
koculu merged commit 2fbce0a into main Oct 15, 2026
4 checks passed
@koculu
koculu deleted the storage-format-edge-cases branch October 15, 2026 22:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant