engine/fetch/chunk: reject peer block indices that overflow signed range geometry (fixes #52) - #61
Conversation
…nge geometry The peer wire carries the block index as an unrestricted uint64, but relay geometry is int64 arithmetic: engine.PeerSource/PeerStreamSource converted with int64(req.Key.Block), and fetch.FetchBlock derived the byte window with signed multiplication. An index above MaxInt64 wrapped negative on conversion, and a smaller index whose start index*BlockSize exceeded MaxInt64 wrapped in the multiplication. A wrapped start either recycled to 0 — a 2^63 index was silently served block 0's bytes and cached under a garbage key — or went negative, where HTTPFetcher.Fetch omits the Range header and a one-block fetch degraded into a whole-object GET (the amplification shape of #15, re-entering via the peer path). Fix at two layers: - chunk.Config.MaxBlockIndex pins the largest index whose whole window [i*BlockSize, i*BlockSize+BlockSize-1] fits int64 (golden-tested against arbitrary-precision values). - The engine's relay sources decline req.Key.Block > MaxBlockIndex before cache lookup, relay selection, size probe, or origin I/O. Declined, not errored: the malformed index is the requester's fault, and a 500 would charge this healthy relay on the requester's circuit breaker. - fetch.FetchBlock rejects negative indices and overflowing windows before invoking the fetcher, so range arithmetic cannot wrap even if a future caller skips the boundary check. Regression tests pin 2^63, 2^64-1, and the first window-overflowing index on both the buffered and streaming relay paths (asserting zero origin contact), the FetchBlock guard with a capture fetcher, and the exact golden window for the largest representable index. Verified to fail on the pre-fix code. Fixes #52
|
@copilot Please review this fix for issue #52 (uint64 peer block indices overflowing signed range geometry). Focus points:
Do NOT push commits to this branch; leave review comments only. |
I also reran the targeted tests locally ( |
Fix: peer/engine uint64 block indices overflow signed range geometry (#52)
What
The peer wire (
GET /peer/v1/block/<chunkHex>/<blockIndex>) carries the blockindex as an unrestricted
uint64, but relay geometry is signedint64arithmetic.
engine.PeerSource/PeerStreamSourceconverted the wire valuewith
int64(req.Key.Block)(engine.go, stream.go ×2) andfetch.FetchBlockderived the byte window with
start := blockIndex * blockSize. Indices thatcannot be represented in signed range geometry wrapped instead of failing.
Triage (verification of the issue's claims against 84aac4e)
Authentic — every cited mechanism reproduced on the exact named commit
(84aac4e, then current main; the cited lines match):
internal/peer/peer.go:131-146—parseBlockPathaccepts anyuint64viastrconv.ParseUint(..., 64)with no geometry check. Confirmed.int64(req.Key.Block)conversion sites (engine.go:453,stream.go:108, stream.go:145). Confirmed.
HTTPFetcher.Fetchomits theRangeheader whenstart < 0. Confirmed.Executable repro (httptest origin counting requests, since deleted):
int64start(BlockSize 4096, size 4500)2^63MinInt64bytes=0-4095— wrong block served, no error2^64-1(MaxUint64)-1-4096MaxInt642^52The issue is understated: besides the full-object amplification it
describes, the wrapped-to-0 shape silently serves and caches block 0's bytes
under a garbage chunk key. On the engine path the old code also computed a
negative expected block length (
blockLenreturning-9223372036854775708). Classification: implementation defect (protocolboundary / integer overflow / transfer amplification) — within the trust model
peers are trusted, but malformed wire geometry must fail cheaply, per the #15
precedent the issue cites.
Fixes
Two layers, so range arithmetic provably cannot wrap:
chunk.Config.MaxBlockIndex()(new): the largest block index whosewhole window
[i*BlockSize, i*BlockSize+BlockSize-1]fits inint64,i.e.
(MaxInt64-BlockSize+1)/BlockSize. Golden-tested againstarbitrary-precision values.
PeerSourceandPeerStreamSourcedeclinereq.Key.Block > uint64(cfg.MaxBlockIndex())before cache lookup,relay selection, size probe, or origin I/O. Declined (
held=false), noterrored: the malformed index is the requester's fault, and a 500 would
charge this healthy relay on the requester's circuit breaker. A legitimate
same-config peer can never send such an index (a real block of a real
object always satisfies the bound).
fetch.FetchBlockarithmetic guard: rejectsblockSize <= 0,blockIndex < 0, andblockIndex > (MaxInt64-blockSize+1)/blockSizebefore invoking the fetcher — defense in depth for any future caller that
skips the boundary check. (fetch deliberately does not import chunk, so it
states the same bound locally, cross-referenced in both comments.)
Docs updated per template:
docs/chunk.md§3.2,docs/fetch.md§3.4,docs/engine.md§3.5/§3.6/§4,docs/peer.md§2, plus test-list rows.Regression tests
All verified to fail on the pre-fix code (sources temporarily reverted,
tests kept):
internal/engine/issue52_test.goTestPeerSourceRejectsMalformedBlockIndex— 2^63, 2^64-1, and the firstwindow-overflowing index (576460752303423488 @ BlockSize 16): cheap
decline, zero origin requests (not even the size probe), no cache
pollution; legitimate control still relays. Old code: served 16 wrong
bytes with
held=truefor 2^63; pulled the whole 100-byte object for2^64-1.
TestPeerStreamSourceRejectsMalformedBlockIndex— same on thecut-through path: nothing streamed, sizer not called, zero origin
requests; legitimate control still streams.
internal/fetch/issue52_test.goTestFetchBlockRejectsOverflowGeometry— 7 malformed geometries × sizeknown/unknown: error before the (capture) fetcher is invoked.
TestFetchBlockMaxInt64Window— the largest representable index reachesthe fetcher with the exact golden window
[9223372036854771712, 9223372036854775807]; ordinary tail clampingunaffected.
internal/chunk/issue52_test.goTestMaxBlockIndexGolden— the bound pinned for BlockSize1/3/16/4096/4MiB against independently computed values; the bound's window
fits int64 and the next index does not.
Verification
Fixes #52