feat: segmented fetch with backoff retry in HttpRangeReader

[AdaWorldAPI]

Problem

curl retry restarts the entire 256 MB chunk from scratch. On flaky connections or HuggingFace rate limits during 801 GB Maverick streaming, a single failure at byte 200M of a 256 MB chunk wastes 200 MB of already-downloaded data.

Solution

Split chunks into 64 MB segments (4 per chunk). Each segment fetched independently with:

• Exponential backoff: 1s → 2s → 4s → 8s (4 attempts)
• curl-level retry (2 attempts for connection drops)
• --connect-timeout 30 + --max-time 300 per segment

Segment cache (LRU, 3 entries = 192 MB):

• Backward seeks within cache are free (no re-fetch)
• Sequential reads cycle through cache naturally

API

No change. HttpRangeReader::new() and ::with_chunk_size() work exactly as before. Internal segmentation is transparent to callers.

Impact on Maverick run

Before: 256 MB failure → re-download 256 MB
After:  64 MB failure → re-download 64 MB (4× less waste)
        + backoff prevents hammering HF during rate limits
        + cache covers 192 MB of backward seek

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 360c5c0c38

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Preserve caller chunk size in custom reader constructor

with_chunk_size no longer honors the caller-provided chunk size as the max fetch unit: it derives segment_size as chunk_size / 4 with a hard minimum of 16 MB, and read() always fetches by segment_size. For inputs that intentionally pass small chunk sizes (e.g. 1–8 MB to cap memory/request size), this now performs much larger network reads than requested and changes runtime memory/network behavior compared to the prior API contract.

Useful? React with 👍 / 👎.