fix(fiber): split DA Submit at Fibre's 128 MiB upload cap + duration log

[walldiss]

Issue

Under sustained txsim load the DA submitter batched up to 10 pending data items into a single Upload() call, producing a flat payload of ~144 MiB. Fibre's per-upload server-side cap is hard at ~128 MiB (blob size exceeds maximum allowed size: data size 144366912 exceeds maximum 134217723) and rejected every batched upload. With MaxPendingHeadersAndData=10 that took down 170 consecutive submissions before the daemon halted itself with Data exceeds DA blob size limit.

The Submit path also had no per-call observability — failures showed up as DeadlineExceeded or oversized blob after the fact, with no measurement of how long uploads actually took. During load-test debugging this turned into a guessing game over whether RPCTimeout, pending cap, or batch sizing was the right knob to turn next.

Solution

• fiberDAClient.Submit: wrap the fiber.Upload call in a chunker (chunkBlobsForFibre) that groups input blobs into ≤120 MiB chunks (8 MiB headroom under Fibre's 128 MiB cap for flattenBlobs's per-blob length-prefix overhead) and uploads each chunk separately. Aggregates submitted counts and BlobIDs across chunks; on first chunk failure, returns the error with the partially-submitted count so the submitter's retry/backoff sees a coherent state.
• Per-Submit upload-duration log (info on success, warn on failure): duration, flat blob bytes, blob count, chunk index. Cheap (one time.Since) and gives the operator concrete numbers — e.g. 17 blobs / 115 MiB / 1.5 s — to reason about whether the upload pipeline or something downstream is the bottleneck.
• evnode-fibre: block.SetMaxBlobSize(120 → 100 MiB). Companion safety: after the chunker splits a multi-blob batch, a single oversized blob would still end up alone in its own chunk and fail server-side. Capping per-block data at 100 MiB ensures even a single block_data item fits in one Fibre upload.

Test plan

• No more data size N exceeds maximum 134217723 rejections under sustained load
• No more single item exceeds DA blob size limit halts
• Per-submit upload duration line in evnode logs

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 1a377a03-cfee-4c25-90b7-345b911eb2bc

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}