format: archives can compress what they hold

[donislawdev]

One axis for both containers: compression with none (default), fast, default and best, meaning levels 0, 1, 6 and 9. The words are the intent rather than the mechanism — ZIP sets a method per entry, TAR.GZ compresses the whole stream, and those are the same choice said two ways.

The archive still comes out exactly the size that was ordered. What changes is how much of it is content and how much is padding.

Why the plan cannot do this

A compressed length is only knowable by compressing. Measured: our content compresses at ~50 MB/s, and TestPlanningAContainerDoesNotGenerateWhatItHolds plans 3 GB of declared contents in milliseconds against a 20 s ceiling — compressing to learn the length would take about a minute. The project had already measured this from the other side on TIFF, where deflate moves the length with the seed while uncompressed is flat.

So the plan keeps its stored arithmetic and the writer settles the difference.

ZIP: one pass

Every structural field of a zip is fixed width, so a compressed archive differs from a stored one by the entry data and nothing else. The plan works the padding out for the stored archive; the writer gives back what the compressor freed, measured with a registered compressor rather than by counting headers.

Two things had to be measured rather than reasoned about:

• an entry's compressed bytes do not reach the writer until the entry closes, and Flush does not close one. The first version read zero and overshot by exactly the compressed length of the entries.
• the filler must stay stored. It is random, so deflate grows it — measured 65195 → 65220 B — and a padding entry whose length cannot be aimed is useless.

TAR.GZ: two passes

gzip has no per-entry method. The filler carries the bulk (random, so it survives compression at ~+0.031%) and the gzip extra field closes the remainder exactly, since it sits in the uncompressed header and costs n+2. Measured across levels 1, 6, 9 and targets 64 KB → 10 MB: every one landed exactly, with 567–3759 B left for the field against the 65531 it holds (O163).

Cost, paid at write and never at planning: a 10 MB archive is ~25 ms at fast and ~140 ms at default, against 8 ms stored, and .tar.gz pays it twice.

Two refusals, each naming both settings

• compression + size-from-contents — the length would be knowable only by compressing.
• compression + password — a locked entry goes through CreateRaw, which writes the length before the data, so a compressed one would have to be held in memory whole.

No breaking change

The default is none, so no existing archive changes by a byte and no version needs bumping.

Verification

Five new guards, all proven by mutation (10 caught including repointed ones), plus one that exists specifically to catch a compression axis that quietly stores everything and only grows the padding — that would pass the size guard perfectly.

Independent readers rather than Go alone (O163's lesson): 7-Zip and GNU tar accept all four levels in both containers, and Python sees deflate on the entries and store on the filler with content unchanged.

preflight --quick green on all 11 checks. zip.go was split by subject to stay under the crowding cap, and the solve was written as a bounded recursion so the depth counter stayed at 52 rather than raising a ratchet.

🤖 Generated with Claude Code