Skip to content

Metadata cache can store transport-decompressed bytes for signed indexes #300

Description

@andrew

fetchUpstreamMetadata in internal/handler/handler.go uses the default http.Transport behaviour, which sends Accept-Encoding: gzip and transparently decompresses any response carrying Content-Encoding: gzip. The metadata cache stores the decoded body and a content type, but not the content encoding.

For ecosystems that cache signed or hash-pinned index files through this path (debian Release/Packages.gz, rpm repomd.xml, helm index.yaml, conda repodata.json, apk APKINDEX.tar.gz/Packages.adb), a CDN or origin that serves the file with Content-Encoding: gzip results in the proxy caching and re-serving different bytes from what the upstream signed, breaking client verification.

Raised by @pinguinfuss in review on #293 (#293 (comment)); it's a shared-path issue rather than APK-specific, so tracking it here.

Fix shape: on the shared metadata HTTP client, set Transport.DisableCompression = true (or send Accept-Encoding: identity) so Go stops auto-decompressing; persist Content-Encoding alongside Content-Type in the metadata cache row and replay it on serve. Roughly 30 lines in internal/handler/handler.go plus a metadata schema/column addition and a migration.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions