Skip to content

s3:// and gs:// absorb URL userinfo into the bucket name instead of refusing it, so the user is told the object does not exist #995

Description

@OffgridwithJD

http:// refuses userinfo by name. s3:// and gs:// silently absorb it into the bucket name, and the user is told the object does not exist.

http://u:p@127.0.0.1:1/x.parquet   ->  22023  columnar: userinfo in "..." is not supported
s3://u:p@pgc-bucket/vh.parquet     ->  58P01  columnar: "..." does not exist (HTTP 404)

The second message is true and useless: the object does not exist because we turned u:p@pgc-bucket into a bucket name. s3://user:key@bucket/obj is a form several tools accept, so this is a mistake a user will make, and what they get back sends them looking for a missing object.

This is not an SSRF or a credential-injection bug, and I checked specifically. Nothing in our code splits the authority at @, so the userinfo never becomes HTTP basic auth and never redirects the request to another host. Severity is consistency and diagnosis, not exposure.

Measured with an endpoint configured, and with the control that makes it mean something

My first attempt proved nothing: with no credentials every s3 URL returned 28000 requires AWS_ENDPOINT_URL, including the clean control, so a userinfo refusal was indistinguishable from "s3 is unreachable here". Re-run against test/objstore_addressing.sh's fixture with AWS_ENDPOINT_URL, keys and region set, on main at a0b916dc. The fixture was temporarily instrumented to log the Host and Authorization headers; that instrumentation was reverted and the worktree verified clean.

[path-style, CLEAN control]  -> 4 10                     HOSTHDR s3.local:30829   HEAD /pgc-bucket/vh.parquet
[path-style, userinfo]       -> HTTP 404                 HOSTHDR s3.local:30829   HEAD /u%3Ap%40pgc-bucket/vh.parquet
[VIRTUAL, CLEAN control]     -> 4 10                     HOSTHDR pgc-bucket.s3.local:30829
[VIRTUAL, userinfo]          -> could not resolve "u:p@pgc-bucket.s3.local": Name or service not known
[write, userinfo]            -> NO ERROR RAISED          PUT /u%3Ap%40pgc-bucket/ui.parquet
[write, CLEAN control]       -> t                        PUT /pgc-bucket/ok.parquet

Reading those rows:

  • Path style: the userinfo is percent-encoded into the first path segment and the Host header is untouched. So the bucket becomes u:p@pgc-bucket, and the 404 is the fixture correctly reporting that no such bucket exists.
  • Virtual-host style: the bucket — userinfo and all — becomes the leftmost label of the hostname, and the whole string is handed to the resolver as a literal host. It fails at DNS, before any connection. Not userinfo@host: the @ stays inside the name being resolved.
  • The write path is the one worth looking at twice. export_parquet to s3://u:p@pgc-bucket/ui.parquet raised no error and the fixture logged the PUT. Against this fixture that means the bytes went to a bucket the caller did not name. On real S3 it would be NoSuchBucket, so the practical outcome there is a failed export with a misleading reason — but the acceptance is ours either way. (I verified the PUT was logged and no error was raised; I did not go on to check the object on disk.)
  • A userinfo-carrying AWS_ENDPOINT_URL IS refused, at 22023 — but as has an invalid host or port, not as userinfo. Fail closed, imprecise reason.

The fix already exists, unlanded, and it is reachable

I found it parked as an uncommitted 17-day-old change in my clone (#706 follow-on, 2026-08-26), stashed rather than discarded. Two guards in os_resolve_s3:

if (memchr(bucket, '@', slash - bucket) != NULL)
	ereport(ERROR,
			(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
			 errmsg("columnar: userinfo in \"%s\" is not supported", url)));

and the same on a configured endpoint's host. main carries six userinfo refusals and neither of these two is among them.

The reachability question is settled, which is the only reason this is worth reporting. The measurement above shows a userinfo URL reaching the authority parse in both addressing modes, so the guard fires rather than sitting behind an earlier refusal — the an unreachable guard arm trap. Had the clean-URL control failed the way my first probe did, I would have had no way to tell a working guard from an unreachable one.

What a fix should carry

  • Both arms, because path style and virtual style fail differently and only one is covered by any single message.
  • The write path as well as the read path: that is the half where acceptance has a consequence beyond a confusing error, and it is the same asymmetry #706 found between os_open and os_write_handle.
  • The endpoint case retitled from invalid host or port to the userinfo message, so the three schemes say one thing.
  • Arms in test/objstore_userinfo.sh, whose whole subject this is, with a clean-URL control in the same run — without it the arms cannot distinguish a refusal from an unreachable object store, which is exactly how my first probe wasted a run.

I have not opened a PR: the guard is someone's call on the message wording, and #706's author should say whether the endpoint message moves.

Related: #706.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Uf6UoeBRZYLQZa4KxNiw8a

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions