net/http: follow redirects in the client again

[yohimik]

net/http: follow redirects in the client again

Repository tinygo-org/net. Branch upstream-pr/http-redirects, base main.

What this does

The client keeps CheckRedirect and ErrUseLastResponse, but Client.do lost
its redirect loop, so a 3xx went straight back to the caller with an unread
body. A request for a GitHub release asset returned 302 and zero bytes where Go
returns 200 and the file.

This ports the loop of Go back into http/client.go. It has

• up to 10 hops through checkRedirect and defaultCheckRedirect,
• redirectBehavior with the 301, 302 and 303 change to GET and the 307 and 308
  replay with GetBody,
• Location resolved against the URL of the current request,
• the drop of sensitive headers when a hop leaves the initial domain,
• the read and close of an intermediate body,
• ErrUseLastResponse honoured,
• refererForURL with the https to http rule.

The code follows the upstream Go source, with its comments, so a later update
from upstream is a straight comparison.

Two changes from upstream

• shouldCopyHeaderOnRedirect compares the hostnames as they are and not
  through idnaASCII, which needs golang.org/x/net/idna and its tables. The
  Unicode and the punycode spelling of one IDN host thus count as two hosts and
  lose their sensitive headers. That is the safe direction. The rule for a
  subdomain is the upstream one.
• roundTrip uses Request.URL.Host when Request.Host is empty, because only
  NewRequest fills Request.Host and a redirect hop builds its request
  directly.

Evidence

There is no CI in this repository. Checked by hand on macOS 26.6 arm64 with a
TinyGo build that carries the matching toolchain change.

• Before, a GET of a GitHub release asset returned 302 and 0 bytes.
• After, the same GET returns 200 and the file.
• A program that does http.Get("https://example.com/") builds and completes.

A downstream product ships binaries built with these changes in a production
release. dispat v1.4.0 is published and is not a prerelease. It carries
dispat-tiny-linux-amd64 and dispat-tiny-linux-arm64, built by the fork
release v0.42.0-net.4 from sha256-pinned tarballs and smoke-executed under
binfmt before upload, beside six binaries from the gc toolchain.
https://github.com/yohimik/dispat/releases/tag/services%2Fdispat%2Fv1.4.0

The acceptance record of that repository is committed at
packages/docs/docs/internals/tinygo.md. It reports the net.2 to net.4
acceptance history, an integration suite of 694 rows that passes with 0 failures
and 1 documented skip on darwin, and a size table of 0.58x to 0.63x against the
gc equivalents with TinyGo -opt=z -no-debug against go build -trimpath -ldflags "-s -w". Those figures come from that document. They are not a
measurement of this branch.

The self-update path of that program downloads a release asset over HTTPS, which
is the redirect case above.

Scope

• One file, http/client.go. No API change.
• The change is independent of the darwin work in this series and applies to
  linux as it stands.

Known gaps

• http.Client.Timeout is still inert in this port. The loop does not add a
  deadline of its own. That is a separate piece of work.
• There is no Transport implementation, so a caller-supplied Transport is
  still driven directly and does not take part in the redirect handling.

Related pull requests

This change is part of one body of work. Together the changes make programs that use the network and child processes work on hosted linux and macOS. A full CLI was tested end to end with all of them and ships binaries built this way, see dispat v1.4.0 in the evidence section.

In tinygo-org/tinygo

• sync: hand the RWMutex over instead of a re-test of the reader count tinygo#5630 sync, RWMutex hand-over. Bug fix, standalone.
• runtime: deliver os/signal notifications under the threads scheduler tinygo#5631 is CLOSED in favor of runtime: deliver signals under the threads scheduler when blocked on I/O tinygo#5530 for signal delivery with the threads scheduler.
• compiler: implement go:cgo_import_dynamic and lower Darwin cgo import trampolines (#5604) tinygo#5612 owns darwin fcntl. compiler,runtime: call fcntl through a C wrapper on darwin tinygo#5632 is closed in its favor. Integer and pointer tests were offered on #5612.
• runtime: implement weak.runtime_makeStrongFromWeak tinygo#5633 runtime, weak.runtime_makeStrongFromWeak. Needed by real crypto/tls.
• os: start processes with posix_spawn tinygo#5634 is limited to process support with posix_spawn.
• loader: use the real crypto/tls on hosted linux and darwin tinygo#5635 loader, real crypto/tls on hosted linux and darwin.
• builder: declare Darwin socket and process symbols tinygo#5636 adds builder socket and spawn symbols at e8394f67. It is independent. Its direct syscall test needs no net pin.

In this repository

• net/http: follow redirects in the client again #72 net/http, follow redirects in the client again.
• net: report a name that does not exist as a not-found lookup error #73 net, report a name that does not exist as a not-found error.
• net: extend the host netdev to darwin #74 net, extend the host netdev to darwin.
• net/http: give the HTTPS client trust roots on darwin #75 net/http, give the HTTPS client trust roots on darwin.

A merge order that works. The remaining bug fixes are independent. tinygo-org/tinygo#5633 goes before tinygo-org/tinygo#5635. HTTPS on linux needs only tinygo-org/tinygo#5633 and tinygo-org/tinygo#5635. Full darwin support also needs tinygo-org/tinygo#5636, the net changes and a new src/net submodule pin.

Related owner work

• net: non-blocking host netdev I/O with an epoll poller #77 owns Linux I/O cancellation and deadline interruption. Its epoll code needs an OS split when combined with the darwin support in net: extend the host netdev to darwin #74.
• net: report the kernel-assigned port after listening on port 0 #80 owns the kernel-assigned TCP listener port. Do not add a second port fix.
• net/http: add ListenAndServeTLS / ServeTLS (software TLS) #79 owns server TLS methods. This is separate from client redirects and client trust roots.
• http: port Go net/http timeout handling #67 owns Client.Timeout and transport selection. Its native cancellation gap is recorded in the review. net/http: follow redirects in the client again #72 must be combined with that work at http/client.go.
• net: API-compat fills for grpc and wazero (SyscallConn, Dialer.Control, IPConn.Read) #78 owns Dialer.Control API fields and other API additions. It does not implement ListenConfig.

The Crier listener and close audit is separate from these PR measurements. The historical Linux arm64 run passed 142 tests with local patches. It predates later Crier changes and is not release validation.

Current integration status

Darwin fcntl work is in tinygo-org/tinygo#5612. tinygo-org/tinygo#5632 is closed and remains a history reference only. The integer and pointer tests were offered on #5612. Builder socket and spawn symbols in tinygo-org/tinygo#5636 are an independent prerequisite at e8394f67. The direct syscall test needs no net update, so that PR does not wait for this net series. tinygo-org/tinygo#5634 is limited to process support.

#82 supplies the separate ListenConfig implementation. Close guard tests and a limited shutdown fix stay on the fork branch codex/close-audit for coordination with the #77 owner. They are not equivalent to the poller. No second shutdown PR was opened.

Current Crier evidence

The Crier report separates the original size comparison from a new CI-built candidate test. The original stripped Linux ARM64 result is 13,829,248 bytes versus Go's 30,277,794 bytes. Two render fixtures exceed pixel tolerance.

Candidate e7d34c8c126e0eecd2ce711915f2f88d112d833a, with net 0f460803, passed all 24 fork CI checks. Its downloaded compiler artifacts, with no source overlays, build unchanged Crier 7edaff9. Linux ARM64 E2E passed 143 cases with no failures or skips. Darwin ARM64 passed 142 with no failures and one platform trust-store skip. Darwin startup now works. A separate Darwin local-TLS matrix passes certificate rejection, plaintext refusal, update to a Go 1.1.0 target, and offline rollback. Both platforms pass spawn, signal, cookiejar, os/fcntl, and network probes.

These are combined-candidate results, not proof that this PR alone supplies all features. No new pixel or stripped-size comparison, amd64 execution, or current Dispat acceptance is claimed. WaitDelay and the recorded net limits remain open. No fork release was published.

Owner sync, 6 September

The owner of #77 adopted our repeated-Close guard as 9dc11e1, with tests adapted to its nonblocking sockets. The owner of #80 adopted the Zone correction as 7b7aacb and removed the test dependency on #82. These are PR branches, not upstream merges.

New #83 restores RoundTripper dispatch. A Git merge check against #72 reports a conflict in http/client.go; both behaviors need combined tests before integration. New #84 uses the existing integer-conversion shim in tlssock.go. Neither new PR is included in the tested candidate e7d34c8c. No duplicate PR is needed.

[yohimik]

tinygo-org/net main has not moved since this branch was opened. It is still
70037cf, so the branch needs no rebase and it is current. v0.42.0 of the
toolchain pins src/net at that same commit, so the change applies to the
released toolchain.