Skip to content

loader: use the real crypto/tls on hosted linux and darwin - #5635

Draft
yohimik wants to merge 2 commits into
tinygo-org:devfrom
yohimik:upstream-pr/hosted-crypto-tls
Draft

loader: use the real crypto/tls on hosted linux and darwin#5635
yohimik wants to merge 2 commits into
tinygo-org:devfrom
yohimik:upstream-pr/hosted-crypto-tls

Conversation

@yohimik

@yohimik yohimik commented Aug 30, 2026

Copy link
Copy Markdown

Review status: the loader implementation is ready for review and does not wait for Dispat or Crier. The weak hook remains a separate dependency in #5633. Head c5f0e2ba has no standalone CI run; earlier branch CI and current published-fork tests cover combined changes, not this isolated head. The owner closed #5645 in favor of #5633 and this PR and agreed to the Linux/Darwin guard. Merge still needs the dependency and relevant CI.

loader: use the real crypto/tls on hosted linux and darwin

Depends on upstream-pr/weak-strong-from-weak.

What this does

TinyGo replaces crypto/tls with a stub whose handshake does nothing, so a
program that dials https:// gets a plaintext connection behind the TLS API.
That stub is correct for a target with no OS below it, which has neither the
code size for a full TLS implementation nor usually a socket to speak it over.
Hosted linux and macOS have both, and there the crypto/tls of the Go standard
library compiles and runs.

The override becomes conditional. Without an entry in the map, crypto/tls
falls under the "crypto/" merge, which links the package of the standard
library into the synthetic GOROOT. GOOS alone cannot decide this, because a
baremetal target reports GOOS=linux, so the build tags decide as well. The
goroot cache key is a hash of the merge links, so the two variants get separate
cache entries.

Evidence

testdata/hostcryptotls.go makes an ECDSA key and a self-signed certificate at
run time, then does a TLS handshake over net.Pipe. It needs no network and no
netdev. TestHostCryptoTLS in main_test.go runs it on the host, and skips
where the host is not linux or darwin, in the same way as
TestTimerStopResetRace.

Measured on macOS 26.6 arm64.

Build Output
current dev, with the stub negotiated an unexpected version: 0
with this change got: pong and unknown certificate refused

The first row shows what the stub does. It reports a completed handshake and a
version of zero, and it does not verify anything. The second row shows a real
TLS 1.3 handshake, data through the connection, and a client with an empty root
pool refusing the certificate.

loader/goroot_test.go covers the decision itself, with the baremetal case that
reports GOOS=linux.

A downstream product ships binaries built with these changes in a production
release. dispat v1.4.0 is published and is not a prerelease. It carries
dispat-tiny-linux-amd64 and dispat-tiny-linux-arm64, built by the fork
release v0.42.0-net.4 from sha256-pinned tarballs and smoke-executed under
binfmt before upload, beside six binaries from the gc toolchain.
https://github.com/yohimik/dispat/releases/tag/services%2Fdispat%2Fv1.4.0

The acceptance record of that repository is committed at
packages/docs/docs/internals/tinygo.md. It reports the net.2 to net.4
acceptance history, an integration suite of 694 rows that passes with 0 failures
and 1 documented skip on darwin, and a size table of 0.58x to 0.63x against the
gc equivalents with TinyGo -opt=z -no-debug against go build -trimpath -ldflags "-s -w". Those figures come from that document. They are not a
measurement of this branch.

The self-update path of that program runs over real TLS against a live host, and
its suite has negative rows as well. An unknown CA is refused, and a plaintext
server on a TLS port is refused.

Scope and known gaps

  • Windows keeps the stub. So do wasm, wasip1, wasip2, wasm_unknown,
    nintendoswitch and every baremetal target. Their behaviour does not change.
  • Only a program that imports crypto/tls sees a size change, and such a
    program did not work before.
  • crypto/x509 on darwin uses the platform verifier, and
    crypto/x509/internal/macos in TinyGo is a stub, so a nil RootCAs fails on
    darwin. net/http in tinygo-org/net handles this by giving the client a root
    pool. A caller that dials tls.Dial directly on darwin has to supply
    RootCAs. See the tinygo-org/net PR "net/http: give the HTTPS client trust
    roots on darwin".
  • http.Client.Timeout is inert in the tinygo-org/net client. It is not
    addressed here.
  • crypto/tls API mismatch: missing tls.X509KeyPair and ClientAuth constants, LoadX509KeyPair stubbed #5204 lists crypto/tls API gaps such as tls.X509KeyPair. This change makes
    the real package available on hosted targets, so those gaps disappear there,
    but the stub keeps them on every other target.

Related

This is a smaller change than the netdev reworks in #4187, #4273 and #4498. It
does not replace the net package or add a netpoller. It only stops the
substitution of the TLS stub where the standard library package works.

Enables

A command line program that needs an HTTPS client path. Self-update, an API
client, a webhook sender. Client side only. There is no server claim here beyond
what the tests show.

Related pull requests

Each open PR in this series has a separate change. A dependency is not a copied commit.

In tinygo-org/net

Full Darwin networking also needs the merged net changes and a later src/net pin update. No upstream merge or current full-suite pass is implied by this list.

Additional independent gaps found in Crier are covered by #5655 (cookie-jar loader merge) and tinygo-org/net#82 (ListenConfig). They do not duplicate the process, TLS, deadline or server-TLS changes above. Current Crier comparison work is separate from its standard-Go release.

Published fork and downstream evidence

The net.2 fork release combines the coordinated changes at 95fba82a, with net 0f460803. It differs from accepted candidate e7d34c8c only in the version constant. Linux, macOS and Windows branch CI and tag CI passed on their first attempts.

Dispat source 909dc401, with harness 0990c6db, passed 796 test events with no failures or skips on each native Darwin ARM64 and Linux ARM64 candidate run. Crier source 7d687fc8 passed raw and stripped E2E on native Linux ARM64 and emulated Linux AMD64, each with 144 top-level tests and 156 passing events, no failures or skips. Both applications use TinyGo-built update fixtures and test trusted TLS, certificate refusal, original backup hashes and byte-identical offline rollback. Their workflows also exercise files, environment variables, concurrency and child processes. Crier includes real FFmpeg and webrender/canvas rendering.

Crier's unchanged 13-image pixel gate passed. It uses an approved two-line explicit-rounding webrender build patch for both compilers. The earlier gradient mismatch was permitted fused arithmetic, not a TinyGo compiler error. Candidate stripped sizes are 13,835,824 versus 30,277,794 Go bytes on ARM64 (54.30% smaller), and 16,446,968 versus 32,518,306 on AMD64 (49.42% smaller).

These are combined-candidate application results, not proof that this PR alone supplies the features. They supersede the earlier Crier comparison. Published-toolchain probes and application acceptance have since completed. The final Crier v1.1.1 release evidence is below. Dispat controls its own publication. WaitDelay, in-flight deadlines and full descriptor lifetime remain open. This enables tested CLI client workflows, not general Go or server compatibility.

The owner agreement is recorded in the closing response on #5645. The overlap is resolved. The ready-for-review request was attempted again on 6 September at the user's request. GitHub denied markPullRequestReadyForReview for the current account. A user or maintainer with permission must change the draft flag. No other route was used. This is an access restriction, not a downstream test hold or design disagreement.

Published Crier v1.1.1 evidence

Crier v1.1.1 is public at source acac2f0e and uses published TinyGo 0.43.0-net.2. Its final acceptance report and SHA-256 manifest identify the exact release bytes. The public tag, asset sizes and report digest were checked. These final sizes supersede the candidate sizes above.

Linux target Standard Go bytes Stripped TinyGo bytes Reduction
ARM64 30,277,794 13,835,840 54.30%
AMD64 32,518,306 16,447,000 49.42%

The report records 144 top-level tests and 156 passing events for each raw and stripped run on native ARM64 and emulated AMD64, with no failures or skips. It covers real CLI files, environment and concurrent work, child processes, TLS, update/rollback fixtures, uploads, real FFmpeg, and webrender/canvas rendering. The unchanged 13-image gate passes on both targets. AMD64 is exact; ARM64 has 12 exact images and four event-card pixels with channel difference 1. Both compilers use the same explicit-rounding webrender build patch. Standard Go tests, 90.6% coverage, lint and docs also pass.

This is combined-fork application evidence, not isolated proof for this PR or general server support. The generic emulated AMD64 os closure assertions still fail and also fail with ordinary Go under that emulation; they are not counted as passing. Native AMD64 CI and the final published AMD64 net package pass. The report retains other platform and deadline/descriptor limits. Tiny binaries are opt-in; normal install/self-update selects standard Go assets.

Published Dispat v1.8.1 CLI evidence

Dispat v1.8.1 CLI is public. Its size and SHA-256 manifest records build source 40c58236, Go 1.26.8 and TinyGo 0.43.0-net.2. The public asset metadata and manifest digest were checked.

Linux target Standard Go bytes TinyGo bytes Reduction
ARM64 9,896,098 6,299,032 36.35%
AMD64 10,895,522 6,697,528 38.53%

These are final published CLI asset sizes, not the earlier candidate measurements. They do not replace the separately identified test evidence or remove known runtime limits. The full release workflow has now completed successfully at the recorded build source, including its Windows, macOS and Ubuntu checks. This does not change the test and platform limits stated above.

@yohimik

yohimik commented Sep 2, 2026

Copy link
Copy Markdown
Author

Rebased on dev after the 0.42.0 release. The change applies on top of v0.42.0
as released without a conflict. An observation about the released toolchain:
testdata/hostcryptotls.go from this branch, built with the official v0.42.0
tarballs, prints "negotiated an unexpected version: 0" on linux/arm64 and on
darwin/arm64. The stub gives back a plaintext connection from tls.Dial that
verifies nothing. This change puts the real package in its place.

TinyGo replaces crypto/tls with a stub whose handshake does nothing, so a
program that dials https gets a plaintext connection behind the TLS API. That
stub is correct for a target with no OS below it, which has neither the code
size for a full TLS implementation nor usually a socket to speak it over.
Hosted linux and macOS have both, and there the crypto/tls of the Go standard
library compiles and runs.

Make the override conditional. Without an entry in the map, crypto/tls falls
under the "crypto/" merge, which links the package of the standard library into
the synthetic GOROOT. GOOS alone cannot decide this, because a baremetal target
reports GOOS=linux, so the build tags decide as well.

The goroot cache key is a hash of the merge links, so the two variants get
separate cache entries.

testdata/hostcryptotls.go does a TLS handshake over an in-memory pipe with a
certificate that it makes at run time. On the current dev branch it prints
"negotiated an unexpected version: 0", because the stub does no handshake. With
this change the handshake completes, the data goes through, and a client that
does not trust the certificate refuses it. loader/goroot_test.go covers the
targets that keep the stub, the baremetal one that reports GOOS=linux included.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant