[Artifacts] Separate deployable public files from compiler-private metadata

[cssbruno]

Priority

High — information disclosure prevention and artifact-contract clarity

Context

GOWDK currently places deployable web files and compiler/runtime metadata in the same build output tree. Generated applications copy most regular files from that tree into the embedded application filesystem, and the runtime can serve most embedded files by URL.

The current copy policy in internal/appgen/files.go is a blocklist:

filepath.WalkDir(sourceRoot, ...)
if unsafeEmbeddedDirectory(rel) { ... }
if unsafeEmbeddedFile(rel) { ... }
copyFile(sourcePath, targetPath)

runtime/app and gowdk serve similarly deny only a narrow set of filenames during request routing. In particular, gowdk-security.json is blocked, but other generated metadata can remain requestable.

Examples of files currently emitted at or below the public output root include:

• gowdk-routes.json;
• gowdk-assets.json;
• gowdk-build-report.json;
• gowdk-build-timings.json when enabled;
• openapi.json;
• asyncapi.json;
• sitemap and robots output;
• generated runtime assets and source maps depending on mode;
• audit/security metadata in selected flows.

Some of these are intentionally web-facing in certain deployments. Others exist for the compiler, generated runtime, CI, or operators and should not automatically become public merely because they share an output directory.

There is also a case-sensitivity gap in internal/safeasset: file basenames are lowercased, but blocked directory basenames are currently compared without normalization. A directory named Secrets, .Git, or Private may therefore bypass the intended directory-name blocklist on a case-sensitive host.

Problem

The current blocklist model has several weaknesses:

1. Compiler-private reports can become public. Build report events may contain source paths, page IDs, routes, handler symbols, guard metadata, contract identities, and generation details useful for debugging but unnecessary for public clients.
2. New metadata files default to public. Every newly emitted regular file is copied and served unless someone remembers to add it to multiple deny lists.
3. Runtime-required does not mean browser-public. The generated runtime may need route or asset metadata internally, but that does not require exposing the raw JSON at a public URL.
4. Deployment intent is ambiguous. OpenAPI, AsyncAPI, source maps, and similar outputs may be public, private, or omitted depending on the application, but the current artifact model does not encode that decision.
5. Policy is duplicated. Build generation, app embedding, disk serving, playground export, and other paths maintain overlapping but non-identical unsafe-file checks.
6. Case variants can bypass directory filtering. Sensitive directory checks are not normalized consistently.

A deny list is especially fragile for a compiler that continues to add reports and manifests.

Goal

Represent artifact visibility explicitly during planning and make all copy/embedding/serving paths fail closed: only artifacts classified as public should be reachable through the generated HTTP file-serving surface.

Proposed artifact classes

Introduce a typed visibility/ownership classification in the canonical application/artifact plan, for example:

type ArtifactVisibility string

const (
    ArtifactPublic         ArtifactVisibility = "public"
    ArtifactRuntimePrivate ArtifactVisibility = "runtime-private"
    ArtifactBuildPrivate   ArtifactVisibility = "build-private"
)

type PlannedArtifact struct {
    LogicalName string
    RelativePath string
    Kind string
    Visibility ArtifactVisibility
    Hash string
    Size int64
}

Suggested meanings:

Public

Files intentionally reachable by a browser or static host:

• HTML pages;
• public CSS/JavaScript/WASM/image/font assets;
• favicon and declared public component assets;
• sitemap.xml and robots.txt;
• explicitly opted-in API documentation artifacts.

Runtime-private

Files or data required by generated application code but not public URLs:

• route dispatch metadata;
• asset lookup metadata when not needed by clients directly;
• generated guard/endpoint metadata;
• build compatibility identity;
• internal precomputed runtime indexes.

These may be compiled into generated Go, embedded under a non-routable filesystem, or loaded through an internal API.

Build-private

Compiler, CI, audit, and operator artifacts:

• build reports and timings;
• security manifests and audit evidence;
• source maps when disabled for public deployment;
• diagnostic/inspection snapshots;
• temporary or provenance metadata.

These should remain available to tooling without being copied into the browser-served filesystem.

Proposed direction

1. Produce one authoritative artifact manifest

Artifact planning should emit a complete typed list of generated files and visibility. Avoid inferring visibility later from filename extensions or basenames.

The list should be deterministic, versioned, and reusable by:

• disk publication;
• generated app embedding;
• gowdk serve/preview;
• Docker/deployment packaging;
• clean/stale-file removal;
• artifact verification;
• future gowdk changes output.

2. Serve from an allowlist, not a deny list

Generated runtime and disk-backed serving should only resolve files present in the public artifact set. An unlisted regular file under the output root must not become publicly reachable.

This changes the default from:

public unless blocked

to:

private unless explicitly planned public

3. Separate embedded filesystems

Generated applications should avoid exposing one embedded tree containing both public and private artifacts through the static handler.

Possible designs include:

• embed only public files and compile runtime metadata into generated Go;
• embed public/ and internal/ trees separately, routing only public/;
• embed the complete generation but expose an fs.Sub containing only the allowlisted public tree.

The selected design must make accidental serving of runtime-private data structurally difficult.

4. Make documentation artifacts explicit

OpenAPI, AsyncAPI, source maps, and similar outputs require an explicit deployment policy:

• generated but private;
• generated and public;
• omitted.

Do not silently change existing intended public behavior without a compatibility decision, but encode the decision instead of relying on directory placement.

5. Centralize sensitive path policy

internal/safeasset or its replacement should centralize residual source/copy protections that remain necessary. Normalize directory and file comparisons consistently with strings.ToLower/case-fold policy where appropriate.

Sensitive directory checks must cover case variants such as:

.git  .Git  GIT
private  Private
secrets  Secrets

Visibility metadata should be the primary public-serving control; sensitive-name checks remain defense in depth for copying user-controlled trees.

6. Define migration behavior

Existing output layouts may contain metadata at the root. Migration should document:

• which paths remain generated;
• which stop being HTTP-accessible;
• whether runtime-private files move;
• whether static-host deployments should exclude private files or receive a separate public directory;
• how gowdk clean handles old public copies.

A compatibility note and release entry are required because scripts may currently fetch some JSON files directly.

Security considerations

• Do not assume that a public repository makes source paths, symbols, guard IDs, or deployment metadata harmless in every generated application.
• Public classification must be decided by compiler-owned artifact kind or explicit user policy, not by file extension alone.
• Unknown artifact kinds must default to private or fail planning.
• Runtime-private metadata should not be retrievable through path guessing, alternate index paths, case variants, or URL encoding.
• Source maps should follow production/development policy explicitly.
• Build-private files may still contain secrets accidentally emitted by user tooling; preventing HTTP exposure is defense in depth, not permission to serialize secrets.

Test plan

Add tests covering:

• public HTML/CSS/JS/assets remain servable;
• gowdk-build-report.json is not servable from generated apps or gowdk serve unless an explicit future policy opts it in;
• timings, audit/security, and inspection artifacts are private;
• route/asset metadata remains usable by the generated runtime without being browser-public where the selected design permits;
• OpenAPI/AsyncAPI public/private/omitted policy;
• source-map behavior in development and production;
• unknown planned artifact kind fails closed;
• files manually added to the output directory are not served merely because they exist;
• case variants of blocked source directories are skipped consistently;
• clean migration removes stale previously embedded/private files;
• backend-only and split frontend/backend output;
• disk serve, dev, preview, generated app, and Docker packaging consume the same classification.

Include an HTTP regression matrix that requests every generated metadata filename and proves only explicitly public artifacts return success.

Acceptance criteria

• Every compiler-owned artifact has an explicit kind and visibility classification before publication.
• Browser-facing file serving uses a public artifact allowlist rather than a filename deny list.
• Generated applications structurally separate public files from runtime/build-private metadata.
• Newly introduced artifact kinds default to private or fail validation.
• Build reports, timings, security/audit data, and inspection metadata are not publicly served by default.
• Runtime-required metadata remains available internally without requiring a public URL.
• OpenAPI, AsyncAPI, and source-map exposure have explicit documented policies.
• Sensitive directory checks are case-normalized consistently.
• gowdk serve, dev, preview, generated app embedding, packaging, and clean use the same artifact inventory.
• Regression tests enumerate generated metadata and verify its HTTP exposure policy.
• Migration and release documentation identify any paths that stop being public.

Non-goals

• Preventing users from intentionally copying a private file into a declared public asset location.
• Replacing application authorization for intentionally public API documentation.
• Encrypting build reports or audit output at rest.
• Treating filename blocklists as the primary solution.

Related

• [Supply Chain] Pin privileged workflow inputs and harden installer bootstrap #640 — supply-chain and release hardening
• Publish build and generated-app output transactionally #669 — transactional artifact publication
• [Testing] Add incremental-versus-clean build equivalence testing #686 — incremental-versus-clean artifact equivalence
• [Artifacts] Embed a deterministic build identity across all generated outputs #687 — deterministic build identity
• [Compatibility] Version and publish schemas for all public JSON contracts #689 — schemas for public JSON contracts
• [Serve] Prevent symlink escapes outside the selected output root #692 — rooted disk-serving containment