Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .dagger/modules/e2e/fixtures/generate-deno/app/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
/**/node_modules/**
/.env
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
name = "generate-deno-app"
engineVersion = "v1.0.0-beta.9"

[runtime]
source = "typescript"
5 changes: 5 additions & 0 deletions .dagger/modules/e2e/fixtures/generate-deno/app/deno.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"tasks": {
"dev": "deno run main.ts"
}
}
9 changes: 9 additions & 0 deletions .dagger/modules/e2e/fixtures/generate-deno/app/src/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
import { object, func } from "@dagger.io/dagger"

@object()
export class GenerateDenoApp {
@func()
hello(): string {
return "hello from deno"
}
}
49 changes: 49 additions & 0 deletions .dagger/modules/e2e/generate.dang
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,55 @@ type GenerateChecks {
null
}

"""
A Deno module gets the same bindings, bundle and entrypoint as any other —
only its config differs, because Deno resolves imports through deno.json
rather than node_modules plus tsconfig paths.

The failure this guards against is generating a Deno module as if it were a
Node one: the tree would look complete while the runtime had no way to
resolve @dagger.io/dagger at all.
"""
generateDenoModuleCheck(ws: Workspace!): Void @check {
let path = fixtures.denoModule
let changes = typescriptSdk.mod(ws, path: path).generate(ws)
let tree = changes.after.directory(path)

Asserts.generated(changes, path + "/sdk/client.gen.ts")
Asserts.generated(changes, path + "/__dagger.entrypoint.ts")
Asserts.generated(changes, path + "/deno.json")

Asserts.notAdded(
changes,
path + "/package.json",
"a Deno module should not be given a package.json",
)
Asserts.notAdded(
changes,
path + "/tsconfig.json",
"a Deno module resolves through deno.json, not tsconfig paths",
)

let denoConfig = tree.file("deno.json").contents
Asserts.stringContains(
denoConfig,
"./sdk/index.ts",
"deno.json should import @dagger.io/dagger from the generated sdk",
)
Asserts.stringContains(
denoConfig,
"experimentalDecorators",
"deno.json should enable the decorators modules are written with",
)
Asserts.stringContains(
denoConfig,
"\"dev\"",
"generation should preserve the user's own deno.json entries",
)

null
}

"""
Generating a module with a local dependency must resolve that dependency and
emit its typed bindings alongside the module's own — otherwise the module's
Expand Down
7 changes: 7 additions & 0 deletions .dagger/modules/e2e/util.dang
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,12 @@ type Fixtures {
let configModule: String! = root + "/config/app"
let configuredModule: String! = root + "/config/configured"
let configuredDenoModule: String! = root + "/config/configured-deno"

"""
A Deno module managed by this SDK: exercises the one branch in generation
that depends on the runtime, deno.json instead of package.json + tsconfig.
"""
let denoModule: String! = root + "/generate-deno/app"
let configBareModule: String! = root + "/config/bare"

# A client binds to client/app (which depends on client/dep) and is generated
Expand Down Expand Up @@ -185,6 +191,7 @@ type Fixtures {
managedTomlModule,
depAppModule,
depLibModule,
denoModule,
parentSourceModule,
]
}
5 changes: 5 additions & 0 deletions dagger.toml
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,11 @@ path = ".dagger/modules/e2e/fixtures/skip/app"
[[modules.typescript-sdk.as-sdk.modules]]
path = ".dagger/modules/e2e/fixtures/managed-toml/app"

# A Deno module: same bindings and entrypoint as node/bun, but its config lives
# in deno.json, the only part of generation that branches on runtime.
[[modules.typescript-sdk.as-sdk.modules]]
path = ".dagger/modules/e2e/fixtures/generate-deno/app"

[[modules.typescript-sdk.as-sdk.modules]]
path = ".dagger/modules/e2e/fixtures/generate-deps/app"

Expand Down
113 changes: 113 additions & 0 deletions design/client-bundle.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
# Design: dev-engine clients via a user-provided local bundle

Status: proposed.

Companion to [`client-gen.md`](./client-gen.md). Covers **option (b)** from the
codegen review — letting a generated client work against an engine version that
isn't published to npm — **without any engine changes**.

## Problem

A generated client pins `@dagger.io/dagger` to the bound module's `engineVersion`
(`config-updator` → `npmVersion(engineVersion)`). Released engine → real npm
version, `npm install` works. Dev/unreleased engine (e.g. `1.0.0`, `1.0.0-0`,
`…-dev.…`) → not on npm, install fails.

## Approach — respect a user-vendored local `./sdk`

Rather than have the SDK ship/build a bundle or the engine expose one (too much
complication), let the **user** point `@dagger.io/dagger` at a local bundle in
their client dir and have codegen **respect** it:

```jsonc
// <client-dir>/package.json (user-authored, for a dev engine)
{
"dependencies": {
"@dagger.io/dagger": "./sdk" // or "file:./sdk"
}
}
```

with the bundle vendored at `<client-dir>/sdk/` (core.js, core.d.ts, index.ts,
telemetry.ts + a small `sdk/package.json` — see "Obtaining the bundle").

The **only** SDK responsibility is: on (re)generation, do not clobber a
`@dagger.io/dagger` dependency the user has set to a local path. This mirrors
upstream's `Local` SDK-lib origin (`detectSDKLibOrigin`: `"@dagger.io/dagger" ==
"./sdk"` ⇒ Local), but with zero engine/bundle work on our side.

### What has to change (two small things)

1. **`config-updator` must preserve a local `@dagger.io/dagger`.**
`updateClientPackageJSON` currently *always* overwrites the dep with the
engine version:

```go
packageJSON, _ = sjson.Set(packageJSON,
"dependencies."+gjson.Escape(daggerLibPathAlias), npmVersion(engineVersion))
```

Change it to: **if the existing value is a local reference** (starts with
`.`, `file:`, or `link:`) keep it; **otherwise** set the engine-version pin.
So:
- fresh client / no dep / a version → engine-version pin (Remote, today's behavior);
- user set `./sdk` / `file:./sdk` → **preserved** (local bundle).

tsconfig/deno writers already use `setIfNotExists`-style semantics, so a
user-provided `paths`/`imports` mapping for `@dagger.io/dagger` is already
preserved; no change needed there beyond confirming they don't clobber a
user's `./sdk` mapping.

2. **Generation must read the existing client-dir config.** Today
`generateClient` / `generateAllClient` pass an **empty** directory as
`existing` to `clientDirectory`, so `config-updator` reads a non-existent
`/existing/package.json` and starts from `{}` — the user's `./sdk` dep is
never seen. Pass the real client-dir config instead, e.g.
`ws.directory(<client.path>, include: ["package.json", "tsconfig.json",
"deno.json"])`, mounted at `/existing`. (Guard for a not-yet-existing client
dir → fall back to empty.)

The vendored `sdk/` directory itself survives regeneration for free: the fork
stages the client via `withDirectory(client.path, generated)`, which is an
**overlay** — files the user has under `<client-dir>/sdk/` that codegen
doesn't emit are preserved.

That's it. No new engine primitive, no bundle shipped by this repo, no new
generator modes.

## Obtaining the bundle (user side, out of SDK scope)

The SDK does not produce the bundle. For a dev engine the user vendors
`@dagger.io/dagger` into `<client-dir>/sdk/` — the same static library the engine
builds (`core.js` from `bun build ./src/index.ts --external=typescript
--target=node`, `core.d.ts` via the rollup dts config; see
`toolchains/engine-dev/build/sdk.go:158-177`) plus a thin `index.ts`
(`tsutils/client/index.ts`), `telemetry.ts`, and a minimal `sdk/package.json`
(`name: @dagger.io/dagger`, `main: ./core.js`, `types: ./core.d.ts`, `exports`
including `./telemetry`). We can document this / ship a helper script later, but
it is not required for the codegen change above.

## Open questions

- **Runtime resolution.** `"@dagger.io/dagger": "./sdk"` (or `file:./sdk`) makes
`npm/yarn/bun install` link `./sdk` into `node_modules`, so a standalone client
the user runs with `node`/`tsx` resolves the specifier at runtime. Confirm this
round-trips (the `sdk/package.json` `main`/`exports` must be correct); tsconfig
`paths` alone only covers typecheck, not runtime.
- **Local-ref detection.** Preserve on prefix `.` / `file:` / `link:`. Confirm
that covers the shapes we want (`./sdk`, `file:./sdk`) and never matches a real
version/range.
- **Should a version pin also be preserved when the user set a specific one?**
Proposal: no — the SDK keeps owning the *version* pin (updates it to track the
engine on regen); it only steps aside for **local** refs. Revisit if users want
to pin a specific published version.

## Key references

- This repo: `helpers/config-updator/main.go` (`updateClientPackageJSON`,
`npmVersion`, `setIfNotExists`), `typescript-sdk.dang` (`clientDirectory`,
`generateClient`, `generateAllClient` — the `existing` argument),
`design/client-gen.md`.
- Upstream `dagger/dagger`: `sdk/typescript/runtime/config.go`
(`detectSDKLibOrigin`, the `./sdk` = Local rule), `tsutils/client/index.ts`,
`toolchains/engine-dev/build/sdk.go:158-177` (how the bundle is built).
9 changes: 9 additions & 0 deletions design/module-gen.md
Original file line number Diff line number Diff line change
Expand Up @@ -566,6 +566,15 @@ Validated three ways rather than one:
really proven by executing a module.
- **By an e2e check** asserting the whole tree contract, not one marker file.

**Freshness is enforced already, without a bespoke check.** The engine surfaces
each `@generate` function as a check that fails when running it would produce
changes — so `packager:library-bindings` and `packager:library-bundle` are the
staleness check, with a better message than a hand-written one:
*"generate function packager:library-bindings produced changes; run
'dagger generate packager:library-bindings' to apply"*. Verified by tampering
with the committed bindings and watching it fail. An engine bump therefore
shows up as a failing check rather than silent drift.

**VCS files are not written** (decided): no `.gitignore`, no `.gitattributes`.
The engine appends to both around codegen, but for a workspace module the
ignore list is reduced to `node_modules`/`.pnpm-store` anyway, which is the
Expand Down