Added Rolling Application Log Files - #16
Closed
seanpar203 wants to merge 819 commits into
Closed
Conversation
Turn the gateway runner into a library seam an embedding host can drive, so a host process starts the gateway in-process and keeps its own main thread. `spawn` in `runner.rs` loads config, provisions, and binds on a dedicated thread with its own multi-thread runtime, confirms readiness through an mpsc handshake, and returns a `GatewayHandle` exposing `url()`, `shutdown()`, and `join()`. `run()` becomes a thin wrapper - spawn, install the Ctrl-C handler, join - so the binary's behavior is unchanged. - The bound listener is the readiness signal: startup errors (config, provisioning, bind) cross the handshake to the caller, and a bind conflict fails `spawn` with `StartupErrorKind::Bind` rather than reporting ready. - `shutdown_on_send` resolves only on an explicit send; a sender dropped without sending (a failed Ctrl-C handler) parks forever, so no failure can masquerade as an interrupt and stop the server. - Dropping the `GatewayHandle` signals shutdown without waiting; `shutdown()` signals and then joins the gateway thread. - Spawn tests use an ephemeral port and a raw TCP HTTP GET, keeping an HTTP client out of the dev-dependencies; they pin that readiness means `/health` answers, shutdown releases the port, drop signals shutdown, and error kinds cross the handshake.
Give the gateway config an optional `[workshop]` section describing the workshop UI server the gateway can host, with client credentials derived rather than duplicated. `WorkshopConfig` (new `workshop.rs` in the config crate) carries `bind` (default `127.0.0.1:7910`), `open_browser` (default false), and optional `[workshop.voice]` and `[workshop.tape]` sub-tables under `deny_unknown_fields`, and `ServerConfig::client_url()` derives the workshop's gateway base URL from the `[server]` bind, swapping an unspecified IP for loopback. Like `[server]`, the section is boot-only: `check_workshop_matches_boot` in `runner.rs` refuses a profile whose merged `[workshop]` differs, both at boot and on a mid-run switch. - There is no `[workshop.gateway]` sub-table: the workshop client's base URL and api_key derive from `[server]`, so no credential is duplicated and none can drift. - `tape_path` anchors an absent or relative tape path against the directory holding the boot config, never the process current directory. - A one-sided `[workshop]` (present on only one side) is refused, mirroring the strict `[server]` value-equality rule; the mid-run refusal reaches the caller as the switch stream's terminal error event and leaves live state untouched. - The gateway `AppState` boot arguments are grouped into a `BootOwned` struct holding the boot `[server]` and `[workshop]`. - Unit tests pin all four match-check cases and the boot-time refusal; an integration test pins that a mismatched switch fails with a terminal error event and leaves the live profile intact.
Let the gateway optionally host the workshop UI server on a second, loopback-only listener in the same process. `promptforge-ws-server` becomes an optional dependency behind a new `workshop` feature, with `workshop-cuda` forwarding to its `cuda` feature; the default feature set stays empty, so headless builds never pull whisper, CUDA, or the Node UI build into the graph. When the feature is compiled in and the boot config carries a `[workshop]` section, `spawn_if_configured` in the new `workshop.rs` builds the workshop `Config` programmatically and spawns it after the gateway listener is bound, and the `WorkshopHandle` rides inside `GatewayHandle`. - Shutdown sequences workshop first, then gateway (in both `shutdown()` and `Drop`), so the workshop's final gateway calls never hit a dead socket; workshop stop outcomes are logged, never returned. - A non-loopback workshop `bind` is refused at spawn with a config-kind `StartupError`; only the gateway's own listener may bind wider. - The workshop's client URL derives from `ServerConfig::client_url()` with the actually bound port swapped in for a port-0 bind, and the same api_key authorizes it; the tape path anchors to the boot config's directory. - `open_browser` opens the system browser at the logged workshop URL, and a browser that will not open is a warning, not a startup failure; `dep:open` joins the `workshop` feature for this, gated so the bare graph stays clean. - Without the feature the module is a stub whose `spawn_if_configured` hosts nothing and warns on a `[workshop]` section, so the runner stays feature-blind. - `/health` and `/v1/models` exist on both routers; a comment marks the collision as the known blocker for nesting the workshop under a path on the gateway listener, which is not built.
Make the shell boot one process that serves both the inference gateway and the workshop UI, instead of talking to a standalone workshop server. The shell's dependency swaps from `promptforge-ws-server` to `promptforge-gateway` with the `workshop` feature, and its default `cuda` feature forwards to `workshop-cuda`. Boot calls `promptforge_gateway::spawn` with profile `default`, waits on the hosted workshop's `/health` through the existing health-wait, opens the window at `workshop_url`, and shuts the `GatewayHandle` down on window close. - Discovery keeps its search-order shape (exe directory, current directory, the user profile's `.promptforge`) but looks for `gateway.toml`; the legacy `workbench.toml` fallback is dropped. - First run generates the pair the gateway needs to boot: `gateway.toml` with a loopback `[server]` bind on 8081, a random hex api_key from the OS-seeded CSPRNG (`rand` joins the shell dependencies), and a `[workshop]` section with the voice-model download sources, plus `profiles/default.toml` including the boot file. - The profile is written before the boot config and never clobbered when present, so regeneration after a deleted `gateway.toml` cannot erase a customized profile, and a failed generation never leaves a discoverable boot config missing its profile. - A boot config without a `[workshop]` section is a reported error, not a window on nothing: the shell compiles the `workshop` feature in, so `None` from `workshop_url` means there is no page to open. - Generated output is tested against the gateway crate's own profile resolution, so first-run files the gateway would refuse fail the suite. - The legacy `workshop.toml` flow and the standalone `promptforge-ws-server` binary stay untouched for development against an external gateway.
Record the workshop-hosting design in the two places a reader looks. The gateway README gains a "Hosting the workshop" section: the `workshop` and `workshop-cuda` feature flags with their build implications, field tables for `[workshop]`, `[workshop.voice]`, and `[workshop.tape]` including the boot-config-relative tape anchoring, the derived client credentials, and the boot-only rule with its mid-run switch refusal. The design log gains entries 88 through 92, each with choice, evidence, and cost. - Entry 88 records the second loopback listener with nesting under `/workshop/` as a documented future option, blocked by the `/health` and `/v1/models` route collisions. - Entries 89 and 90 record the derived credentials (no `[workshop.gateway]`) and the boot-only `[workshop]` rule, one-sided presence refused included. - Entries 91 and 92 record the workshop-first shutdown order and the shell booting the merged gateway with first-run generation of `gateway.toml` and `profiles/default.toml`. - Docs-only commit: no code or tests change.
Keep the serve shutdown tests wedging connections as intended now that the cross-site guard filters requests. The `wedge_http_connection` fixture sent `host: workshop`, which the guard refuses with an immediate 403 before the body is ever polled, so the connection drained gracefully and the tests expecting a forced shutdown failed. The fixture now sends a loopback `Host` and declares `application/json`, so the request passes the guard, reaches the body-consuming chat handler, and holds the graceful drain open until torn down. - Test-only change: production shutdown code is unchanged; the 403 is the guard working as designed and the fixture was stale.
Bring `module-ceilings.toml` back in step with the crate after the recent hardening work. Four new modules get entries at their measured sizes (`atomic.rs` 205, `backoff.rs` 229, `cross_site.rs` 338, `deadline.rs` 109), and seven grown modules have their ceilings raised to actual line counts with no padding. - The largest raises are `chat_ws.rs` (2851 to 2963) and `gateway.rs` (1107 to 1300); the rest are `app.rs`, `assets.rs`, `heartbeat.rs`, `relay.rs`, and `workspace.rs`. - Ledger-only change: no code or tests move.
The design document `design/what-promptforge-is.md` adopts the product's current name: the Workbench is now the Workshop. The swap covers the body prose, the section titles `1.2` and `7.1`, and the references entry. - Each of the 31 changed lines swaps the product name and changes nothing else; no content was added or removed.
The sweep in `sweep_orphaned_temps` removed only temp files ending in `TEMP_SUFFIX`, so a `workshop-state.json.tmp` orphan left by the pre-helper menu scheme survived. The sweep now also removes that fixed legacy name, held in a new documented constant `LEGACY_TEMP_NAME`. The doc on `sweep_orphaned_temps` now states that a missing directory is tolerated silently and reserves the logged-and-tolerated wording for the other failures. - The legacy name lives in `LEGACY_TEMP_NAME` beside `TEMP_SUFFIX`, with a doc comment recording which scheme wrote it. - The match widens to `!name.ends_with(TEMP_SUFFIX) && name != LEGACY_TEMP_NAME`, so the fixed name is removed even without the suffix. - A new test `the_sweep_removes_a_legacy_menu_temp_orphan` plants the legacy orphan beside an intact state file and checks the sweep removes one and spares the other. - The `atomic.rs` ceiling in `module-ceilings.toml` rises from 205 to 230, the actual line count after the constant, the wider match, and the test.
The test `a_stalled_route_answers_408_at_its_deadline` spent wall clock waiting for the deadline layer to answer 408 over a handler asleep for 30s. It now runs with `#[tokio::test(start_paused = true)]`, so the stall and the deadline advance virtually. Tokio's `test-util` feature is added as a dev-dependency of `promptforge-ws-server`, which `start_paused` requires. - A comment in the test records that paused time freezes real socket I/O and that the test stays safe only because the socketless `oneshot` does none. - The `deadline.rs` ceiling in `module-ceilings.toml` rises from 109 to 112, the actual line count after the added comment.
The jitter draw in `ReconnectBackoff` computed `xorshift(&mut state.rng) % (span + 1)`, where `span` falls back to `u64::MAX` when the nanos conversion overflows; the add would have wrapped had that fallback fired. The draw now uses `span.saturating_add(1)`. The generator `xorshift` is now `pub(crate)` and the decoder test in `gateway.rs` imports it instead of carrying its own copy. - `xorshift` moves from private to `pub(crate)` in `backoff.rs`, and its doc notes that the gateway tests seed it explicitly so each randomized failure names its seed. - The test-local `xorshift` copy inside `decoder_is_chunking_invariant_under_random_splits` is deleted; the test keeps its three explicit seeds. - The `backoff.rs` ceiling in `module-ceilings.toml` rises from 229 to 232, the actual line count; `gateway.rs` shrank and keeps its ceiling.
The parity comment above the traversal tests in `assets.rs` said a debug build must refuse names resolving outside `ui/dist/`, without noting the limit of that guarantee. One added sentence scopes the guarantee to request-supplied names and names the symlink bypass as outside it. No behavior changes; the tests are untouched. - The comment now records that rust-embed 8.12.0 deliberately still serves an out-of-root symlink planted inside `ui/dist/`, a bypass outside the parity the tests pin. - The `assets.rs` ceiling in `module-ceilings.toml` rises from 66 to 69, the actual line count after the added sentence.
`spawn` routed two non-bind failures through `StartupError::bind`, whose Display reads "failed to bind the listener", and both handshake-failure arms discarded the thread's join result. A new `StartupErrorKind::Thread` with a matching private `StartupRepr::Thread` and a `StartupError::thread` constructor now names those failures. Both arms join the thread through a new `failed_handshake` helper that downcasts the panic payload into the returned error text. - `StartupErrorKind` is `#[non_exhaustive]`, so the new `Thread` variant adds no breaking change for downstream matches. - `failed_handshake` reads the payload through `&*payload`: a comment records that `&payload` would unsize-coerce the Box itself into `dyn Any` and hide the real payload type from the downcasts. - `panic_message` downcasts to `&str`, then `String`, and falls back to "non-string panic payload" for a `panic_any` call. - Three new tests cover the helper: `a_panicked_gateway_thread_folds_its_panic_message_into_the_error`, `a_silent_thread_exit_is_thread_kind_not_bind_kind`, and `a_reported_handshake_error_survives_the_join_unchanged`; the `api_error.rs` test now also covers `StartupError::thread`.
The both-present arm of `check_workshop_matches_boot` reported only that "the profile's workshop settings differ", while the adjacent `check_server_matches_boot` names the exact differing field and its values. The check now compares the four fields in declaration order through a new `first_workshop_difference` helper and names the first difference in the validation message. The message keeps the "[workshop] mismatch" prefix the existing tests assert on. - `bind` and `open_browser` print both values; `voice` and `tape` print both Debug forms, and neither carries a secret. - The helper's fallback arm is unreachable until the config grows a field the check does not name yet; its doc says so. - A new test `workshop_mismatch_names_the_first_differing_field` exercises all four fields and asserts both values appear for `bind`.
The no-workshop stub of `spawn_if_configured` suppressed `clippy::unnecessary_wraps` with `#[allow]`, which stays silent forever. The suppression is now `#[expect]` with the same reason, so it warns once it goes stale, for example if the stub ever gains a fallible path. - Verified against `-D warnings` in both feature configurations: the lint still fires in the no-workshop build, so the expectation is fulfilled, and the workshop build compiles the hosted variant instead.
The `open_browser` honor called `open::that` on the workshop URL inline, so no test could observe it without opening a real browser. `spawn_if_configured` splits into a thin production wrapper that passes `open::that` over a new `spawn_with_opener` core that takes the opener as a plain closure. - The opener parameter is `impl FnOnce(&str) -> std::io::Result<()>`, a closure injected per call rather than stored state. - Three tests cover the honor: `the_open_browser_honor_opens_the_workshop_url`, `the_opener_never_runs_without_the_open_browser_honor`, and `a_failing_opener_does_not_fail_the_spawn`. - Each test spawns a real workshop on an ephemeral loopback port with the tape anchored in a tempdir, matching the runner's existing spawn fixtures.
`GatewayHandle::shutdown` stops a hosted workshop, waiting out its bounded drain, before sending the gateway's graceful-shutdown signal, so the workshop's final gateway calls never hit a dead socket. No test asserted that order. A test-only `mpsc` observer on `GatewayHandle` now records `ShutdownStep::WorkshopStopped` after the drain returns and `ShutdownStep::GatewaySignaled` after the shutdown send. - The seam is `cfg(test)`-gated: the `observer` field, the `ShutdownStep` enum, the `observe_shutdown` setter, and the two `record` calls vanish from production builds. - Both records are synchronous inside `shutdown()`, so the tests collect with `try_iter` and carry no timing dependence. - `shutdown_drains_the_workshop_before_signaling_the_gateway` asserts the two-step order; `shutdown_without_a_workshop_signals_the_gateway_only` covers the no-workshop path, which also keeps the seam used in every test build.
The gateway's startup path called `load_server` and `load_workshop` on the boot file back to back, and each ran its own `collect_config_chain` plus `${VAR}` interpolation, so the same include tree was read, parsed, and merged twice per boot. `promptforge-gateway-config` gains `load_boot_sections`, which resolves the chain and interpolates once, then extracts the `[server]` and optional `[workshop]` sections from the same document. `load_startup` in the gateway now makes one boot-file pass instead of two, with the parity checks unchanged.
- Section extraction moves into shared `server_section` and `workshop_section` helpers used by all three loaders, so the single-section entry points keep their exact behavior: `load_server` still requires `[server]`, and `load_workshop` still returns `None` for a missing section even when `[server]` is absent.
- The combined loader deliberately does not back `load_workshop`: a workshop-only file has no `[server]`, and the existing include-chain test for that case pins the tolerant behavior.
- New tests cover the combined loader: `load_boot_sections_reads_both_sections_without_full_validation`, `load_boot_sections_returns_none_workshop_when_the_section_is_absent`, and `load_boot_sections_requires_a_server_section`.
The shell's `workshop_url` mapped the gateway handle's `Option<&str>` to the window URL inline, so the user-facing error arm - a boot config with no `[workshop]` section, leaving the shell with no page to open - had no test coverage. The mapping now lives in `workshop_url_from`, a pure `Option<&str>` to `anyhow::Result<String>` function that `workshop_url` delegates to. Behavior is unchanged: the same context message, the same `Ok` passthrough. - `workshop_url_from_passes_the_url_through` pins that a present URL passes through untouched. - `workshop_url_from_names_the_missing_workshop_section` pins that the `None` arm's error names the `[workshop]` section, so the message cannot drift into telling the user nothing actionable.
The shell's README sends readers to the gateway README for the field reference, and the `[workshop]` tables landed, but the required `[server]` section's `bind` and `api_key` were documented nowhere in it. A short "The `[server]` section" table now sits between Usage and Hosting the workshop: both fields required, both accepting `${VAR}` interpolation, with a pointer to the boot-ownership rule the two sections share.
`serve_thread` wrapped a tokio runtime build failure in `StartupError::bind`, whose Display reads "failed to bind the listener", misnaming what actually failed. The runtime build now reports through `StartupError::thread`, and the `Thread` kind's doc widens to name the runtime build alongside the spawn, exit, and panic cases. - Forcing a real runtime build failure takes resource exhaustion, so the path is covered by the existing kind-mapping test on the constructor rather than a dedicated spawn fixture. - The kind enum is `#[non_exhaustive]`, so the widened doc is the only API-surface change.
`failed_handshake`'s payload reader had three arms and a test for one: the panicked-thread fixture exercises the `&str` payload, but the owned `String` arm (a `panic!` with format args) and the non-string fallback (a `panic_any` call) were new behavior nothing would catch breaking. One unit test now calls `panic_message` directly with each payload shape. - `panic_message_reads_each_payload_shape` passes a borrowed `&str`, an owned `String`, and a `u64` standing in for a `panic_any` payload, pinning the "non-string panic payload" fallback text.
The `send` feature's borrow tracking relied on compiler internals and broke on newer rustc; the fix shipped in mlua 0.12 and was never backported to 0.10, so the pin had to move before the coroutine work touches `lua/`. `Cargo.toml` bumps `mlua` to 0.12 with features `lua55`, `vendored`, `serialize`, and `send`; the `async` feature stays off. The breakage is mechanical: `set_hook` and `set_metatable` are fallible in 0.12, so their call sites now propagate `Result`. - `install_instruction_budget` now returns `Result<()>`; VM construction propagates a hook-install failure through the existing `construction_failed` path. - The `set_metatable` calls in `crates/promptforge-core/src/lua/sys.rs` and `crates/promptforge-core/src/lua/tools_bridge.rs` now propagate with `map_err(Error::lua)`. - No test logic changed: one call site in `crates/promptforge-core/src/lua/tests.rs` gained an `expect` for the fallible hook install. Stale doc comments saying Lua 5.4 now say 5.5.
Every exit path out of a section entry must tear the VM down exactly once, and the driver's explicit `SectionContext::teardown` calls left that to each caller. `SectionContext` now owns the boundary through a `Drop` impl with an armed/disarmed `completed` flag: the success path calls `mark_completed` after the final `var` read-back, so the drop fires the `LUA_TEARDOWN_STARTED`/`LUA_TEARDOWN_SUCCEEDED` pair on every path and `SECTION_FINISHED` only when armed. The frame holds its VM as `Option<SectionVm>` plus its `name` and `execution`, so the destructor needs no parameters. - The constructors run `setup_section_vm` on the bare VM before the frame exists; a setup failure calls `vm.teardown` directly and no `SectionContext` is ever created. - The `h1_try!` macro in `crates/promptforge-core/src/execute/h1.rs` is gone; the fanout arm replaces `frame.teardown(&worker.name)` with `drop(frame)` at the same point, so the teardown pair still precedes the arm's terminal observation. - The write-only fields `write_scope` and `execute_depth` are removed; their values reach the control globals and the VM setup through locals. - The live H1 frame is never marked completed, so `SECTION_FINISHED` stays a walked section's boundary and never fires for the setup pass. - A new test, `an_erroring_section_tears_down_exactly_once_without_finishing`, pins the error path: the teardown pair fires exactly once and `SECTION_FINISHED` does not fire.
The yield/resume boundary needs validated message types: what a script can cause the host to do must be one short read with compiler-checked per-variant fields. New module `crates/promptforge-core/src/execute/protocol.rs` defines a closed `Request` enum with all four variants (`Infer`, `Execute`, `Fanout`, `Mcp`) and an `Answer` enum that renders the `(ok, result)` resume envelope. Validation is strict: `Request::from_yield` checks every field at the trust boundary, and any yield that is not a well-formed request table fails with the fixed `Error::Lua` message "scripts may not yield directly". - Field reads go through `raw_field`, so a script-space metatable cannot intercept or forge a request field. - Each `Answer` variant owns its typed `Error` until `into_envelope` consumes it, returning the rendered envelope plus the typed error, so the driver never sees a stringified failure; this holds for leaf and structural variants alike. - A received `Mcp` request is a typed protocol error through `mcp_reserved()`: the variant is reserved, not dispatched. - Nothing outside the module calls it yet; `pack_sequence` in `crates/promptforge-core/src/lua/vm.rs` was promoted to `pub(crate)` for the fanout envelope rather than duplicated. - Twenty-one unit tests cover well-formed parses, malformed rejections, and envelope round-trips.
Yield cannot cross the C boundary, so the host calls that suspend must become Lua shims that yield request tables. The new `crates/promptforge-core/src/lua/__impl_coro.lua` turns `models.infer`, `handle:infer`, and `execute` into `coroutine.yield` wrappers that consume the `(ok, result)` envelope and raise failures with `error(result, 0)`, so no position prefix leaks into script errors. `setup_section_vm` gains a `VmSetupMode`: `Legacy` (the default) keeps the existing Rust control globals, and `Scheduler` installs the shims after the host tables exist and before `replay_shared`. - The shim source is embedded with `include_str!`, compiled once behind a `LazyLock` through `LuaProgram::compile_internal`, and named `@crates/promptforge-core/src/lua/__impl_coro.lua`, so shim errors render as verbatim file:line references the line mapper never rewrites. - Model handles reach author code as sealed proxy tables through `wrap_handle`: `infer` is a Lua method that yields the inner userdata, and `__metatable` seals the proxy so `getmetatable` cannot hand the unshimmed userdata back to author code. - The `coroutine` standard library loads at shim-install time and the global is stripped again before the install returns, so author code cannot yield directly; legacy VMs never load it. - `SectionVm` gains a `coro_shims` flag so the captured model alias globals install as shim-wrapped proxies; `var_snapshot_table` in `crates/promptforge-core/src/lua/sys.rs` materializes the guarded `var` as a plain table for the `execute` request's snapshot. - No production caller constructs `Scheduler` mode yet, and there is no `fanout` shim; the change scopes itself to `infer` and `execute`. Seven focused tests cover well-formed yields, prefix-free error raising, unmapped shim frames, and proxy sealing.
The scheduler needs a chunk execution path that can suspend on a shim yield, so `SectionVm` gains a resume-based path next to the legacy `Function::call` path, which stays untouched. `start_block_coro` creates one coroutine per Lua block on the section's persistent VM, `resume_block_coro` drives a suspended block, and the new `CoroStep` enum (`Yielded` or `Done`) reports the boundary. Jump-slot precedence, runtime-error mapping, and scalar-return handling match the legacy path, so block results and rolled-forward VM state are indistinguishable. - Instruction hooks are per-coroutine in PUC Lua, so the budget/cancellation hook moves into `InstructionBudget`, a shared `Arc<AtomicU64>` counter installed on each block coroutine through `Thread::set_hook`; one counter spans every chunk of a section and now bites inside resumed coroutines. - Six focused tests pin the resolved spikes: the hook fires inside a resumed coroutine, the budget spans block coroutines on one VM, a yield crosses `pcall`, `jump` propagates through `Thread::resume` unchanged, `@`-prefixed chunk names render verbatim through resume, and scalar returns plus `var`/`reply` state roll forward across blocks. - No production caller exists yet: `start_block_coro`, `resume_block_coro`, and `CoroStep` carry dead-code allowances until the scheduler's driver loop consumes them.
Section execution needs a driver that resumes a chain's coroutine, matches the yielded request, dispatches it, and resumes with the answer, all on one thread. New module `crates/promptforge-core/src/execute/scheduler.rs` adds `Scheduler`, which owns a chain arena indexed by a `ChainId` newtype, a LIFO execute stack, a FIFO ready queue, a pending table mapping in-flight requests to chains, and an unbounded answer channel fed by `spawn_local` infer tasks. A `Chain` owns its `SectionContext`, position, in-flight block coroutine, walk-scoped `reply`/`var` slots, and a per-chain `execute_depth` field. - `Scheduler` lives entirely in the driver loop's stack frame - no `Arc`, no `Mutex`, unreachable from Lua; `RunContext` stays the ambient read-mostly context and the two are deliberately not merged. - The recursion cap reads the chain's `execute_depth` field, never the stack length, because fanout arms will increment depth without sitting on the execute stack. - Dispatch failures (the depth cap, target resolution, child construction) resume the caller through the error envelope, so an author `pcall` catches them exactly as on the legacy callback path. - `Infer` dispatch spawns one gateway round through the new `infer_round`, which shares its reporting tail with the legacy hook through the extracted `accept_infer_completion`; `Execute` pushes and pops the chain stack; `Fanout` and `Mcp` fail with the typed reserved errors `fanout_reserved()` and `mcp_reserved()`. - The driver body runs inside a `tokio::task::LocalSet` and selects on the answer channel and the cancellation notification; cancellation aborts the in-flight I/O tasks and returns `Error::Interrupted`. - `run_section_prose` was extracted from `run_one_section_impl`, so the scheduler's driver and the legacy block loop run the identical prose path. - A completed jump still errors the chain (the walk translation lands later), the `JoinState` join table is defined but unused, and no production caller exists until the flip. Six `current_thread` tests cover the nested-execute-plus-inference gate, cancellation while suspended on infer, the depth cap, prose client seeding, reply read-back, and dispatch failure into `pcall`.
The scheduler now runs the walk's core rules as chain transitions, so a run advances through sections in document order with walk-scoped state. `Chain` gains the `reply`, `var`, and `addressed` slots, and frame construction moves out of `start_chain` into a new `enter_section` transition. At a section's end, `end_section` reads the final `reply` and `var` back while the VM is live and rolls them forward. - `enter_section` skips off-walk sections on fall-through before any frame exists; an addressed arrival (an execute target) runs its section anyway, and one entry consumes the flag. - `Chain.frame` is now `None` before the first entry and between sections; `finish` takes the chain's `reply` slot as the result text when the walk runs off the slice's last section. - `end_section` arms the frame with `mark_completed` before the drop, so the teardown boundary fires `SECTION_FINISHED` for the completed section. - An execute chain's `var` slot seeds from the caller's snapshot and is discarded with the chain, so the caller never sees the chain's writes. - A jump still returns the typed later-step `Error::Lua`; a received `fanout` or `mcp` request stays the protocol's reserved error. - The change touches only `execute/scheduler.rs` and its test module; fifteen new scheduler tests pin fall-through order, off-walk skips, reply and `var` roll-forward, and the run-global id counter.
The scheduler's walk now applies a Lua `jump` as a control transfer instead of failing the chunk with the placeholder `Error::Lua`. `Chain` gains a `positions` stack of suspended parent walk positions: `apply_jump` reads `reply` and `var` back from the jumper's live frame, marks the frame completed, resolves the heading with `resolve_jump_target`, and moves the walk. A `JumpTarget::Sibling` sets the chain's index within its slice, addressed; a `JumpTarget::Child` pushes the current position and descends into the jumper's child slice, and `pop_position` resumes the parent after the jumper when the child level exhausts. - The jumper's frame closes as completed before the target resolves, so `SECTION_FINISHED` fires for the jumper and an author's `reply = nil` steers what the target sees; a failed resolution returns `Error::Lua` after that close. - A scalar return inside a descended child level ends the whole chain, and the run-global id counter counts entries across descents. - Twenty-six new tests in `execute/tests/scheduler.rs` mirror the legacy jump, return, and observation cases; the legacy suite is untouched, and a received `fanout` or `mcp` request still fails with the typed reserved error.
Keep `cargo doc --workspace --no-deps` green after the taxonomy rename. Demote broken `[...]` links on private items in `config.rs` and `session_agents.rs` to plain backticks so rustdoc no longer fails the workspace doc build. - Grep matrix for old crate names is clean outside allowlisted historical paths. - `cargo test --workspace --locked` stays green (~2477 passed).
Drop the unused headless CLI and its scratch runbook. Remove the crate, guide chapter, GUIDES entry, and root README install/run rows so the Workshop and library facade own the product surface. Regenerates the user guide and clears the rustdoc name collision with the `promptforge` facade.
Apply `cargo fmt --all` so CI `cargo fmt --all --check` passes. Mostly import reordering and line wrapping from the gateway/shared/build renames.
The dissolved offline suite now lives under promptforge-core integration tests and inherits workspace `-D clippy::expect-used`. Match the gateway IT pattern with a crate-level expect so setup panics stay idiomatic.
Delete superseded design docs, orphaned configs, stale research, and unreferenced assets that drifted from the code. Repair the README, gitignore, and workshop-server README references that pointed at them.
The tape mechanism is dead. Remove WorkshopTapeConfig and tape_path() from gateway-config, the config UI's [workshop.tape] subsection, the example config block, the gitignore entry, and rewrite the stale parity-gate docstring.
Distribution is signed installers, not the registry. Mark every crate publish = false and remove the docs.rs and cargo-semver-checks metadata tables. dist-workspace.toml and the release workflows stay.
The current-state inventory drifted with every refactor. Replace crate names, file counts, tuning knobs, and build-stage status with the four product-level roles: the engine, the gateway, the workshop, the library. The thesis, the history, and the dated measurements stay concrete. Deleted products disappear into the general terms.
Remove the per-crate user guides, the topic pages, and the assembled single file. guide/src is now a minimal skeleton; the four audience sets land next. build-user-guide is gutted to a compiling skeleton ahead of its rework into the book assembler. guide/scratch/ is gitignored for the generator's pipeline intermediates.
A compressed adaptation of dokuman as a frontier-harness tool file: lens blocks per audience, manifest-contract extraction, evidence firewall, single checkpointed writer, per-chapter thoroughness gate. Written in STE under the prompts-rulebook protocol.
First output of the tools/document.md generator, run with the language lens: ten chapters covering frontmatter through fanout, extracted from promptforge-parser, promptforge-core, and the shipped examples. Gate 1: 93 of 93 manifest files extracted. Gate 2: all ten chapters carry an approve verdict.
Prominent evergreen download links for the Workshop on Windows, macOS, and Linux, headed by a new banner-style download image. The six existing banners and the portrait stay in place. The per-crate table and registry references are gone; the guide link points at the four audience sets. The evergreen assets were backfilled onto the workshop-latest release from v0.2.0, so every link resolves today.
The crate now walks the four guide/src/<set>/ directories, synthesizes per-part landing pages, regenerates SUMMARY.md in audience order, writes the per-set single-file exports, and fails on any unresolved SUMMARY link. Chapters carry numeric prefixes so a name sort is the reading order; the generator's audit stage writes them that way. Unit tests cover the walk, the landing pages, the TOC shape, the link check, and determinism.
Records the ownership split (assembler owns SUMMARY.md and the part landing pages, the generator owns introduction.md, chapters are hand-editable), the no-freshness-gate rule, and the one-command rebuild procedure.
Generated with the gateway lens of tools/document.md: ten chapters covering install, the configuration file, remote and local models, speech, profiles, dominions, safe editing, the configuration UI, and serving. Gate 1: 226 of 226 manifest files extracted. Gate 2: all ten chapters carry an approve verdict.
Generated with the agent lens of tools/document.md: ten chapters from the agent loop through the full reference loop, extracted from promptforge-agent, promptforge-lua, and the shipped chat agent. Gate 1: 33 of 33 manifest files extracted. Gate 2: all ten chapters approved.
Generated with the workshop lens of tools/document.md: ten chapters from the application shell through the update flow, extracted from the workshop crates including the TypeScript UI sources. Gate 1: 197 of 197 manifest files extracted. Gate 2: all ten chapters approved.
The assembler regenerates SUMMARY.md, the four part landing pages, and the four single-file exports over the checked-in sets. The introduction comes from the intro lens of tools/document.md over the generalized design doc: the thesis, the four moving parts, and audience routing, verified by gate 2.
Replace push_str(&format!(..)) with write! and use a case-insensitive extension check in build-user-guide, and extend the test-module allows in the workshop bridge for the pointer lints the #[implement] macro expansion trips.
The gateway package now opts into cargo-dist with `dist = true` under `[package.metadata.dist]` in `crates/gateway/Cargo.toml`. The setting sits beside the existing `features = [workshop]` and `include = [packaging/gateway.service]` entries, so the dist build of the gateway carries the workshop feature and the sample systemd unit.
A failed model round left the status bar stuck: the turn-dispatch push set the sustained Thinking activity, and only on_assistant_reply's idle push cleared it, so a round that failed never released the amber LED. SessionObserver::observe now also calls push_failure with Activity::General on Observation::ModelTurnFailed, beside the existing error-frame send. - module-ceilings.toml raises the session_agents.rs ceiling to 1100 and records the reason, as the ratchet requires. - The new a_failed_model_turn_pushes_a_terminal_failure_status test pins the error severity, the Activity::General activity, and the error-frame text.
The workshop desktop app lacked persistent logging for diagnostics and panic backtraces. `crates/workshop/src/logging.rs` introduces `init_logging`, configuring a `tracing_subscriber` with daily rolling files via `tracing-appender` alongside stderr output, and installs a custom panic hook. `main` in `crates/workshop/src/main.rs` initializes the subscriber and replaces ad-hoc `eprintln!` calls with structured `tracing` events. - `resolve_log_dir` places log files under platform per-user directories and allows an override through `PROMPTFORGE_LOG_DIR`. - `install_panic_hook` captures unhandled panics and backtraces through `tracing::error!` before invoking the previous panic hook. - Existing test suites are unchanged; new unit tests in `crates/workshop/src/logging.rs` verify directory resolution paths and fallbacks.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Essentially this PR introduces capturing logging that's happening in different parts of the application and storing logs in the idiomatic place PER platform:
• macOS:
~/Library/Logs/PromptForge/• Windows:
%LOCALAPPDATA%\PromptForge\logs\• Linux / BSD:
${XDG_STATE_HOME:-~/.local/state}/promptforge/logs/And supports an overridable custom location by setting the
PROMPTFORGE_LOG_DIRenv variable.