Skip to content

feat(config): user-tunable context window with log-scale slider#120

Merged
quiet-node merged 21 commits intomainfrom
worktree-curious-hatching-wren
May 2, 2026
Merged

feat(config): user-tunable context window with log-scale slider#120
quiet-node merged 21 commits intomainfrom
worktree-curious-hatching-wren

Conversation

@quiet-node
Copy link
Copy Markdown
Owner

@quiet-node quiet-node commented May 1, 2026

Summary

  • Adds a user-tunable num_ctx setting with a log-scale slider (2K → 1M) and an editable token chip in the AI tab. Slider thumb visually aligns with tick labels at every doubling, and the value chip reflects the exact selected token count.
  • Plumbs the configured num_ctx through the /search pipeline so the router and universal judge calls send the same value the chat path and warmup use. Previously these calls omitted num_ctx, causing Ollama to reload the model at its default 4096-token window after the first /search message even when a larger context was configured.
  • Folds in the Keep-Warm work this branch was already carrying (pre-load on activation, configurable inactivity timeout, indefinite mode, manual unload, loaded-model status row) since both features touch the same [inference] config section and warmup path.

Context window slider details

  • 10 tick labels at every doubling (2K, 4K, 8K, 16K, 32K, 64K, 128K, 256K, 512K, 1M) match the log scale, where each 1/9 of slider travel doubles the value.
  • Ticks are absolutely positioned at i / 9 * 100% with an 8px horizontal margin matching the slider thumb radius, so each label sits under the thumb center at that value.
  • Fill gradient ends at the thumb center via calc(8px + var(--fill) * (100% - 16px) / 100%) instead of at --fill% of full track width.
  • Editable token chip accepts any value >= 2048; out-of-range values fall back to the previous valid value on blur.

Search-path num_ctx fix

  • RouterOptions carries num_ctx; request_json, call_router_merged, and call_judge accept it as an argument.
  • DefaultRouterJudge and DefaultJudge carry num_ctx, populated from app_config.inference.num_ctx in search/mod.rs.
  • Two new wire-level tests assert the request body contains "num_ctx":<value> for both router and judge calls.
image

Test plan

  • bun run test:all:coverage — 1152 vitest tests + 670 cargo tests pass with 100% line coverage gate
  • bun run validate-build — lint + format + typecheck + frontend + backend release bundle clean
  • Manually verify slider thumb position matches the value chip across the full range (2K → 1M)
  • Manually verify /api/ps reports the user's configured num_ctx after the first /search message (no auto-reset to 4096)
  • Manually verify Keep Warm: pre-load on activation, custom inactivity timeout, indefinite mode (-1), and the Unload now button

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
…tegration

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
… and eject

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
@quiet-node quiet-node force-pushed the worktree-curious-hatching-wren branch from 9db4279 to 9f10f68 Compare May 2, 2026 07:17
@quiet-node quiet-node changed the title feat(keep-warm): keep active model in VRAM between conversations feat(config): user-tunable context window with log-scale slider May 2, 2026
quiet-node added 10 commits May 2, 2026 02:29
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
@quiet-node quiet-node merged commit 1c18ddf into main May 2, 2026
3 checks passed
@quiet-node quiet-node deleted the worktree-curious-hatching-wren branch May 2, 2026 17:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant