Skip to content

Certified local mode: managed on-device model with verified setup and egress guard #1162

Description

@anandgupta42

Data teams in compliance-bound environments (finance, healthcare) cannot send warehouse schemas, query results, or dbt project contents to cloud LLM providers — today that rules altimate-code out for them entirely, or forces a hand-rolled Ollama/LM Studio setup with no verification that the endpoint actually works for agent workloads (tool calls, long-context prefill, reasoning renders).

Proposal: a managed local mode — altimate local — that makes the local path a first-class, verified experience:

  • one command: hardware detection → pinned, SHA-256-verified model + runtime download → server lifecycle on 127.0.0.1certification probes (tool-call round trip, reasoning render, 8K prefill) that must pass before any config is touched
  • a model registry (multi-model by design; --model <id>, altimate local models)
  • an egress guard: wiring local mode adds ask permission rules for the web tools (websearch/webfetch/codesearch) so a local-first session reaches the internet only with per-step approval; reversible, never overwrites user-set permissions
  • context diet for small-context local models (skill-listing compaction under the existing ALTIMATE_TOOL_RETRIEVAL flag)
  • first-run TUI picker row + docs integration so the option is discoverable

Out of scope for the first cut (tracked as follow-ups): Linux AMD/Intel GPU auto-detection (runtime already works via Vulkan), native-Windows certification, live prefill progress in the TUI.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions