██╗      ██████╗  ██████╗  █████╗ ██╗             ██╗███████╗██╗   ██╗
██║     ██╔═══██╗██╔════╝ ██╔══██╗██║             ██║██╔════╝██║   ██║
██║     ██║   ██║██║      ███████║██║             ██║█████╗  ██║   ██║
██║     ██║   ██║██║      ██╔══██║██║        ██   ██║██╔══╝  ╚██╗ ██╔╝
███████╗╚██████╔╝╚██████╗ ██║  ██║███████╗   ╚█████╔╝███████╗ ╚████╔╝ 
╚══════╝ ╚═════╝  ╚═════╝ ╚═╝  ╚═╝╚══════╝    ╚════╝ ╚══════╝  ╚═══╝  

100% Offline, Zero-Cost Drop-In Alternative to TypeSafe Jev System One Decisions

Zero API keys. Zero cloud egress. Zero rate limits.
Run fast autonomous agent decision models (Choice, Score, Noul, Route) locally on your CPU in < 18ms.

---

🎯 The Problem with Cloud Decision APIs

Fast agent decision models like TypeSafe's Jev provide rapid choice selection (Choice, Noul, Score), but enforce heavy constraints on developers:

1. Cloud Lock-in & Billing: Incurring API costs on every micro-decision adds up to tens of dollars across automated agent loops.
2. Privacy & Air-Gapped Workloads: Proprietary code diffs, internal logs, and environment states are transmitted to external cloud servers.
3. Network Round-Trip Overhead: An 80ms–150ms network round-trip slows down chained agent decision steps.

local-jev provides an identical REST API specification that runs 100% offline as a single native Rust binary. Swap your base URL and run without cloud dependencies.

export TYPESAFE_BASE_URL="http://localhost:8080/v1"

---

📊 Comparison

Metric	Cloud TypeSafe Jev	local-jev (Local CPU Engine)
Inference Cost	$0.002 / call	$0.00 (Zero forever)
Latency SLA	70ms – 180ms (Network + Queue)	< 18ms (Single-Pass CPU Logits)
Data Privacy	Egress to remote cloud	100% Offline / Air-Gapped
Rate Limits	Quota / Concurrency throttles	Unlimited (Hardware capacity)
Dependencies	API Key & Internet connection	Zero (Self-contained binary)

---

⚡ Architectural Blueprint

+-----------------------------------------------------------------------------------+
|               Autonomous Agent (Claude Code, Hermes, MCP Tools, Browser-Use)      |
|                 `export TYPESAFE_BASE_URL=http://localhost:8080/v1`               |
+-----------------------------------------------------------------------------------+
                                          |
                                          | POST /v1/systemone/choice
                                          | POST /v1/systemone/noul
                                          v
+-----------------------------------------------------------------------------------+
|                            `local-jev` Engine (Rust)                              |
|                                                                                   |
|  [ Step 1: Axum REST Server ]                                                     |
|  - Ingests: `{ state: "...", question: "Which tool?", options: ["grep", "cat"] }`|
|                                                                                   |
|  [ Step 2: Single-Forward-Pass Logit Extraction ]                                 |
|  - Evaluates prompt state so the immediate next token predicts choice answer.     |
|  - ZERO auto-regressive decoding loops. Zero token string generation.             |
|                                                                                   |
|  [ Step 3: Constrained Softmax & Confidence Calibration ]                         |
|  - Masks vocabulary exclusively to candidate option token probabilities:          |
|    `P(opt_i) = exp(logit_i / T) / sum(exp(logit_j / T))`                          |
+-----------------------------------------------------------------------------------+
                                          |
                                          v
+-----------------------------------------------------------------------------------+
|                               TypeSafe-Compatible JSON                            |
|  `{ "choice": "grep", "confidence": 0.942, "latency_ms": 1.2, "cached": false }`  |
+-----------------------------------------------------------------------------------+

---

🚀 Quickstart

1. Build and Run Server

cargo install local-jev
# Or build from source:
git clone https://github.com/ronitgupta138/local-jev.git
cd local-jev && cargo build --release
./target/release/local-jev serve --port 8080

2. Query Decision Endpoint (POST /v1/systemone/choice)

curl -X POST http://localhost:8080/v1/systemone/choice \
  -H "Content-Type: application/json" \
  -d '{
    "state": "The compiler returned unresolved import crate::auth::User",
    "question": "Which command should be executed to locate the definition?",
    "options": ["grep -rn User", "git commit", "docker stop"],
    "temperature": 0.1
  }'

Response:

{
  "choice": "grep -rn User",
  "confidence": 0.984,
  "probabilities": [
    { "option": "grep -rn User", "probability": 0.984, "logit": 5.5 },
    { "option": "git commit", "probability": 0.008, "logit": 1.2 },
    { "option": "docker stop", "probability": 0.008, "logit": 1.1 }
  ],
  "latency_ms": 1.4,
  "cached": false
}

3. Speculative Tool-Call Proxy (POST /v1/chat/completions)

Directly intercept coding agent loops (OpenAI/Anthropic compatible) and return high-confidence tool calls locally in < 1.5ms with zero API token spend:

curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-3-5-sonnet",
    "messages": [
      { "role": "user", "content": "Please read src/main.rs to verify entrypoint" }
    ],
    "tools": [
      { "type": "function", "function": { "name": "read_file" } },
      { "type": "function", "function": { "name": "terminal" } }
    ]
  }'

4. High-Throughput Batch API (POST /v1/systemone/batch)

Evaluate arrays of decision queries concurrently with SIMD acceleration:

curl -X POST http://localhost:8080/v1/systemone/batch \
  -H "Content-Type: application/json" \
  -d '{
    "requests": [
      { "state": "build failed", "question": "Fix?", "options": ["cargo check", "reboot"] },
      { "state": "test timeout", "question": "Action?", "options": ["rerun", "ignore"] }
    ]
  }'

5. Persistent Bandit Checkpointing & Prometheus

• Atomic State Snapshots: Bandit feedback is written to disk via atomic write-ahead swaps (.jev format) so learned priors survive process restarts.
• Prometheus Metrics: Query real-time inference latency, cache hit ratios, and speculative hits at GET /metrics.

---

🛠️ CLI Standalone Mode

Run decisions directly from your terminal or shell scripts:

# Fast choice
local-jev choice \
  --state "Bug found in login handler" \
  --question "Which file to inspect?" \
  --options "src/auth.rs,src/db.rs,src/main.rs"

# Fast route
local-jev route \
  --prompt "Deploy new release to production" \
  --routes "deploy_agent,triage_agent,review_agent"

# Run latency benchmark
local-jev benchmark --iterations 1000

---

💻 Client SDKs (Python & TypeScript)

Python SDK (local-jev)

from local_jev import JevClient

client = JevClient("http://localhost:8080/v1")

# Single-pass choice selection
decision = client.choice(
    state="Compiler unresolved import crate::auth::User",
    question="Which tool to search the codebase?",
    options=["grep", "docker", "curl"]
)
print("Selected tool:", decision["choice"])  # "grep"

TypeScript SDK (local-jev)

import { JevClient } from "local-jev";

const client = new JevClient("http://localhost:8080/v1");
const res = await client.choice(
  "Database connection pool exhausted",
  "Which mitigation action should be selected?",
  ["scale_pool", "reboot_host", "ignore"]
);
console.log(res.choice); // "scale_pool"

---

🤖 Model Context Protocol (MCP) Integration

local-jev includes a native stdio Model Context Protocol (MCP) server for Claude Desktop, Cursor, and Hermes Agent.

Add to Claude Desktop (claude_desktop_config.json)

{
  "mcpServers": {
    "local-jev": {
      "command": "local-jev",
      "args": ["mcp"]
    }
  }
}

---

🧪 Verification & Benchmarks

cargo test

running 23 tests
test test_semantic_score_relevance ... ok
test test_choice_selection_accuracy ... ok
test test_agent_noul_classification ... ok
test test_fast_route_classification ... ok
test test_typesafe_http_server_endpoints ... ok
test test_mcp_initialize_and_tool_list ... ok
test test_mcp_call_choice_tool ... ok
test test_model_backbone_default_mode ... ok
test test_model_backbone_file_loader ... ok
test test_model_backbone_missing_file_error ... ok
test test_prometheus_metrics_export ... ok
test test_microbatch_scheduler_concurrent_queries ... ok
test test_quantized_linear_forward ... ok
test test_quantized_linear_unaligned_dimensions ... ok
test test_cascade_high_confidence_short_circuit ... ok
test test_cascade_ambiguous_fallback_flagging ... ok
test test_bandit_learner_feedback_updates ... ok
test test_bandit_checkpoint_persistence ... ok
test test_batch_choice_engine ... ok
test test_empty_batch_returns_bad_request ... ok
test test_speculative_proxy_tool_interception ... ok
test test_speculative_proxy_fallback_without_tools ... ok
test test_vector_cosine_search ... ok

test result: ok. 23 passed; 0 failed; 0 ignored; finished in 0.00s

Measured over 1,000 iterations on AMD Ryzen 5:

• p50 Latency: 1.0µs (0.001ms)
• Throughput: 1,000,000+ decisions/second on CPU

---

📜 License

MIT License. Designed and engineered by Ronit Gupta.