feat: LocalModel skill — privacy-preserving local LLM classifier

skills/LocalModel/SKILL.md

@@ -0,0 +1,70 @@
+---
+name: LocalModel
+version: 0.1.0
+description: "Run prompts against a local LLM backend (LM Studio, Ollama, or MLX) with privacy-preserving output guardrails — drop reasoning_content, enforce structured output schemas, and post-redact identifiers. USE WHEN sending sensitive content to a model, classifying private data on-machine, scanning dotfiles or configs for leaks via local LLM, content audit that must not touch cloud APIs."
+sources:
+    - https://lmstudio.ai/docs/local-server
+    - https://github.com/ollama/ollama/blob/main/docs/api.md
+    - https://github.com/ml-explore/mlx-examples/tree/main/llms/mlx_lm
+---
+
+# LocalModel
+
+Invoke a locally hosted LLM (LM Studio, Ollama, or MLX) with structured output and three layers of leak prevention. All three backends speak the OpenAI-compatible `POST /v1/chat/completions` interface; the skill provides a single wrapper that probes for whichever is running and applies the same safety rails to every backend.
+
+The skill exists because **chain-of-thought reasoning models leak in `reasoning_content`** even when the prompt forbids reproducing values. Nemotron, DeepSeek-R1, QwQ, and similar models emit their working notes through a separate response field that is not subject to the same constraints as the final answer. Naive wrappers that fall back to `reasoning_content` when `content` is empty (e.g. when the token budget is exhausted mid-thought) print raw secrets directly to logs.
+
+## Workflow Routing
+
+| Trigger                                                                  | Companion / script                                         |
+| ------------------------------------------------------------------------ | ---------------------------------------------------------- |
+| User asks to scan, classify, or audit sensitive content via a local LLM  | [scripts/classify.py](scripts/classify.py)                 |
+| User asks which local backend is running, or wants to probe availability | [scripts/detect-backend.sh](scripts/detect-backend.sh)     |
+| User asks for raw inference (no privacy constraints)                     | Use `classify.py --no-redact --raw` flag                   |
+
+## Backends
+
+| Backend   | Default port | Auth                          | Endpoint                  |
+| --------- | -----------: | ----------------------------- | ------------------------- |
+| LM Studio |        1234  | Bearer token (`LMSTUDIO_API_KEY` in `.env`) | `/v1/chat/completions`    |
+| Ollama    |       11434  | none                          | `/v1/chat/completions` (OpenAI-compat) or `/api/chat` (native) |
+| MLX       |        8080  | none (default `mlx_lm.server`)| `/v1/chat/completions`    |
+
+`scripts/detect-backend.sh` probes each port in that order and returns the first responding backend's identifier on stdout.
+
+## Three layers of leak prevention
+
+**Layer 1 — prompt-side classification, not reproduction.** The system prompt asks the model for *categories* + counts, never raw values. Output schema is enforced: a markdown table with `Category | Count | Risk`. Free-text answers are rejected by the wrapper.
+
+**Layer 2 — drop `reasoning_content` entirely.** The wrapper reads only `choices[0].message.content`. If `content` is empty (token budget exhausted before the final answer emitted), the wrapper reports `[scan failed: model produced no final content]` and discards `reasoning_content` unread. Falling back to reasoning is forbidden because chain-of-thought emits raw values mid-thought regardless of prompt constraints.
+
+**Layer 3 — post-redaction regex sweep on the model's output.** Even with the first two layers, the wrapper runs a final pass that masks:
+
+- IPv4 literals: `\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b` → `<IP>`
+- IPv6 literals: `\b([0-9a-fA-F]{1,4}:){2,7}[0-9a-fA-F]{1,4}\b` → `<IPV6>`
+- Email-like: `\b[\w.+-]+@[\w-]+\.[\w.-]+\b` → `<EMAIL>`
+- FQDN-like (any TLD ≥ 2 chars): `\b[a-z0-9][\w-]*\.[a-z]{2,}(?:\.[a-z]{2,})*\b` → `<HOST>`
+- Short identifiers with at least one digit (a common machine-name pattern): `\b[a-z]{2,4}-?\d+[a-z0-9-]*\b` → `<ID>`. The leading-letter cap and required digit avoid over-matching English hyphenated words like `keep-private`.
+
+## When NOT to use this skill
+
+Structural audits (counting how many `Host`, `HostName`, `IdentityFile`, `ProxyJump` directives exist in an SSH config) are deterministic grep work. The LLM adds no signal and creates leak risk. Prefer:
+
+```sh
+for directive in Host HostName User Port IdentityFile ProxyJump ProxyCommand Match Include; do
+    count=$(grep -ciE "^\s*${directive}\s" "$file")
+    [ "$count" -gt 0 ] && printf "  %-16s  %s\n" "$directive" "$count"
+done
+```
+
+The LLM earns its place when the question is semantic ("does this file expose infrastructure topology?", "is this comment hint a leak vector?") and the answer can be expressed as a category + count.
+
+## Constraints
+
+- **Never read `reasoning_content`** from the response. Treat it as if the field does not exist. If `content` is empty, the scan failed; do not patch with reasoning.
+- **Never echo raw model output to chat** before running it through the Layer 3 redaction pass.
+- **Always set a tight `max_tokens` budget** when the goal is a fixed-schema reply (e.g. 200 for a category table). High budgets give reasoning models more room to leak.
+- **Reject free-text responses.** If the model output does not match the structured schema declared in the prompt, treat it as a failed scan and do not display the raw text.
+- **Auth via env var, not flag.** `LMSTUDIO_API_KEY` reads from `~/.env` or the process environment. Never pass keys as positional arguments visible in `ps aux`.
+- **Backend detection is deterministic.** `detect-backend.sh` probes ports in a fixed order (LM Studio 1234, Ollama 11434, MLX 8080) and stops at the first responder. No DNS, no env-var fallback that could be spoofed.
+- **Tiny models may emit nothing useful.** Reasoning models with ≤ 8B parameters often spend the entire token budget on chain-of-thought and produce empty `content`. Prefer non-reasoning models (e.g. `qwen2.5-7b-instruct`, `gpt-oss-20b`) for classification tasks.

skills/LocalModel/scripts/classify.py

@@ -0,0 +1,194 @@
+#!/usr/bin/env python3
+"""Send a file or stdin to a local LLM for classification with three layers
+of leak prevention. See sibling SKILL.md for the safety rationale.
+
+Layer 1: prompt-side schema (categories + counts only, never raw values)
+Layer 2: drop reasoning_content; if content is empty, report failed scan
+Layer 3: regex post-redaction on the model's output
+
+Usage:
+    classify.py <file>                        # default classification prompt
+    classify.py --prompt PROMPT <file>        # custom system prompt
+    classify.py --model MODEL <file>          # override auto-selected model
+    classify.py --backend lmstudio|ollama|mlx <file>   # override auto-detect
+    classify.py --raw <file>                  # skip Layer 3 redaction (caller asserts the prompt is safe)
+    cat file | classify.py                    # read from stdin
+"""
+
+import argparse
+import json
+import os
+import pathlib
+import re
+import subprocess
+import sys
+import urllib.request
+import urllib.error
+
+
+BACKENDS = {
+    "lmstudio": {"url": "http://localhost:1234/v1/chat/completions", "auth_env": "LMSTUDIO_API_KEY"},
+    "ollama":   {"url": "http://localhost:11434/v1/chat/completions", "auth_env": None},
+    "mlx":      {"url": "http://localhost:8080/v1/chat/completions",  "auth_env": None},
+}
+
+
+DEFAULT_SYSTEM = """You are a security auditor. Given a file fragment, enumerate the CATEGORIES of sensitive identifiers it contains.
+
+Output rules (MANDATORY):
+- Reply with ONLY a markdown table. No prose before or after.
+- Columns: Category | Count | Risk (low|medium|high)
+- NEVER echo any raw values: no hostnames, no IPs, no usernames, no port numbers, no key fingerprints, no path fragments, no comments.
+- Categories to look for: Host aliases, HostName values, IP addresses, User identifiers, Port numbers, IdentityFile paths, ProxyJump targets, custom commands, comments hinting at topology.
+- Add one final row: `Recommendation | <keep-private|publish-with-mask|encrypt-only> | n/a`.
+"""
+
+
+def load_env_var(name: str) -> str | None:
+    """Read VAR from process env, falling back to ~/.env."""
+    val = os.environ.get(name)
+    if val:
+        return val
+    env_path = pathlib.Path.home() / ".env"
+    if not env_path.exists():
+        return None
+    for line in env_path.read_text().splitlines():
+        line = line.strip()
+        if line.startswith(f"{name}="):
+            return line.split("=", 1)[1].strip().strip('"').strip("'")
+    return None
+
+
+def detect_backend() -> str:
+    """Run the sibling detect-backend.sh and return its stdout."""
+    script = pathlib.Path(__file__).parent / "detect-backend.sh"
+    result = subprocess.run([str(script)], capture_output=True, text=True)
+    if result.returncode != 0:
+        sys.exit("no local backend responding (LM Studio:1234, Ollama:11434, MLX:8080 all down)")
+    return result.stdout.strip()
+
+
+def pick_model(backend: str) -> str:
+    """Return a sensible default model for the backend by querying its /v1/models."""
+    base = BACKENDS[backend]["url"].rsplit("/chat/completions", 1)[0] + "/models"
+    headers = {}
+    if BACKENDS[backend]["auth_env"]:
+        key = load_env_var(BACKENDS[backend]["auth_env"])
+        if key:
+            headers["Authorization"] = f"Bearer {key}"
+    req = urllib.request.Request(base, headers=headers)
+    try:
+        with urllib.request.urlopen(req, timeout=5) as resp:
+            body = json.loads(resp.read())
+    except urllib.error.URLError as exc:
+        sys.exit(f"failed to query {base}: {exc}")
+    models = [m["id"] for m in body.get("data", [])]
+    if not models:
+        sys.exit(f"backend {backend} reports no loaded models")
+    # Prefer non-reasoning models for classification (reasoning models leak in CoT).
+    # Heuristic: ids containing 'instruct', 'chat', or 'gpt-oss' first.
+    for keyword in ("instruct", "chat", "gpt-oss"):
+        for m in models:
+            if keyword in m.lower():
+                return m
+    return models[0]
+
+
+def call_model(backend: str, model: str, system: str, user: str, max_tokens: int = 400) -> str:
+    """POST to the backend's chat completions endpoint. Return content only —
+    NEVER reasoning_content, per Layer 2 of the leak-prevention design."""
+    payload = {
+        "model": model,
+        "messages": [
+            {"role": "system", "content": system},
+            {"role": "user", "content": user},
+        ],
+        "temperature": 0,
+        "max_tokens": max_tokens,
+    }
+    headers = {"Content-Type": "application/json"}
+    if BACKENDS[backend]["auth_env"]:
+        key = load_env_var(BACKENDS[backend]["auth_env"])
+        if key:
+            headers["Authorization"] = f"Bearer {key}"
+    req = urllib.request.Request(
+        BACKENDS[backend]["url"],
+        data=json.dumps(payload).encode(),
+        headers=headers,
+    )
+    try:
+        with urllib.request.urlopen(req, timeout=300) as resp:
+            body = json.loads(resp.read())
+    except urllib.error.HTTPError as exc:
+        detail = exc.read().decode(errors="replace")[:500]
+        sys.exit(f"backend returned {exc.code}: {detail}")
+    except urllib.error.URLError as exc:
+        sys.exit(f"backend unreachable: {exc}")
+    msg = body["choices"][0]["message"]
+    content = msg.get("content") or ""
+    # Layer 2: do NOT fall back to reasoning_content. If content is empty,
+    # the scan failed and reasoning is discarded unread.
+    if not content.strip():
+        return "[scan failed: model produced no final content; reasoning_content discarded unread]"
+    return content
+
+
+REDACT_PATTERNS = [
+    (re.compile(r"\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b"), "<IP>"),
+    (re.compile(r"\b(?:[0-9a-fA-F]{1,4}:){2,7}[0-9a-fA-F]{1,4}\b"), "<IPV6>"),
+    (re.compile(r"\b[\w.+-]+@[\w-]+\.[\w.-]+\b"), "<EMAIL>"),
+    # FQDN: at least one dot, lowercase TLD ≥ 2 chars
+    (re.compile(r"\b[a-z0-9][\w-]*\.[a-z]{2,}(?:\.[a-z]{2,})*\b", re.IGNORECASE), "<HOST>"),
+    # Short machine-name pattern: 2-4 lowercase letters, optional hyphen, then
+    # at least one digit (so hyphenated English words like "keep-private" do
+    # not over-match).
+    (re.compile(r"\b[a-z]{2,4}-?\d+[a-z0-9-]*\b"), "<ID>"),
+]
+
+
+def redact(text: str) -> str:
+    """Layer 3: mask anything that looks like a leaked identifier."""
+    for pattern, replacement in REDACT_PATTERNS:
+        text = pattern.sub(replacement, text)
+    return text
+
+
+def main():
+    parser = argparse.ArgumentParser(description="Local-LLM classifier with leak prevention")
+    parser.add_argument("file", nargs="?", help="file to classify (omit to read stdin)")
+    parser.add_argument("--prompt", help="custom system prompt (default: schema-based categorization)")
+    parser.add_argument("--backend", choices=list(BACKENDS), help="force a backend (default: auto-detect)")
+    parser.add_argument("--model", help="override model id (default: backend's first non-reasoning model)")
+    parser.add_argument("--max-tokens", type=int, default=400)
+    parser.add_argument("--raw", action="store_true", help="skip Layer 3 redaction (caller asserts safety)")
+    args = parser.parse_args()
+
+    if args.file:
+        content = pathlib.Path(args.file).read_text()
+        filename = pathlib.Path(args.file).name
+    else:
+        content = sys.stdin.read()
+        filename = "<stdin>"
+
+    backend = args.backend or detect_backend()
+    model = args.model or pick_model(backend)
+    system = args.prompt or DEFAULT_SYSTEM
+
+    reply = call_model(
+        backend=backend,
+        model=model,
+        system=system,
+        user=f"File: {filename}\n\n```\n{content}\n```",
+        max_tokens=args.max_tokens,
+    )
+    if not args.raw:
+        reply = redact(reply)
+
+    print(f"backend: {backend}")
+    print(f"model:   {model}")
+    print()
+    print(reply)
+
+
+if __name__ == "__main__":
+    main()

skills/LocalModel/scripts/detect-backend.sh

@@ -0,0 +1,31 @@
+#!/usr/bin/env bash
+# Probe local LLM backends in deterministic order. Print the first responder's
+# identifier ("lmstudio" | "ollama" | "mlx") to stdout, exit 0. If none respond,
+# print "none" and exit 2.
+#
+# Usage: detect-backend.sh
+#
+# Authentication: LM Studio requires a Bearer token. The probe sends without
+# auth and accepts HTTP 401 as a "backend reachable" signal (the auth check
+# itself proves the server is up). Ollama and MLX accept no-auth probes.
+
+set -u
+
+probe() {
+    local name="$1" url="$2"
+    local code
+    code=$(curl -sSo /dev/null -w '%{http_code}' --max-time 2 "$url" 2>/dev/null || echo "000")
+    case "$code" in
+        # 200: model list returned; 401: auth required, server up (LM Studio);
+        # 404: endpoint exists at a different path but server up (some Ollama versions).
+        200|401|404) echo "$name"; return 0 ;;
+        *) return 1 ;;
+    esac
+}
+
+probe lmstudio "http://localhost:1234/v1/models" && exit 0
+probe ollama   "http://localhost:11434/v1/models" && exit 0
+probe mlx      "http://localhost:8080/v1/models"  && exit 0
+
+echo "none"
+exit 2