From 8ef7b89dc2f4e8526788b9cbcfd035c62e6c50da Mon Sep 17 00:00:00 2001 From: roshantaneja Date: Tue, 28 Jul 2026 18:05:37 -0700 Subject: [PATCH 1/2] docs: correct four claims that contradict the code or the rest of the docs site/index.html contradicted itself on the same page: the Distribution paragraph said "ad-hoc signed ... not yet notarized" while the install steps directly below, the README, and site/downloads/latest.json all say Developer ID signed and notarized by Apple. Three sources agree, so the paragraph was the stale one. SETUP.md documented `unittest discover` as the test command. CI deliberately overrides that with pytest, because backend/tests/conftest.py -- which only pytest loads -- pins CORTEX_DB_PATH/CORTEX_VAULT_PATH to a throwaway temp dir at import time. Under `unittest discover` the suite binds to the default backend/data/Cortex.vault instead, persists across runs, and writes into the working tree. Matches SETUP.md to CI and says why. The CI comment justifying that choice claimed conftest.py "provides fixtures". It declares none; the import-time env isolation is the whole job, and it matters because backend.app.main builds a process-singleton store on first import. Corrected so the next reader doesn't inherit the wrong reason. docs/ARCHITECTURE.md hand-copied the MCP tool list and had drifted to 42 stale entries against 110 in TOOLS -- query_memory and expand, which docs/CMP_PROTOCOL.md references, were both absent. Replaced with the core ten, the group structure, and a pointer to mcp_tools.py as source of truth, rather than re-copying a list that will drift again. Also documents the tool-surface mechanism (Cursor caps at 40, ChatGPT at 128) and that surface is advertisement only, never authorization. Every tool name cited was verified present in mcp_tools.py. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01M12nsqf2Mw9cvEBeRiDBv1 --- .github/workflows/ci.yml | 10 +++-- SETUP.md | 7 +++- docs/ARCHITECTURE.md | 84 ++++++++++++++++++++-------------------- site/index.html | 2 +- 4 files changed, 54 insertions(+), 49 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 7fab48f0..f23fa164 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -48,9 +48,13 @@ jobs: ui.py - name: Run backend unit tests - # pytest is the canonical runner: conftest.py provides fixtures + temp-dir - # isolation that `unittest discover` bypasses, and some tests/subtests only - # behave correctly under pytest. + # pytest is the canonical runner. The tests are all unittest.TestCase (no file + # imports pytest), but conftest.py — which only pytest loads — pins + # CORTEX_DB_PATH/CORTEX_VAULT_PATH to a throwaway temp dir at import time. It + # declares no fixtures; that env isolation is the whole job, and it matters + # because backend.app.main builds a process-singleton store on first import. + # `unittest discover` bypasses it and the suite then binds to the default + # backend/data/Cortex.vault, persisting across runs and writing into the tree. run: python -m pytest backend/tests -q - name: Run retrieval quality regression gate diff --git a/SETUP.md b/SETUP.md index 751cc8ca..b8af6f6b 100644 --- a/SETUP.md +++ b/SETUP.md @@ -56,10 +56,13 @@ For backend development without opening the app: ./scripts/dev_backend.sh ``` -Run the main verification set: +Run the main verification set. Use `pytest` rather than `unittest discover`: the tests are +`unittest.TestCase` classes, but `backend/tests/conftest.py` pins the vault/DB to a throwaway +temp directory at import time, and only pytest loads it. Under `unittest discover` the suite +binds to the default `backend/data/Cortex.vault` and writes into the working tree. ```bash -python3 -W error::ResourceWarning -m unittest discover backend/tests +python3 -m pytest backend/tests -q python3 scripts/retrieval_eval.py python3 scripts/adaptation_eval.py ./macos/build.sh diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 0bb04884..b7f85b32 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -258,50 +258,48 @@ Operational readiness adds a sanitized support bundle that omits captured text, ## MCP Tools -The backend exposes an MCP-style JSON-RPC endpoint with these tools: - -- `remember_this` +The backend exposes an MCP-style JSON-RPC endpoint at `POST /mcp`. The tool registry is +`TOOLS` in `backend/app/mcp_tools.py` and is the source of truth — it currently holds 110 +tools, so this document deliberately does not mirror the full list (a hand-copied list here +drifted to 42 stale entries before). + +Clients cap how many tools they will accept (Cursor 40, ChatGPT 128), so tools are advertised +by *surface* — `MCP_TOOL_SURFACES` in the same module, with presets `core`, `coding`, +`chatgpt`, and `full`. The surface rides on the token's label. It is advertisement only and +never authorization: scopes (`read`, `write`, `export`, `maintenance`, `destructive`) plus the +user's Trust policy decide what a call is allowed to do, whatever the surface advertises. + +The `core` surface is the default ten: + +- `use_cortex` +- `get_context` +- `ask_memory` +- `query_memory` +- `expand` - `search_memory` -- `get_recent_context` -- `get_memory_graph` -- `get_decisions` -- `get_open_questions` -- `get_daily_review` -- `get_product_loop` -- `get_style_profile` -- `get_project_context` -- `get_procedure` -- `list_supported_import_sources` -- `list_source_connectors` -- `build_context_pack` -- `get_about_person` -- `get_about_entity` -- `list_memory_topics` -- `list_memory_entities` -- `get_memory_stats` -- `get_memory_inbox` -- `get_memory_diagnostics` -- `get_reliability_report` -- `get_support_bundle` -- `get_trust_summary` -- `get_audit_log` -- `get_personal_profile` -- `get_agent_adaptation` -- `connect_source_account` -- `sync_source_records` -- `sync_connected_sources` -- `approve_memory_capture` -- `archive_memory_capture` -- `create_memory_backup` -- `restore_latest_memory_backup` -- `delete_memory_backups` -- `delete_all_user_data` -- `repair_memory_storage` -- `rebuild_memory_search` -- `forget_memory` -- `delete_memory_capture` -- `rebuild_index_from_vault` -- `export_memory` +- `get_entity_context` +- `get_person_map` +- `remember_this` +- `list_capabilities` + +Beyond core, the registry groups roughly into: review/write (`propose_memory`, +`approve_memory_capture`, `archive_memory_capture`, `forget_memory`), profile and adaptation +(`get_personal_profile`, `get_agent_adaptation`, `get_style_profile`, `get_project_context`), +connectors (`list_source_connectors`, `connect_source_account`, `sync_connected_sources` plus +one `sync_*` per connector), sessions and context packs (`start_agent_session`, +`build_context_pack`, `verify_context_pack`), provenance and audit (`get_decision_history`, +`get_belief_timeline`, `verify_belief_proof`, `get_audit_log`, `get_trust_summary`), shared +memory, and portability (`export_memory_bundle`, `verify_memory_bundle`, +`create_memory_backup`). + +Two tools are named `search` and `fetch` purely because the ChatGPT deep-research connector +requires those exact names; they are kept out of `core` so the default Claude/Cursor surface +is unchanged. + +Desktop clients do not speak to `/mcp` directly. They launch the bundled stdio proxy +`scripts/cortex_mcp_stdio.py`, which pipes stdio JSON-RPC to `http://127.0.0.1:8766/mcp` with +a `cxm_` bearer token. Root `mcp_server.py` is the legacy prototype and is not used by +packaged builds. The local source-account sync endpoint is enough for beta local app integrations, MCP bridges, and connector processes. Production ChatGPT/Claude and cloud-service connectors should add full remote MCP/OAuth flows on top of the same account, cursor, citation, and review contracts. diff --git a/site/index.html b/site/index.html index d45a63d7..6977c036 100644 --- a/site/index.html +++ b/site/index.html @@ -159,7 +159,7 @@

Reliability report

Distribution

Download the local macOS beta.

- This build is for early testers who want local AI memory today. It is ad-hoc signed for local beta testing and not yet notarized for broad public distribution. + This build is for early testers who want local AI memory today. It is Developer ID signed and notarized by Apple, so it opens with a normal double-click. For support or deletion questions, email support@trydoppl.com.

From 0b31fe2ab6ae2a94ac9cb5855fba4d4772bc1ca4 Mon Sep 17 00:00:00 2001 From: roshantaneja Date: Tue, 28 Jul 2026 18:05:54 -0700 Subject: [PATCH 2/2] docs: add CLAUDE.md with the architecture facts that need multiple files to learn Onboarding guidance for Claude Code sessions, focused on what is not discoverable from any single file: the two-server split and its parity footgun, vault-as-truth vs SQLite-as-disposable-index, the review approval gate, the CMP/SMP invariants, and the hosted control plane that is built but dormant in the local product. The load-bearing one: semantic near-dup, rerank, MMR and entity boost are all gated on provider != "hash", and hash is exactly what CI runs, since backend/requirements.txt does not install model2vec. A green CI run proves the deterministic skeleton, not the semantic system -- rerank_eval.py exits 0 with status=skipped when the model is absent, and only release packaging enforces it via CORTEX_REQUIRE_MODEL=1. Also records the two cross-cutting conventions that surprise people: the Python tests that assert on the text of Swift/HTML/CSS sources, and the docs-currency phrase gate. Drop this commit if the repo would rather not carry an agent-guidance file. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01M12nsqf2Mw9cvEBeRiDBv1 --- CLAUDE.md | 294 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 294 insertions(+) create mode 100644 CLAUDE.md diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 00000000..862d55a9 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,294 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## What this is + +Cortex (by Doppl) is a **local-first personal memory / "operating model" for AI tools**. A native SwiftUI +macOS app bundles a Python engine on `127.0.0.1:8766`. Notes and AI-chat exports are ingested into typed, +layered, cited memory stored in a user-owned Markdown/JSON **vault** with a rebuildable **SQLite** index +(FTS5 + `sqlite-vec`). AI clients (Claude Desktop, Cursor, Zed, ChatGPT/Claude web via the browser +extension) retrieve approved memory over **MCP**. + +## Commands + +### Backend + +```bash +# Dev server (FastAPI + uvicorn, reload, dev token) +./scripts/dev_backend.sh # 127.0.0.1:8766, CORTEX_ALLOW_INSECURE_DEV_TOKEN=1 + +# Test suite — pytest is the CANONICAL runner, but the tests are all unittest.TestCase +# (0 of 165 files import pytest). conftest.py has NO fixtures: it is import-time env +# isolation only, because backend.app.main builds a process-singleton store on first +# import. `unittest discover` bypasses that and tests then write a real default vault. +python -m pytest backend/tests -q +python -m pytest backend/tests/test_context_engine.py -q # one file +python -m pytest backend/tests/test_context_engine.py::ClassName::test_name -q # one test +python -m pytest backend/tests -q -k "retrieval and not scale" # by expression +``` + +Install deps: `pip install -r backend/requirements.txt` (dev/hosted plane) — `backend/runtime-requirements.txt` +is the *packaged-app* runtime set (`pysqlite3`, `sqlite-vec`, `model2vec`). Root `requirements.txt` belongs to +the legacy prototype scripts only. + +There is **no linter, formatter, or type checker** in this repo — no ruff/black/eslint/prettier/mypy. CI's only +static Python check is `python -m compileall -q backend scripts capture.py github_store.py ingest.py +instrumentation.py mcp_server.py redis_store.py ui.py`. Match surrounding style by hand. + +`SETUP.md` still documents `python3 -m unittest discover backend/tests`; CI deliberately overrides that with +pytest because `unittest discover` bypasses `conftest.py`'s temp-dir isolation. **Use pytest.** + +### Quality gates (all of these block CI — run before claiming done) + +```bash +python scripts/retrieval_eval.py # retrieval-quality regression gate +python scripts/tool_routing_eval.py # MCP tool-routing gate +python scripts/adapter_contract_eval.py # universal adapter contract +python scripts/context_pack_eval.py # context-pack quality +python scripts/token_calibration_eval.py # per-model token calibration +python scripts/agent_task_eval.py --deterministic +python scripts/delivery_eval.py # delivery/push safety +python scripts/adaptation_eval.py # adaptation quality +python scripts/rerank_eval.py # model2vec lift; SKIPS when model not provisioned +python scripts/check_connector_baseline.py # connector baseline contract +python scripts/check_docs_current.py # docs-currency phrase gate (see below) +python scripts/backend_beta_smoke.py +python scripts/ops_readiness_check.py --skip-tests --skip-build +python scripts/appstore_compliance_lint.py # App Store 2.5.1/2.5.2/4.8 + privacy/entitlements +python scripts/check_distribution_site.py +python scripts/validate_update_manifest.py site/downloads/latest.json +bandit -c .bandit.yaml -r backend scripts -ll # fails on MEDIUM+ +pip-audit --strict -r backend/requirements.txt +``` + +### macOS app + +```bash +./macos/build.sh # -> macos/build/Cortex.app +codesign --verify --deep --strict --verbose=2 macos/build/Cortex.app +open macos/build/Cortex.app + +# CI/dev-mode build (no bundled interpreter, ad-hoc signature): +CORTEX_BUNDLE_PYTHON=0 CORTEX_CODESIGN_IDENTITY=- CORTEX_CODESIGN_TIMESTAMP=0 ./macos/build.sh +``` + +`build.sh` invokes **`swiftc` directly** on `macos/Sources/*.swift` and hand-assembles the `.app` bundle — +there is no Xcode project and no SwiftPM manifest. Target is `arm64-apple-macosx13.0`. + +### Obsidian plugin + +```bash +scripts/check_obsidian_plugin.sh # typecheck + build (Node 22) +``` + +## Architecture + +### Two HTTP server implementations — keep them in sync + +This is the single most important structural fact in the repo. + +- **`backend/app/standalone_server.py`** — a pure-stdlib `http.server`/`ThreadingHTTPServer` implementation. + **This is what ships.** The macOS app launches `python3 -S -s -m app.standalone_server --host 127.0.0.1 + --port 8766` (`macos/Sources/CortexApp.swift:2802`) against a bundled interpreter that has **no + FastAPI/uvicorn/pydantic**. The smoke/verify/bench scripts boot this one too (`verify_full_pipeline.py`, + `verify_pipeline_live.py`, `live_product_smoke.py`, `e2e_external_connections.py`, `cmp_demo.py`, + `demo_seed.py`, `bundle_bench.py`, `cmp_scale_bench.py`). The `*_eval.py` gates do **not** — they exercise + `CortexStore` in-process, so they will not catch a route wired into only one server. +- **`backend/app/main.py`** — the FastAPI app (~209 `@app.*` routes). This is the dev server + (`scripts/dev_backend.sh`) and the hosted/multi-tenant plane (`scripts/load_test_10k.py` runs it under + uvicorn in sharded mode). + +Both dispatch into the same `CortexStore` and the same `mcp_tools` registry, but they are **not route-identical +in either direction** — verified: + +- Only in `main.py`: the whole accounts / OAuth-login / billing / admin plane (`/v1/auth/*`, `/v1/admin/*`, + `/v1/billing/webhook`). The shipping Mac app never serves `/v1/auth/signup`. +- Only in `standalone_server.py`: `/v1/context-file/preview`, `/v1/context-file/sync`, `/v1/delivery/preview`, + `/v1/delivery/send`, `/v1/imports/detect`, `/v1/pair`. + +So "edit both files" is the default for shared product endpoints, but decide deliberately which plane a new +route belongs to. ⚠️ `test_fastapi_contract.py` and `test_standalone_server.py` each test their own surface; +**nothing in the suite diffs the two route sets**, and the `*_eval.py` gates run in-process. A route wired into +only one server ships green. Check by hand. + +### Storage: vault is truth, SQLite is a cache + +- **`backend/app/storage.py`** (~33k lines) is the monolith holding `CortexStore` — nearly all business + logic lives here. Expect to `grep` rather than read it. +- The **local vault** (`~/Library/Application Support/Cortex/Cortex.vault/`, or `CORTEX_VAULT_PATH`) holds + human-readable JSON/Markdown records: `captures/`, `memories/`, `tasks/`, `entities/`, `graph_edges/`, + `imports/`, `source_accounts/`, `sync_cursors/`, plus `manifest.json`, `settings.json`, `events.jsonl`, + `backups/`. +- `index.sqlite` (FTS5 + `sqlite-vec`) is **derived and disposable**. `POST /v1/maintenance/rebuild-index-from-vault` + reconstructs it. Never let index-only state become authoritative — if you write a record, write the vault + record too (`vault.py`, `vault_markdown.py`, `mirror.py`). +- `sharding.py` (`StoreRegistry`) maps users to shard DB/vault paths; `shard_mode="local"` is the single-user + packaged case, other modes are the hosted path. + +### Memory model + +Captures have a lifecycle `pending → approved | archived` (plus hard delete). There is **no `rejected` +state** — archive is the reject. **Nothing reaches AI tools until the user approves it in the Review inbox**; +this gate is a product invariant, and `CORTEX_AUTO_APPROVE_CAPTURES` is deliberately **overridden by a +connected source's own review policy**, so connector trust can't be bypassed by a global flag. +`backend/app/models.py:8-9` +is the authoritative taxonomy: `MemoryKind` = claim, decision, event, preference, observation, action, question, +summary, style, negative, procedure; `MemoryLayer` = semantic, episodic, style, decision, preference, negative, +procedural. (The README's "seven layers — voice, preferences, decisions, facts, episodic, entities, topics" is +marketing vocabulary, not these identifiers; entities/topics are join tables, not layers.) Entities +(person/project/org/topic) plus edges form the knowledge graph used by `graph_analysis.py` — bounded +strongest-path walk, plus a `personalized_pagerank` mode. + +`docs/ARCHITECTURE.md` maps the endpoint surface and vault layout; `docs/CMP_PROTOCOL.md` is the CMP spec +and names its own source-of-truth files. Note that `ARCHITECTURE.md`'s MCP tool list is already stale — +`query_memory` and `expand` exist in `mcp_tools.py:1037,1069` but are absent from it. Trust the code. + +### Ingestion and connectors + +`source_ingest.py` + `backend/app/connectors/` (obsidian, notion, gmail, google_drive, calendar, slack, jira, +linear, github, outlook, raindrop, readwise, zotero, agent_sessions). Every connector must satisfy the +baseline contract enforced by `scripts/check_connector_baseline.py` and the `test_connector_*` tests +(cursor/pagination data-loss, redaction via `connectors/_redaction.py`, SSRF guards). + +Extraction: `extractor.py` runs deterministic extractors; `condense.py` adds an LLM pass when +`ANTHROPIC_API_KEY` is set. **Without the key the backend must still work** via the deterministic extractor — +don't introduce hard LLM dependencies in the ingest path. + +### Retrieval and CMP + +`query_plan.py` builds the plan (intent classification, entity extraction, decomposition) behind flags +`query_plan_enabled()`, `context_hop_enabled()`, `entity_boost_enabled()`, `second_hop_enabled()` with +millisecond deadlines. `CortexStore.search` / `_vector_search` fuse BM25 + `sqlite-vec` KNN + temporal + intent. + +The context engine is `CortexStore.assemble_context` → `_pack_context_knapsack` → `_compute_working_memory_delta`, +projected onto the wire by `smp.py` (`build_smp_envelope`, `SMP_LEGEND`). Key invariants, stated in the +`smp.py` docstring and enforced upstream: **cited-only, superseded-never, global-dedup, provenance**. SMP is a +*pure projection* — it must invent, re-rank, and drop nothing. Answers **cite or abstain**; returning uncited +text is a regression the eval gates catch. + +Consolidation ("dreaming") is real: tables `memory_consolidation_runs`, `memory_consolidation_decisions`, +`hot_context_packs`. The pass detects conflicts, auto-resolves only those clearing deterministic rules +(everything else → `conflicts_review_required`), commits SQLite mutations atomically, then patches the vault +*outside* the transaction and reports mirror failures as warnings rather than faking a rollback. It then +pre-warms hot context packs and verifies each one round-trips before counting it. + +### MCP + +`backend/app/mcp_tools.py` is a **hand-written JSON-RPC handler, not the `mcp` SDK**. `TOOLS` holds **110** +tools, exposed at `POST /mcp` (streamable-HTTP compatible: `Accept: text/event-stream` gets SSE + +`Mcp-Session-Id`; JSON clients get byte-identical `application/json`). Desktop clients connect through the +bundled stdio proxy `scripts/cortex_mcp_stdio.py`. Root `mcp_server.py` is the **legacy prototype**. + +Clients cap tool counts (Cursor 40, ChatGPT 128), so `MCP_TOOL_SURFACES` (`mcp_tools.py:1428`) advertises +subsets — `core` (10), `coding`, `chatgpt`, `full` — selected by the token's label. **Surface is advertisement +only, never authorization**; scopes do the enforcing. The oddly-named `search` / `fetch` tools exist solely +because the ChatGPT deep-research connector requires those exact names. + +Legacy prototype files kept for reference only: `capture.py`, `ingest.py`, `github_store.py`, `redis_store.py`, +`ui.py`. They still must compile (CI runs `compileall` over them). + +### The universal adapter surface + +`GET /v1/tools/schema?format=openai|anthropic`, `POST /v1/tools/call`, and `POST /v1/context` are the +non-MCP integration surface. Both SDKs (`sdk/python` → `doppl-cortex-client`, `sdk/typescript` → +`@doppl-tech/cortex-client`, each zero-dependency) and the browser extension talk to exactly these. +`scripts/adapter_contract_eval.py` gates them. + +The browser extension (`extension/`, MV3, vanilla JS, no build step) is **inject-only, not capture**: it reads +what you typed in ChatGPT/Claude/Notion, asks the local server for cited context, and prepends it to the +prompt. Its auth token is minted by the macOS app via `POST /v1/pair` (`cxm_` prefix). + +### Out-of-tree consumers + +`packages/obsidian-cortex-plugin` is CI-gated and asserts the committed `main.js` matches a fresh build. +`packages/openclaw-cortex-context` implements the portable-memory spec in TypeScript — meaning +`spec/portable-memory/v2/schema.json` (Ed25519-signed, hash-chained export bundles) has **two independent +implementations** and conformance vectors in `spec/portable-memory/v2/test-vectors/`. Changing the format +means changing both sides plus regenerating vectors via `scripts/generate_portable_memory_vectors.py`. +Neither the extension, the SDKs, nor the OpenClaw package run in CI. + +### Hosted control plane (accounts, auth, encryption, billing) + +This layer is fully built and **dormant in the local product** — none of it runs in the packaged app. + +- `accounts.py` — `ControlStore` / `SQLiteControlStore` in a separate control-plane DB + (`/control/accounts.sqlite`). Load-bearing invariant: `user_id` is the immutable non-PII + shard-routing key and account merge is **never** an `UPDATE` of `user_id`. +- `authn.py` — argon2id (scrypt fallback), opaque `cxs_`/`cxr_` tokens, **no JWTs**, refresh rotation with + family revocation on reuse. +- `keyring.py` — envelope encryption: root KEK → per-user master DEK → HKDF purpose subkeys. The **CXE1** + ciphertext format (`b"CXE1" || kek_id || dek_version || nonce || AES-256-GCM`) is AAD-bound to + `user_id || purpose || dek_version`, so a blob copied to another user or purpose fails authentication. + `scripts/cortex_decrypt.py` is the offline decryptor. +- `billing.py` — Paddle + Stripe webhooks, **config-gated**: with no provider/secret the route 404s and a + no-billing deployment is byte-identical. The site advertises free beta only. +- `oauth_broker.py` — hosted-only secret holder for providers without a PKCE public-client path (Notion, + GitHub). Posture: *the auth handshake transits the server, the data does not* — the broker never persists + tokens, and redirect URIs are allowlisted to loopback callbacks. + +`accounts.py`, `authn.py`, and `keyring.py` are deliberately **framework-free stdlib** so the same code can +run inside the bundled interpreter. Don't import fastapi/pydantic into them. + +Tables with no product surface yet: `working_canvas_nodes`, `shared_memory_principals`, +`shared_memory_nonces`, `shared_memory_writes` — currently only counted in diagnostics/isolation checks. + +### Hosted deployment + +`deploy/` is a working kit, not scaffolding: `deploy/push.sh` → `bootstrap.sh` provisions a single Ubuntu VM +(ufw, fail2ban, Caddy TLS reverse-proxying `127.0.0.1:8766`) with systemd units `cortex-api`, `cortex-worker`, +`cortex-backup.timer`; `deploy/macmini/` is the launchd + Cloudflare Tunnel alternative. Hosted mode is the +same FastAPI app with `CORTEX_SHARD_MODE=bucket`, scoped API tokens, a KEK file, and rate limits +(`deploy/cortex.env.example`). `status/` is explicitly a scaffold — Upptime runs in its own repo. + +### macOS client + +`macos/Sources/CortexApp.swift` (~14k lines) holds app state, the backend supervisor, and the custom tab bar. +Three primary tabs — **Model** (`ModelTab.swift`), **Review** (`ReviewTab.swift`), **Ask** (`AskTab.swift`) — +plus the `ConnectionsPrivacySheet.swift` surface for sources, trust, MCP tool setup, backups, and diagnostics. +`OnboardingView.swift` drives first run. Ancillary surfaces: `MenuBarQuickPanel`, `QuickCapture`, +`NotchNotifier`, `LiveActivityPill`, `ConstellationOverlay`, `MemoryWrapped`, `MemoryMapView`, `ImportDiffView`, +`TwinAlertsViews`. `CortexDesign.swift` / `CortexNorthStar.swift` hold the design tokens and product principles. + +### Cross-language contract tests + +Several Python tests assert on the **text of Swift/HTML/CSS source files** — e.g. +`backend/tests/test_macos_ui_quality_contract.py` asserts literal strings like `hosting.sizingOptions = []` +exist in `CortexApp.swift`. Editing a Swift view, the extension, or `site/styles.css` can break `pytest`. +Same family: `test_macos_connector_ui_contract.py`, `test_macos_constellation_contract.py`, +`test_macos_phase_ui_contract.py`, `test_macos_external_connections_contract.py`, +`test_onboarding_sources_contract.py`. + +### Docs-currency gate + +`scripts/check_docs_current.py` fails CI when a doc contains a phrase from its per-file `FORBIDDEN_PHRASES` +list (stale product vocabulary, removed flows, retired tabs). When you change product behaviour, update the +docs *and* that script's expectations together. + +## Conventions that matter + +- **Embeddings — read this before trusting a green CI run.** Default provider is offline `hash` + (deterministic, no network). Semantic near-dup detection, reranking, MMR, and entity boost are all gated on + `provider != "hash"` and **no-op under it** — and `hash` is exactly what CI runs, since + `backend/requirements.txt` doesn't install `model2vec`. **CI proves the deterministic skeleton, not the + semantic system.** `scripts/rerank_eval.py` is the gate for semantic lift and it exits 0 with + `status: skipped` when the model is absent; only release packaging enforces it, via + `macos/package_release.sh` setting `CORTEX_REQUIRE_MODEL=1` plus + `check_vector_runtime.py --require-model2vec`. Test semantic changes with the model provisioned. + Keep `CORTEX_EMBEDDING_DIMENSIONS=384` (the `sqlite-vec` schema depends on it). +- **Source protection.** `build.sh` compiles the bundled backend to `.pyc` and deletes the `.py` files + (`CORTEX_SKIP_PYC=1` to opt out for local debugging). +- **Auth.** REST requires `CORTEX_API_KEY` (Bearer). `/mcp` also accepts a scoped `CORTEX_MCP_API_KEY` with + `CORTEX_MCP_API_KEY_SCOPES`. `CORTEX_ALLOW_INSECURE_DEV_TOKEN=1` is dev-only. +- **Privacy invariants.** No ambient capture (no screen, no mic). `GET /v1/sync/changes` is deliberately + **content-free** — event metadata only, never capture/memory text. Redaction is on by default in context + packs, exports, and agent payloads. Treat these as load-bearing; tests enforce them. +- **Bandit.** `.bandit.yaml` skips exactly three IDs, each with a written justification (B608 — SQL f-strings + interpolate only hardcoded predicates, values are `?`-bound; B310 — urlopen against validated endpoints; + B108 — tmp dirs in fixtures only) and an explicit *"Do NOT add blanket skips"* instruction. Fix the finding, + don't widen the skip list. +- **Release.** `CFBundleShortVersionString` + `CFBundleVersion` in `macos/Info.plist` (currently `0.2.0` / `52`); + each release bumps the build and regenerates `site/downloads/latest.json` (SHA-256 pinned artifacts) via + `macos/package_release.sh` / `scripts/prepare_distribution_site.py`. Commits follow + `release: bump build N -> N+1 (...)`. +- Local-only development needs no Redis, no Docker, no GitHub token, and no hosted account.