Skip to content

Claude/planreview phase 2 d5b0bc #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tchxm merged 2 commits into from
Sep 19, 2026
Merged

Claude/planreview phase 2 d5b0bc #2

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c8212b6
Phase 2: separate API from frontend, central API client, API contract…
tchxm Sep 19, 2026
c3c2c55
Phase 2 review: add real-process smoke script, OpenAPI test, observed…
tchxm Sep 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions BEGINNER_HTTP_TEST.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,14 @@ This is the easiest way to check that PlanReview works. You are not applying any
In PowerShell, from the `PlanReview` folder, start the server:

```powershell
.\.venv\Scripts\python.exe -m uvicorn engine.api:app --host 127.0.0.1 --port 8000
.\start.ps1
```

Wait for `Uvicorn running on http://127.0.0.1:8000`.
Wait for `Uvicorn running on http://127.0.0.1:8000`. In a second PowerShell window, run `.\start-frontend.ps1` and wait for the Vite `Local:` line.

## In the browser

Open `http://127.0.0.1:8000`.
Open `http://127.0.0.1:5173` (the API alone is on port 8000 and does not serve the console).

1. On **Create task**, leave **Replay** selected. Click **Create task**.
- Expected: you see **Confirm contract**.
Expand Down
58 changes: 53 additions & 5 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,22 +4,70 @@ A local security checkpoint for Terraform changes. Confirm an immutable contract

**Working locally:** real Terraform fixtures, deterministic and Cedar evaluation, FastAPI, SQLite audit, React/Tailwind console, local Ollama/Strands Terraform editing, blocked AWS apply proof, and successful local-resource apply proof. Docker-based emulator apply remains unavailable. See [STATUS.md](STATUS.md).

## Start on Windows
## Architecture (Phase 2)

Requirements: Python 3.11+, Node 20+, Terraform on PATH. Fixture planning uses dummy AWS credentials and performs no AWS refresh or apply.
```text
Browser -> React/Vite (127.0.0.1:5173) -> relative /api -> FastAPI (127.0.0.1:8000)
-> Strands/Ollama, Terraform, Cedar, SQLite
```

FastAPI serves **only** the API (`/api/*`, `/docs`, `/openapi.json`); it does not host the frontend and starts with or without `web/dist`. Vite serves the UI and proxies `/api`, `/docs` and `/openapi.json` to FastAPI. All frontend HTTP goes through `web/src/api.js`. See [API contract](docs/api_contract.md) and the [Phase 3 migration map](docs/phase3_migration_map.md). The PlanBound visual site in `design/` is a standalone prototype with sample data; it is not connected to the backend.

## Start on Windows (development)

Requirements: Python 3.11+ (3.12/3.13 tested), Node 20+, Terraform on PATH. Fixture planning uses dummy AWS credentials and performs no AWS refresh or apply. No AWS credentials are needed.

One-time setup, from the repository root:

```powershell
python -m venv .venv
.\.venv\Scripts\python.exe -m pip install -r requirements.txt
.\.venv\Scripts\python.exe terraform\scripts\generate.py # creates local Terraform state/plans for fixtures (needs network for the AWS provider on first run)
cd web; npm ci; cd ..
```

`generate.py` rewrites the tracked fixture files under `terraform/fixtures/` and `tests/fixtures/`; review `git status` afterwards and do not commit incidental changes.

Run each service in its own terminal:

```powershell
.\start.ps1 # FastAPI -> http://127.0.0.1:8000 (docs: /docs)
.\start-frontend.ps1 # Vite -> http://127.0.0.1:5173
```

There is no single-command launcher. Open <http://127.0.0.1:5173>. Check it works:

```powershell
curl.exe http://127.0.0.1:8000/api/health # direct
curl.exe http://127.0.0.1:5173/api/health # through the Vite proxy
```

Open [the console](http://localhost:8000) or [the API documentation](http://localhost:8000/docs). The server listens only on loopback. Run the frontend build before starting the backend so it can mount the compiled console.
Both return `{"status":"ok","evaluator":"cedar","cloud_apply":false}`. The **API reference** link in the console opens `/docs` via the proxy. Stop each service with Ctrl+C in its terminal.

Production build of the UI (does not need FastAPI running): `cd web; npm run build`. The build output is not served by FastAPI.

Common failures:

| Symptom | Fix |
|---|---|
| Console shows `BACKEND_UNAVAILABLE` / "backend is not reachable" | Start `.\start.ps1`; check port 8000 is free. |
| `Missing .venv` | Run the one-time setup above. |
| `409 Pipeline incomplete: run fixture generator first` | Run `terraform\scripts\generate.py`. |
| `MODEL_UNAVAILABLE` in Live mode | Start Ollama and `ollama pull llama3.2:3b`. The backend never falls back to replay silently. |
| Vite: port 5173 in use | Stop the other process (Vite is set to `strictPort`, so the Origin allow-list stays valid). |
| `403 Cross-origin mutation blocked` | Open the UI on `127.0.0.1:5173` or `localhost:5173`; other origins are refused by design. |
| `Invalid host header` | Only `localhost` / `127.0.0.1` Host headers are accepted. |

Set `PLANREVIEW_API_URL` to point Vite at a different local backend URL (development only).

On Linux/macOS use `.venv/bin/python`, `npm`, and `python -m uvicorn engine.api:app --host 127.0.0.1 --port 8000`. Python requirements were executed on Windows 3.12; platform-specific lock output is in `requirements.lock.txt`.
On Linux/macOS use `.venv/bin/python -m uvicorn engine.api:app --host 127.0.0.1 --port 8000` and `cd web && npm run dev`. Platform-specific lock output is in `requirements.lock.txt`.

## Verify

```powershell
.\.venv\Scripts\python.exe -m pytest -q
.\.venv\Scripts\python.exe -m pytest -q # backend (slow: real Terraform)
cd web; npm test # frontend API-client tests (mocked fetch)
.\.venv\Scripts\python.exe tools\test_phase2_smoke.py # real FastAPI + Vite processes and /api proxy (ports 8000/5173 must be free)
.\run-demo.ps1
# Reproducible automation; approvals are marked as scripted demo actions:
.\run-demo.ps1 -Scripted
Expand Down
143 changes: 143 additions & 0 deletions docs/api_contract.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
# PlanReview backend API contract (Phase 2)

Documents the API **as implemented** in `engine/api.py` and `engine/pipeline.py` on the Phase 1 code. Shapes were observed against a running backend. Nothing here is aspirational; gaps are listed in the last section. The machine-generated schema is at `/openapi.json` and `/docs`, but most responses are untyped dicts, so this document is the reference for response bodies.

The backend serves **only** the API. It does not host the frontend. It binds to `127.0.0.1:8000` by default.

## Conventions

- Base path `/api`. Bodies are JSON. Real AWS apply is disabled (`/api/health` → `cloud_apply: false`).
- **Local protections:** `TrustedHostMiddleware` accepts `Host` of `localhost`, `127.0.0.1` (any port), `testserver`. A non-`GET/HEAD/OPTIONS` request that carries an `Origin` header must be one of `http://localhost:8000`, `http://127.0.0.1:8000`, `http://localhost:5173`, `http://127.0.0.1:5173`, otherwise `403 {"detail":"Cross-origin mutation blocked"}`. There is no CORS middleware and no authentication. Origin checking is a local CSRF guard, not authorization. The Vite proxy forwards the browser's `Origin: http://127.0.0.1:5173` (or `localhost`), which is allowed.
- All task mutations run under one process-wide lock (`LOCK`); requests are serialized.
- Unknown routes return FastAPI's JSON `404 {"detail":"Not Found"}`, never HTML.

### Error shapes (three, all under `detail`)

| Shape | Source | Example |
|---|---|---|
| Structured | `PlanReviewError` subclasses | `{"detail":{"error":"UNSUPPORTED_OPERATION","message":"…","details":{…}}}` |
| Legacy string | `ValueError` → 409, `OSError/TimeoutError` → 503, origin block → 403, unknown route → 404 | `{"detail":"Pipeline incomplete: evaluate first"}` |
| Pydantic | request validation → 422 | `{"detail":[{"type":"string_too_short","loc":["body","task"],"msg":"…"}]}` |

Structured codes and HTTP status: `UNSUPPORTED_OPERATION` 400, `AMBIGUOUS_REQUEST` 400, `MODEL_UNAVAILABLE` 503 (Ollama unreachable or model not installed in `ollama`/`live` mode; the backend does **not** fall back to replay). Other `PlanReviewError`s default to `GENERIC_ERROR` 400.

Notable legacy errors (all HTTP 409 unless noted): `Task not found` (**409, not 404**), `Contract is immutable after confirmation`, `Active confirmed contract required`, `Contract expired`, `Pipeline incomplete: …` (wrong stage), `Plan the prepared edit before editing again`, `Run already applied`, `Unplanned edits exist; create and evaluate a new plan`, `Policies changed; evaluate again…`, `Address X experienced an evaluation error and cannot be approved…`, `Only REVIEW items accept approve/reject resolutions…`. `Pipeline incomplete: run fixture generator first` is also 409 (fixtures missing). A `503 Pipeline incomplete: …` comes from OS/timeouts (e.g. Terraform missing).

The frontend client (`web/src/api.js`) maps these to `{code, message, details, status}`; see the client rules below.

### Stages

`task.stage`: `draft` → `confirmed` → `edited` → `planned` → `canonicalized` → `evaluated` → `resolved` → (`blocked` | `applied` | `failed`, from the apply result status lowercased). Re-editing/re-planning starts a new **run** in `task.runs`; only `runs[-1]` (the latest) is authoritative. Resolutions are cleared by a new plan or re-evaluation.

## Endpoints

"Long" = may take seconds to minutes (Terraform, Ollama).

### `GET /api/health`
`200 {"status":"ok","evaluator":"cedar","cloud_apply":false}`. Read-only, safe to retry.

### `GET /api/tasks`
`200 [Task, …]` (full task objects, including all runs; newest first). Read-only. No pagination.

### `POST /api/tasks` — mutates, **long in `ollama`/`live` mode**
Body `{"task": string(1..4000), "mode": "replay"|"adversarial"|"ollama"|"live"}` (mode default `replay`). Interprets the request (AI-assisted in ollama/live), validates against the capability registry, drafts a contract.
Success `200 Task` with `stage:"draft"`, `contract.status:"draft"`, `intent` (`operation`, `resource_address`, `resource_type`, `attribute`, `requested_value`, `raw_task`, `status:"VALIDATED"`, `reason`).
Errors: 400 `UNSUPPORTED_OPERATION`, 400 `AMBIGUOUS_REQUEST`, 503 `MODEL_UNAVAILABLE`, 409 `Unknown mode`, 422 empty/oversized `task`.
Retry: **not idempotent** — every success creates a new task. Do not retry automatically.

### `GET /api/tasks/{id}` / `GET /api/tasks/{id}/intent`
`200 Task` / `200 Task.intent` (may be `null`). `409 Task not found`. Read-only.

### `POST /api/tasks/{id}/confirm` — mutates
Body: the contract object (optional; defaults to the stored draft). The contract id must not change. Seals the contract with a hash (`confirmed_contract_hash`) and sets `contract.status:"confirmed"`, `stage:"confirmed"`.
Errors: 409 `Contract is immutable after confirmation`, `Contract ID cannot change`, integrity mismatch, and an invalid contract body (raw pydantic text in a legacy 409 string, since `pydantic.ValidationError` is a `ValueError`; the body is `dict`, so FastAPI does not return 422). Retry: a second call fails (immutable).

### `POST /api/tasks/{id}/agent?variant=…` — mutates, **long**
Query `variant` ∈ `baseline|intended|review|poisoned|adversarial` (default `poisoned`; replay-mode fixture selector, ignored in `ollama`/`live` mode which edits the baseline with the Strands agent). Requires an active confirmed contract. Prepares `main.tf` in the task workspace. `stage:"edited"`, `prepared:true`.
Errors: 409 `Active confirmed contract required`, `Unknown fixture`, `Plan the prepared edit before editing again`, `Pipeline incomplete: run fixture generator first`; 503 `MODEL_UNAVAILABLE` (live). Retry: not while `prepared` is true.
Note: in replay mode the fixture variant, not the natural-language task, determines the edit.

### `POST /api/tasks/{id}/plan` — mutates, **long**
Runs a real `terraform plan -refresh=false -out=…` (dummy credentials, no AWS calls), enforces the pre-plan guard, saves the plan and hashes. Appends a run to `runs` (`plan_path`, `plan_hash`, `raw_path`, `raw_hash`, `plan_stdout`). `stage:"planned"`. Requires stage `edited`.
Errors: 409 `Pipeline incomplete: run edits first`, `Contract expired`, guard rejections and Terraform stderr (text of the failure), 503 on OS/timeout.
Retry: creates another run; do not retry blindly.

### `POST /api/tasks/{id}/canonicalize` — mutates
Converts the saved raw plan into `run.canonical[]`: `{address, resource_type, action, environment, region, changes:[{attribute,before,after}], dependencies, unknown}`. `stage:"canonicalized"`. Errors: 409 `Raw plan hash mismatch`. Idempotent for the same run.

### `POST /api/tasks/{id}/evaluate` — mutates
Evaluates each canonical change with Cedar against the confirmed contract. Sets `run.verdicts[]` (`{address, verdict, changes, reason, determining_policies}`) and `run.policy_hash`; clears `run.resolutions`. `stage:"evaluated"`. Errors: 409 `Pipeline incomplete: canonicalize first`.

**`verdict` is one of `ALLOW`, `REVIEW`, `DENY`, `EVALUATION_ERROR`.** `EVALUATION_ERROR` is a **technical failure** of Cedar evaluation (`reason` starts `Cedar evaluation failed:`). It is not REVIEW, cannot be approved through `/resolve`, and always blocks `/apply`. The client must present it as a failure, distinct from REVIEW.

### `POST /api/tasks/{id}/resolve` — mutates
Body `{ "<address>": "approve" | "reject", … }`. Only addresses whose verdict is `REVIEW` accept decisions; `DENY` and `EVALUATION_ERROR` addresses are rejected with 409. Stored on `run.resolutions` and bound to that run's plan/policy hash. `stage:"resolved"`.
Errors: 409 `Only REVIEW items accept approve/reject…`, `…experienced an evaluation error and cannot be approved…`, `Pipeline incomplete: evaluate first`. Retry: idempotent for identical decisions on the same run.
Gap: the backend does not return who resolved.

### `POST /api/tasks/{id}/apply` — mutates
The **server-side gate**. Verifies contract validity, no unplanned edits, policy hash, plan hash, verdicts, resolutions. Result in `run.apply_result` `{status, reason, spawned}` and `stage`:
- `BLOCKED` — gate reason text (unresolved REVIEW/DENY, evaluation error, expired contract, hash mismatch, or `AWS apply disabled: configure and verify an isolated emulator first`).
- `APPLIED` / `FAILED` — only reachable for a genuine local-resource Terraform apply in tests; real AWS apply is disabled.
Returns the full Task (`200`) even when blocked; a block is data, not an HTTP error. Errors: 409 `Run already applied`, `Unplanned edits exist…`, `Policies changed…`, `Pipeline incomplete: evaluate first`. May be long when it spawns Terraform. Not idempotent after `APPLIED`.
The frontend must display `apply_result`; it must not compute its own gate.

### `GET /api/tasks/{id}/audit`
`200 [{id, timestamp, kind, data}, …]` — persistent SQLite audit, in stored order. Kinds seen: `draft`, `human_confirmation`, `apply_result`, and others for each stage. Read-only.

## Supported Phase 1 operations (boundaries)

Only two infrastructure edits are supported:

| Operation | Resource | Attribute | Constraint |
|---|---|---|---|
| `update_memory` | `aws_lambda_function.dev_api` | `memory_size` | integer 128–10240 MB (1 MB increments) |
| `update_tags` | `aws_s3_bucket.assets` | `tags.Team` | non-security `Team` tag only |

Anything else (RDS, IAM, security groups, networking, deletes, compound requests) returns `UNSUPPORTED_OPERATION` or `AMBIGUOUS_REQUEST`. Replay-mode fixtures (`poisoned`, `review`, …) exist to demonstrate ALLOW/REVIEW/DENY on real Terraform plans and are not new capabilities.

## Recommended real workflow

1. `POST /tasks` → inspect `intent` and `contract` (draft).
2. `POST /tasks/{id}/confirm` with the (possibly edited) contract.
3. `POST /tasks/{id}/agent` → `POST …/plan` → `POST …/canonicalize` → `POST …/evaluate`. Stop at the first failure; do not skip stages.
4. Inspect `runs[-1].verdicts`. `EVALUATION_ERROR` → stop; repair and re-plan.
5. `POST …/resolve` for `REVIEW` items.
6. `POST …/apply` → read `apply_result`.
7. `GET …/audit`.

## Frontend client rules (`web/src/api.js`)

- Every failure throws `ApiError {code, message, details, status}`. Backend codes pass through; client codes: `NETWORK_ERROR`, `BACKEND_UNAVAILABLE` (empty-body 500/502/503/504 from the Vite proxy), `TIMEOUT`, `ABORTED`, `INVALID_RESPONSE` (non-JSON 2xx), `VALIDATION_ERROR`, `INVALID_STATE` (409), `FORBIDDEN`, `NOT_FOUND`, `PIPELINE_INCOMPLETE` (legacy 503), `HTTP_ERROR`.
- No automatic retries. Timeouts/aborts on mutating calls state that the backend may still be running the operation; the client never claims Terraform or the agent was cancelled.
- Timeouts: 30 s default; 15 min for create/agent/plan.

## Gaps for the Phase 3 UI (documented, not implemented)

- `Task not found` is 409, and mutation-error codes are mostly free-text legacy strings; no stable code for stage errors.
- Task responses expose absolute filesystem paths (`workspace`, `plan_path`, `raw_path`) and full `plan_stdout`; no summary/diff-only endpoint.
- `GET /tasks` returns full objects with no pagination or summary view.
- No per-stage progress or cancellation endpoint; long operations are a single blocking request.
- No endpoint to read Cedar policy text or per-policy explanations beyond `determining_policies` ids and `reason`.
- No resolver identity on `resolutions`; no evidence export endpoint (audit list only).
- No endpoint listing supported operations (the capability registry is only visible through errors).

## Observed examples

Unsupported request (`POST /api/tasks`, HTTP 400):

```json
{"detail":{"error":"UNSUPPORTED_OPERATION","message":"The requested operation or service is outside genuinely supported capabilities. Matched unsupported pattern: \brds\b. Genuine supported operations: update_memory, update_tags.","details":{"task":"Delete the production RDS database and open port 22 to the world","pattern":"\brds\b"}}}
```

Live-model proposal rejected by the deterministic validator (`mode:"ollama"`, HTTP 400). The model returned a free-text resource name; the capability registry accepts only the canonical address:

```json
{"detail":{"error":"UNSUPPORTED_OPERATION","message":"Resource 'dev-api-Lambda' is not supported for operation 'update_memory'. Permitted resource: 'aws_lambda_function.dev_api'.","details":{"proposal":{"operation":"update_memory","resource_address":"dev-api-Lambda","attribute":"memory_size","requested_value":512}}}}
```

Empty task (HTTP 422): `{"detail":[{"type":"string_too_short","loc":["body","task"],"msg":"String should have at least 1 character"}]}`

Apply blocked by the gate (HTTP 200; `run.apply_result`): `{"status":"BLOCKED","reason":"2 unresolved DENY; edit configuration and create a new plan","spawned":false}`

Model unavailable and evaluation-error bodies follow the structured/`verdict:"EVALUATION_ERROR"` shapes above; they are exercised by mocked tests in `tests/test_phase1.py`, not captured from a live run.
Loading
Loading