Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 25 additions & 9 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -342,16 +342,32 @@ ship. That is the case virtual patching exists for, and the pipeline cannot see
non-file identity (fall back to `Finding.endpoint` or the package coordinate) and
`pipeline.py:40`'s dedup key needs the same treatment.

- [ ] **H3** OpenAPI as a discovery input. (M, P2)
`A4` applies a schema the operator supplies. This is the other direction: read a spec and
find the flaws in it, including endpoints absent from the code and code paths absent from
the spec.
- Acceptance: a spec declaring `amount` with no lower bound gets reported as a finding and
routes to `api_schema`; endpoints in the spec with no code match report as
- [x] **H3** OpenAPI as a discovery input. (M, P2) — **DONE:** `vpcopilot scan --spec <path>`,
alone or alongside a repo. `inputs/openapi.py` does a deterministic structural pass (unbounded
numbers/strings/arrays, operations with no `security`, `additionalProperties` left open, `$ref`
followed with a cycle bound) and the spec is then handed to the **existing `discover` agent** as
one more file — no fourth agent, so spec findings verify, triage and generate exactly like code
findings. Verified live end to end.
- **Acceptance, as met:** a spec declaring `amount` with no lower bound produced
`neg-amount-001` → `service_policy` + **`api_schema`**; spec/code drift is reported as
`undocumented_or_orphaned`; findings flow into the same triage table.
- Surfaces: `src/vpcopilot/inputs/openapi.py`, `vpcopilot scan --spec <path>`.
- **Reconciled:** same `inputs/` package question as H1 — settle it there first. `--spec`
changes the same required `repo` positional H1 does; do both in one change.
- **Split by what needs judgement.** The structural pass is code, so recall does not depend on
which model is configured; the agent only decides which facts *matter* (an unbounded `page` is
nothing, an unbounded `amount` on a transfer is a hole). The orphan comparison is pure code —
it is a diff of two documents.
- **Two real bugs the first live run surfaced.** (a) A spec is ONE file declaring MANY endpoints,
so `endpoint_of("pay-api.yaml")` returned the same string for every finding and all six
collapsed onto one `service_policy` coverage key, with five logged "already covered" and given
no band-aid. Spec findings now carry `source` and leave `file` empty, and the coverage identity
prefers `endpoint` — three endpoints, three keys, and only the two genuinely on
`/api/v1/transfer` collapse. (b) The `undocumented_or_orphaned` finding was being sent through
`verify`, which reads the offending source; with no source to read it refuted the finding at
0.10 confidence. It is appended after verify instead — a comparison of two documents is not a
claim about code.
- **`--spec` is additive**, unlike `--cve`: alone it scans the contract, with a repo it also
reports the drift (which needs both). `run.json` records `input_kind` as
`repo` / `spec` / `repo+spec` / `advisory`.
- Fixture: `bench/fixtures/specs/pay-api.yaml`.

---

Expand Down
39 changes: 39 additions & 0 deletions bench/fixtures/specs/pay-api.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
openapi: 3.0.3
info:
title: Nimbus Payments API
version: "1.0"
servers: [{url: https://lab.banknimbus.com}]
paths:
/api/v1/transfer:
post:
operationId: createTransfer
summary: Move funds between two accounts
security: []
requestBody:
required: true
content:
application/json:
schema:
type: object
required: [from, to, amount]
additionalProperties: true
properties:
from: {type: string}
to: {type: string}
amount:
type: number
description: amount to transfer, in USD
memo: {type: string}
responses: {"200": {description: ok}}
/api/v1/accounts/{accountId}:
get:
operationId: getAccount
summary: Read an account
parameters:
- {name: accountId, in: path, required: true, schema: {type: string}}
responses: {"200": {description: ok}}
/api/v1/admin/reset:
post:
operationId: adminReset
summary: Reset the ledger (internal)
responses: {"204": {description: done}}
31 changes: 31 additions & 0 deletions docs/USAGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,37 @@ Three things about OSV worth knowing, each found by querying it:
Set `VPCOPILOT_ADVISORY_CACHE=<dir>` to cache advisories on disk so a demo does not depend on the
network.

### Scan an OpenAPI spec (H3)

A4 goes one way: you hand XC a schema and it enforces it. This is the other direction — read the
spec and find the flaws **in** it. A spec is a security artifact whether or not anyone treats it as
one, and it is often the only thing you have for a service you do not own.

```sh
vpcopilot scan --spec ./openapi.yaml --out out # the contract alone
vpcopilot scan ./app --spec ./openapi.yaml --out out # …and the code, plus the drift between them
```

`--spec` is **additive**: alone it scans the contract; alongside a repo it also compares the two.
(`--cve` is the exception — an advisory scan cannot be combined with either.)

**Split by what needs judgement.** A deterministic pass finds what the document leaves unstated —
a number with no `minimum`, a string with no `maxLength`, an operation with no `security`, an object
with `additionalProperties` open. Those are facts, so recall does not depend on which model is
configured. The agent then decides which of them *matter*: an unbounded `page` is nothing, an
unbounded `amount` on a transfer is a business-logic hole.

**Spec/code drift is a finding in both directions**, reported as `undocumented_or_orphaned`:

- **declared but unserved** — dead documentation, or a shadow API removed from one place and not
the other
- **served but undeclared** — the one that bites: applying an `api_schema` band-aid built from this
spec would start rejecting those routes

That finding is a comparison of two documents, so it skips the verify agent entirely — asking an
adversarial code reviewer to confirm a vulnerability in source it cannot see got it refuted at 0.10
confidence.

## 4. Apply a band-aid (mutates XC — gated + reversible)
```sh
vpcopilot apply --from-scan out/policies/<artifact>.json --lb <lb> --url <host> --dry-run # preview
Expand Down
9 changes: 6 additions & 3 deletions src/vpcopilot/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ def _root(
def scan(
repo: str = typer.Argument(None, help="path to the target application repo (omit when using --cve)"),
cve: str = typer.Option(None, "--cve", help="scan a security advisory instead of a repo: CVE-YYYY-NNNNN, GHSA-xxxx-xxxx-xxxx, PYSEC-YYYY-NN, GO-… or RUSTSEC-…"),
spec: str = typer.Option(None, "--spec", help="an OpenAPI/Swagger spec to scan for flaws IN THE CONTRACT — alone, or alongside a repo to also report spec/code drift"),
out: str = typer.Option("out", help="output directory for findings/policies/PRs"),
config: str = typer.Option(None, "--config", help="path to agents.yaml"),
min_confidence: float = typer.Option(0.5, "--min-confidence", help="drop verified findings below this confidence"),
Expand All @@ -51,13 +52,15 @@ def scan(
With --cve the input is a security advisory instead of a repo: the advisory is resolved from
OSV.dev, an agent derives its HTTP exploitation profile (or says there isn't one), and the
result enters the same triage and generate stages."""
if bool(repo) == bool(cve):
raise typer.BadParameter("pass a repo path or --cve, not " + ("both" if repo else "neither"))
if cve and (repo or spec):
raise typer.BadParameter("--cve scans one advisory; it cannot be combined with a repo or --spec")
if not (repo or cve or spec):
raise typer.BadParameter("pass a repo path, --cve, or --spec")
if code_fixes is None: # match the console default (app.py /api/defaults) so headless == UI
code_fixes = os.environ.get("VPCOPILOT_SCAN_REMEDIATE", "1").lower() not in ("0", "false", "no")
summary = run_pipeline(repo, out_dir=out, config_path=config, min_confidence=min_confidence,
concurrency=concurrency, max_files=max_files, max_bytes=max_bytes,
draft_code_fixes=code_fixes, advisory=cve,
draft_code_fixes=code_fixes, advisory=cve, spec_path=spec,
log=lambda m: rprint(f"[dim]{m}[/dim]"))
rprint(Panel.fit(
"\n".join(f"[bold]{k}[/bold]: {v}" for k, v in summary.items()),
Expand Down
16 changes: 10 additions & 6 deletions src/vpcopilot/console/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -594,6 +594,7 @@ class ScanReq(BaseModel):
# yields "". Every existing payload stays valid and nothing can 422.
repo: str = ""
cve: str = ""
spec: str = ""
out: str = "out"
min_confidence: float = 0.5
max_files: int = 200
Expand All @@ -603,14 +604,15 @@ class ScanReq(BaseModel):

def _run_scan(repo: str, out: str, min_confidence: float = 0.5,
max_files: int = 200, max_bytes: int = 60_000, draft_code_fixes: bool = True,
cve: str = ""):
cve: str = "", spec: str = ""):
_scan.update(state="running", log=[], summary=None, error=None)
try:
from ..pipeline import run_pipeline
summary = run_pipeline(repo or None, out_dir=out, config_path=_active_config,
min_confidence=min_confidence, max_files=max_files,
max_bytes=max_bytes, draft_code_fixes=draft_code_fixes,
advisory=cve or None, log=lambda m: _append(_scan["log"], m))
advisory=cve or None, spec_path=spec or None,
log=lambda m: _append(_scan["log"], m))
_scan.update(state="done", summary=summary)
except Exception as e: # noqa: BLE001
_scan.update(state="error", error=str(e))
Expand All @@ -624,17 +626,19 @@ def start_scan(body: ScanReq):
# The console reads results from OUT — so point OUT at the dir this scan writes to, or Review /
# Mitigate would read a different (empty) dir. Makes the Output-dir field authoritative even when
# it differs from the model-switcher default (e.g. out-claude-vampi).
if bool(body.repo.strip()) == bool(body.cve.strip()):
raise HTTPException(400, "give a repo path or a CVE/GHSA id, not "
+ ("both" if body.repo.strip() else "neither"))
if body.cve.strip() and (body.repo.strip() or body.spec.strip()):
raise HTTPException(400, "a CVE scan cannot be combined with a repo or a spec")
if not (body.repo.strip() or body.cve.strip() or body.spec.strip()):
raise HTTPException(400, "give a repo path, a CVE/GHSA id, or an OpenAPI spec")
global OUT
OUT = Path(body.out)
# kwargs, not a positional tuple: H2/H3 add more inputs here and a positional args tuple is one
# reordering away from scanning the wrong thing.
threading.Thread(target=_run_scan, daemon=True,
kwargs=dict(repo=body.repo, out=body.out, min_confidence=body.min_confidence,
max_files=body.max_files, max_bytes=body.max_bytes,
draft_code_fixes=body.draft_code_fixes, cve=body.cve)).start()
draft_code_fixes=body.draft_code_fixes, cve=body.cve,
spec=body.spec)).start()
return {"state": "running", "out": str(OUT)}


Expand Down
13 changes: 8 additions & 5 deletions src/vpcopilot/console/static/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,8 @@
<label>Target repo / directory</label><input type="text" id="scanRepo" list="scanRepoList" placeholder="pick a sibling app repo, or type a path" />
<label>…or a security advisory <span class="muted">(H1 — scan a CVE instead of a repo)</span></label>
<input type="text" id="scanCve" placeholder="CVE-2024-23334 · GHSA-… · PYSEC-… — resolved from OSV.dev, no credentials needed" />
<label>…or an OpenAPI spec <span class="muted">(H3 — find the flaws in the contract; add it to a repo scan to also report spec/code drift)</span></label>
<input type="text" id="scanSpec" placeholder="path to openapi.yaml or swagger.json" />
<div class="rowf">
<div><label>Output dir</label><input type="text" id="scanOut" list="scanOutList" value="" placeholder="e.g. out-dgx-vampi (or pick a run dir)" /></div>
<div><label>Min confidence</label><input type="number" id="scanMinConf" value="0.5" step="0.05" min="0" max="1" /></div>
Expand Down Expand Up @@ -866,12 +868,13 @@ <h2 style="margin-top:14px">Full matrix</h2><table><thead>${head}</thead><tbody>
// ---- scan (auto-advances to Review on done) ----
let SCAN_N = 0; // lines already appended — the server streams the transcript from here
scanLog.addEventListener("scroll",()=>{ scanFollow.style.display = atBottom(scanLog) ? "none" : "inline-block"; });
async function runScan(){ const repo=scanRepo.value.trim(), cve=scanCve.value.trim();
// Exactly one input — the server enforces this too; this is just a faster no.
if(!repo && !cve){alert("enter a repo path, or a CVE/GHSA id to scan an advisory instead");return;}
if(repo && cve){alert("give a repo path OR an advisory id, not both");return;}
async function runScan(){ const repo=scanRepo.value.trim(), cve=scanCve.value.trim(), spec=scanSpec.value.trim();
// repo and --cve are alternatives; a spec is additive. The server enforces this too — this is
// just a faster no.
if(!repo && !cve && !spec){alert("enter a repo path, a CVE/GHSA id, or an OpenAPI spec");return;}
if(cve && (repo || spec)){alert("a CVE scan cannot be combined with a repo or a spec");return;}
scanState.textContent="starting…";
try { await jpost("/api/scan",{repo,cve,out:scanOut.value.trim()||"out",min_confidence:parseFloat(scanMinConf.value)||0.5,max_files:parseInt(scanMaxFiles.value)||200,max_bytes:parseInt(scanMaxBytes.value)||60000,draft_code_fixes:scanRemediate.checked}); } catch(e){ scanState.textContent="error: "+e.message; return; }
try { await jpost("/api/scan",{repo,cve,spec,out:scanOut.value.trim()||"out",min_confidence:parseFloat(scanMinConf.value)||0.5,max_files:parseInt(scanMaxFiles.value)||200,max_bytes:parseInt(scanMaxBytes.value)||60000,draft_code_fixes:scanRemediate.checked}); } catch(e){ scanState.textContent="error: "+e.message; return; }
SCAN_N=0; scanLog.textContent=""; scanLogCount.textContent=""; scanLogWrap.style.display="block";
const poll=setInterval(async()=>{
let s; try { s=await jget("/api/scan?since="+SCAN_N); } catch(e){ return; } // transient fetch blip — keep polling
Expand Down
Loading
Loading