Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions apps/web/features/missions/missions-view.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
import { readFile } from "node:fs/promises";
import { describe, expect, it } from "vitest";

const viewUrl = new URL("./missions-view.tsx", import.meta.url);

describe("Missions empty state", () => {
it("names the governed tool and the prerequisites for calling it", async () => {
const source = await readFile(viewUrl, "utf8");
expect(source).toContain("muster_upsert_mission");
expect(source).toContain("workflows.manage");
expect(source).toContain("create-installation");
expect(source).toContain('href="/guides"');
});

it("does not offer a UI create path", async () => {
const source = await readFile(viewUrl, "utf8");
expect(source).toContain("no create path by design");
expect(source).not.toContain("New mission");
expect(source).not.toContain("useUpsertMission");
});
});
27 changes: 26 additions & 1 deletion apps/web/features/missions/missions-view.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,32 @@ export function MissionsView() {
{missions.data && missions.data.length === 0 ? (
<EmptyState
title="No missions defined"
description="Mission definitions are created via governed MCP tools or bootstrap. They appear here once present."
description="Mission definitions are written by the governed MCP tool muster_upsert_mission, which Hermes calls against this workspace. This UI has no create path by design (ADR 0005)."
action={
<div className="max-w-md space-y-2 text-left text-xs text-muted-foreground">
<p>
To get a mission listed here, grant the MCP installation the{" "}
<span className="font-mono">workflows.manage</span> capability
and the{" "}
<span className="font-mono">muster_upsert_mission</span>{" "}
scope, then have Hermes call that tool with a name,
description, and capability envelope.
</p>
<p>
An operator provisions the installation with{" "}
<span className="font-mono">
pnpm --filter @muster/mcp create-installation
</span>
.
</p>
<Link
href="/guides"
className="inline-block font-medium underline-offset-2 hover:underline"
>
Guides: Missions and Audit
</Link>
</div>
}
/>
) : null}
{missions.data && missions.data.length > 0 ? (
Expand Down
113 changes: 113 additions & 0 deletions docs/operations/audit-chain-verification.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
# Operational runbook: audit chain verification

Muster's `audit_events` table is a hash-chained, append-only log. This runbook
covers running the verifier and interpreting its result. For the attestation
and evidence-custody steps that follow a non-strict result, continue with
[incident-recovery.md](incident-recovery.md#audit-integrity-verification-and-attestation).

**The verifier never writes.** Nothing in this runbook asks you to change an
audit row, and nothing ever should. A hash mismatch is investigated, recorded,
and attested — never "repaired".

## Run it

Verification is scoped to one organisation.

```bash
MUSTER_AUDIT_ORGANISATION_ID=<organisation-uuid> pnpm db:verify-audit
# or
pnpm db:verify-audit --organisation=<organisation-uuid>
```

If you do not know the organisation id, run the command with neither. It lists
the organisations in the connected database and exits `64` without verifying
anything:

```
MUSTER_AUDIT_ORGANISATION_ID is not set and --organisation was not passed.

Usage:
MUSTER_AUDIT_ORGANISATION_ID=<uuid> pnpm db:verify-audit
pnpm db:verify-audit --organisation=<uuid>

Organisations in this database:
018f55d8-c4c7-7c3e-88ef-000000000001 muster
```

The machine-readable JSON report goes to stdout; the operator interpretation
goes to stderr. Capture both:

```bash
pnpm db:verify-audit --organisation=<uuid> >report.json 2>report.txt
```

## Exit codes

| Code | Outcome | Meaning |
| ---- | ------------------------------ | -------------------------------------------------------------------------------------- |
| `0` | `strict-valid` | Every event rehashes exactly. Nothing to do. |
| `1` | `invalid` | A mismatch no known legacy path explains. Treat as an incident. |
| `2` | `legacy-compatible-not-strict` | Only the known pre-normalisation `approvalId` defect. Strict verification still fails. |
| `64` | — | Usage: no organisation id, or not a UUID. Nothing was verified. |
| `69` | — | The database could not be reached. Nothing was verified. |

`64` and `69` are deliberately distinct from `1`: an operator who forgot an
environment variable, and a database that is down, must not page anyone with
"audit chain invalid".

## Interpreting `legacy-compatible-not-strict`

This is the expected result on any workspace that recorded integration actions
before audit metadata normalisation. It is what the homelab reports today, with
strict verification failing at sequence 136.

What happened: `integration.action.queued` / `.succeeded` / `.failed` events
were hashed with `approvalId: undefined` present in their metadata object.
PostgreSQL JSONB drops undefined properties on write, so the stored metadata no
longer contains the key the stored hash was computed over. The event is
authentic; its hash is simply reproducible only by re-adding the omitted
property, which is what `packages/audit`'s legacy compatibility path does.

What it does **not** mean:

- It is not a repair. `historicalChainRepaired` is always `false`.
- It does not make strict verification pass. `strict.valid` stays `false`, and
the report keeps reporting the failing sequence.
- It is not evidence of tampering by itself — but it also is not a clean bill
of health, which is why the exit code is not `0`.

Scope check before accepting it: every sequence listed in
`legacyApprovalIdOmissions` must be one of those three integration-action
actions, and the sequence in `strict.brokenAt` must be the first of them. The
verifier enforces this; anything outside that set is reported `invalid`
instead.

Then:

1. Keep `report.json` and `report.txt` with the incident or restore evidence.
2. Record the report time, application revision, command, `strict.brokenAt`,
and every listed legacy sequence in the recovery attestation described in
[incident-recovery.md](incident-recovery.md#audit-integrity-verification-and-attestation).
3. State explicitly in that attestation that the result is a compatibility
reconstruction only and that no audit event was rewritten or removed.

The count of affected sequences is fixed history: it can only stay the same or
be exceeded by a genuinely new defect. If a later run reports a legacy sequence
that was not in the previous run, events are still being written with the old
shape — that is a code regression in the integration-action path, and it is a
bug to fix at the writer, never in the stored rows.

## Interpreting `invalid`

Something rehashes wrong and no known reconstruction explains it. Preserve the
rows exactly as stored, contain writes if warranted, and follow the incident
path in [incident-recovery.md](incident-recovery.md). Do not run any command
that updates audit hashes, metadata, sequences, or outbox history.

## When strict verification will pass again

Never, for a workspace that already holds pre-normalisation events — and that
is correct. Immutable history cannot be made strictly reproducible without
rewriting it. `legacy-compatible-not-strict` is the permanent, honest result
for those workspaces; a workspace bootstrapped after normalisation reports
`strict-valid`.
6 changes: 5 additions & 1 deletion docs/operations/incident-recovery.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,11 @@ MUSTER_AUDIT_ORGANISATION_ID=<organisation-uuid> pnpm db:verify-audit

The command emits an operator-visible JSON report and does not write, repair,
delete, or otherwise alter audit history. It exits `0` only for
`strict-valid`, `2` for `legacy-compatible-not-strict`, and `1` for `invalid`.
`strict-valid`, `2` for `legacy-compatible-not-strict`, and `1` for `invalid`;
`64` and `69` mean nothing was verified (missing organisation id, unreachable
database). Running it, finding the organisation id, and interpreting each
outcome are covered in
[audit-chain-verification.md](audit-chain-verification.md).

`legacy-compatible-not-strict` identifies only the known historical defect:
an integration-action event was hashed before PostgreSQL JSONB omitted an
Expand Down
108 changes: 108 additions & 0 deletions packages/database/src/audit-integrity-cli.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
import { describe, expect, it } from "vitest";
import { hashAuditEvent, verifyAuditIntegrity } from "@muster/audit";
import type { HashableAuditEvent } from "@muster/audit";
import {
RUNBOOK,
auditIntegrityExitCodes,
describeAuditIntegrity,
resolveAuditOrganisationId,
} from "./audit-integrity-cli.ts";

const organisationId = "018f55d8-c4c7-7c3e-88ef-000000000001";
const base: HashableAuditEvent = {
organisationId,
sequence: 1,
actorId: "actor",
actorType: "human",
action: "room.message.created",
targetType: "message",
targetId: "message",
previousHash: "0".repeat(64),
metadata: { safe: true },
traceId: "trace",
createdAt: "2026-07-26T00:00:00.000Z",
};

function legacyEvent() {
const input = {
...base,
action: "integration.action.queued",
metadata: { operation: "alerts.list", approvalId: undefined },
};
return {
...input,
metadata: { operation: "alerts.list" },
eventHash: hashAuditEvent(input),
};
}

describe("audit organisation id resolution", () => {
it("accepts a UUID from either the environment or the flag", () => {
expect(resolveAuditOrganisationId(` ${organisationId} `)).toEqual({
organisationId,
});
});

it("lists candidate organisations when none was supplied", () => {
const resolved = resolveAuditOrganisationId(undefined, [
{ id: organisationId, slug: "muster", name: "Muster Workspace" },
]);
expect(resolved).not.toHaveProperty("organisationId");
const usage = "usage" in resolved ? resolved.usage : "";
expect(usage).toContain("MUSTER_AUDIT_ORGANISATION_ID is not set");
expect(usage).toContain(organisationId);
expect(usage).toContain("muster");
expect(usage).toContain(RUNBOOK);
});

it("explains a malformed id instead of failing anonymously", () => {
const resolved = resolveAuditOrganisationId("not-a-uuid");
expect("usage" in resolved && resolved.usage).toContain("not a UUID");
});

it("says so when no organisation could be listed", () => {
const resolved = resolveAuditOrganisationId(undefined, []);
expect("usage" in resolved && resolved.usage).toContain("DATABASE_URL");
});
});

describe("audit integrity interpretation", () => {
it("reports a clean chain without ambiguity", () => {
const report = verifyAuditIntegrity([
{ ...base, eventHash: hashAuditEvent(base) },
]);
expect(auditIntegrityExitCodes[report.outcome]).toBe(0);
expect(describeAuditIntegrity(report)).toContain(
"Strict verification passed",
);
});

it("names the failing sequence and forbids repair for a legacy omission", () => {
const report = verifyAuditIntegrity([legacyEvent()]);
expect(report.outcome).toBe("legacy-compatible-not-strict");
expect(auditIntegrityExitCodes[report.outcome]).toBe(2);

const description = describeAuditIntegrity(report);
expect(description).toContain("failed at sequence 1");
expect(description).toContain("approvalId omission");
expect(description).toContain("Do not mutate any audit row");
expect(description).toContain(RUNBOOK);
expect(description).not.toMatch(/repair(ed)? the chain/i);
});

it("escalates an unexplained mismatch as a possible incident", () => {
const report = verifyAuditIntegrity([
{ ...base, eventHash: "0".repeat(64) },
]);
expect(auditIntegrityExitCodes[report.outcome]).toBe(1);

const description = describeAuditIntegrity(report);
expect(description).toContain("no known legacy reconstruction");
expect(description).toContain("Preserve the rows exactly as stored");
});

it("separates operator error and outage from chain failure", () => {
expect(auditIntegrityExitCodes.usage).toBe(64);
expect(auditIntegrityExitCodes.unavailable).toBe(69);
});
});
103 changes: 103 additions & 0 deletions packages/database/src/audit-integrity-cli.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
import { z } from "zod";
import type { AuditIntegrityReport } from "@muster/audit";

export const RUNBOOK = "docs/operations/audit-chain-verification.md";

/**
* 64/69 are sysexits EX_USAGE/EX_UNAVAILABLE: a missing organisation id or an
* unreachable database must not raise the same alarm as a chain that failed
* verification.
*/
export const auditIntegrityExitCodes = {
"strict-valid": 0,
invalid: 1,
"legacy-compatible-not-strict": 2,
usage: 64,
unavailable: 69,
} as const;

export interface AuditOrganisationChoice {
id: string;
slug: string | null;
name: string | null;
}

function usage(
reason: string,
known: ReadonlyArray<AuditOrganisationChoice>,
): string {
const lines = [
reason,
"",
"Usage:",
" MUSTER_AUDIT_ORGANISATION_ID=<uuid> pnpm db:verify-audit",
" pnpm db:verify-audit --organisation=<uuid>",
"",
];
if (known.length === 0) {
lines.push(
"No organisations could be listed from this database. Check DATABASE_URL.",
);
} else {
lines.push("Organisations in this database:");
for (const choice of known) {
lines.push(
` ${choice.id} ${choice.slug ?? choice.name ?? ""}`.trimEnd(),
);
}
}
lines.push("", `Runbook: ${RUNBOOK}`);
return lines.join("\n");
}

export function resolveAuditOrganisationId(
candidate: string | undefined,
known: ReadonlyArray<AuditOrganisationChoice> = [],
): { organisationId: string } | { usage: string } {
const trimmed = candidate?.trim();
if (!trimmed) {
return {
usage: usage(
"MUSTER_AUDIT_ORGANISATION_ID is not set and --organisation was not passed.",
known,
),
};
}
if (!z.string().uuid().safeParse(trimmed).success) {
return {
usage: usage(`"${trimmed}" is not a UUID organisation id.`, known),
};
}
return { organisationId: trimmed };
}

/**
* Operator-readable interpretation of the machine-readable report. Every branch
* states that immutable history stays untouched: the only safe response to a
* hash mismatch is investigation, never a rewrite.
*/
export function describeAuditIntegrity(report: AuditIntegrityReport): string {
const brokenAt = report.strict.valid ? undefined : report.strict.brokenAt;

if (report.outcome === "strict-valid") {
return `Strict verification passed. The chain is intact and nothing was changed. Runbook: ${RUNBOOK}`;
}

if (report.outcome === "legacy-compatible-not-strict") {
const sequences = report.legacyApprovalIdOmissions
.map((match) => match.sequence)
.join(", ");
return [
`Strict verification failed at sequence ${brokenAt}.`,
`Every failing event is a known pre-normalisation approvalId omission (sequence ${sequences}); its stored hash is reproducible only under the legacy compatibility path.`,
"This is expected on workspaces that recorded integration actions before metadata normalisation. The chain still links, and no event was repaired or rewritten.",
`Do not mutate any audit row. Record this outcome against the runbook: ${RUNBOOK}`,
].join("\n");
}

return [
`Strict verification failed at sequence ${brokenAt} and no known legacy reconstruction explains it.`,
"Treat this as a potential tampering or corruption incident. Preserve the rows exactly as stored.",
`Runbook: ${RUNBOOK}`,
].join("\n");
}
Loading
Loading