VaryProof

中文说明

Prove when an HTTP response changes along a request-header axis that its Vary header failed to declare.

A static header check sees only one response. VaryProof sends controlled pairs, repeats every exact sample to reject unstable evidence, then compares body bytes, status, representation metadata, validators, redirect location, and selected CORS headers. The result is a CI exit code plus console, JSON, GitHub annotation, or self-contained HTML evidence.

VaryProof FAIL — http://127.0.0.1:8000/broken-language
[ERROR VP001] Changing Accept-Language changed body_bytes, body_sha256, content-language, but Vary omitted the field for en, fr; a shared cache may reuse the wrong representation.
Accept-Language: undeclared_variance  changed: body_bytes, body_sha256, content-language
Summary: 1 axis/axes, 1 finding(s), 0 inconclusive

One-minute proof

Requirements: Python 3.11+ and uv.

git clone https://github.com/KanadeK/varyproof.git
cd varyproof
uv sync --all-extras --locked
uv run python -m examples.server

In another terminal:

uv run varyproof audit http://127.0.0.1:8000/broken-language --axis Accept-Language=en --axis Accept-Language=fr --allow-private
uv run varyproof audit http://127.0.0.1:8000/correct-language --axis Accept-Language=en --axis Accept-Language=fr --allow-private
uv run python -m scripts.demo --output-dir reports

The broken endpoint exits 1 with VP001; the corrected endpoint exits 0. The demo command also writes a standalone HTML failure report and a JSON pass report.

Install the released wheel without cloning:

python -m pip install https://github.com/KanadeK/varyproof/releases/download/v0.1.0/varyproof-0.1.0-py3-none-any.whl
varyproof --version

Audit a real endpoint

varyproof audit https://example.com/page --axis Accept-Language=en-US --axis Accept-Language=fr-FR --format html --output varyproof.html

Or commit a versioned config:

{
  "schema": "varyproof.config.v1",
  "url": "https://example.com/page",
  "axes": [
    {
      "header": "Accept-Language",
      "values": ["en-US", "fr-FR"]
    },
    {
      "header": "Origin",
      "values": ["https://a.example", "https://b.example"]
    }
  ]
}

varyproof audit --config varyproof.json --format json --output varyproof.json

Each header is varied independently, so the report can attribute a change to that axis. Every value is requested twice. If identical input produces different fingerprints, the axis is inconclusive_unstable and the process exits 2 instead of guessing.

Findings and exits

Rule	Level	Meaning
VP001	error	Observed variance is missing from Vary, and at least one response is not explicitly private or no-store.
VP002	warning	The same mismatch exists, but every response is explicitly private or no-store.

Exit	Meaning
0	Audit completed; no finding reached --fail-on.
1	Audit completed; a finding reached --fail-on.
2	Invalid input, network failure, oversized response, or inconclusive evidence.

--fail-on error is the default. Use --fail-on warning for a stricter gate or --fail-on never to collect evidence without failing CI.

GitHub Action

Commit varyproof.json, then add:

steps:
  - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
  - uses: KanadeK/varyproof@v0.1.0
    with:
      config: varyproof.json
      fail-on: error

The action installs the repository package and emits native GitHub annotations. Its inputs are passed through environment variables rather than interpolated into shell code.

What is retained

Reports contain only status, body byte length/SHA-256, and an allowlist of representation/cache/CORS headers. They never contain response bodies, cookies, authorization values, Set-Cookie, or arbitrary headers. Standard credential axes are rejected.

Private, loopback, link-local, and reserved destinations are blocked unless --allow-private is explicit. Redirects are intentionally not followed in v0.1; a redirect response is itself audited. See SECURITY.md before running untrusted configs.

Why this is not another cache scanner

VaryProof does not crawl, score every cache directive, or generate poisoning payloads. It is a narrow differential proof for one supplied endpoint. The bounded competitor scan compares it with cache libraries, conformance suites, passive analyzers, and security scanners.

The protocol basis is RFC 9110 §12.5.5 and RFC 9111 §4.1. VaryProof always says “observed”: finite samples cannot prove every possible server behavior.

Development and acceptance

uv sync --all-extras --locked
uv run ruff check .
uv run mypy src
uv run pytest --cov=varyproof --cov-branch --cov-report=term-missing --cov-fail-under=90
uv run python -m scripts.release_check

The final command also checks lock consistency, builds wheel/sdist, installs the wheel into a fresh virtual environment, audits both bundled broken and corrected endpoints with the installed executable, and validates the generated JSON against report-v1.schema.json.

If a command fails, use the stage-specific failure playbook. Architecture, report fields, release gates, and the accepted v0.1 contract are documented in docs/ARCHITECTURE.md, docs/REPORT_FORMAT.md, docs/RELEASE_CHECKLIST.md, and specs/initial-release.md.