MemProof: Cryptographic Admission Control Against External Corpus Poisoning in RAG

Reference implementation, evaluation scripts, and JSON outputs for the MemProof project (ACM TAISAP submission). This is a code-and-results release; the manuscript itself is not included here.

Repository: https://github.com/akekulip/MemProof

What MemProof actually does

MemProof is an admission-control protocol for one specific subproblem: external corpus poisoning by sources the operator never authorized. It does not try to solve every RAG-poisoning threat. The contribution is the framing plus a cryptographic enforcement layer that, on the external-poisoning subproblem, matches the attack-success reduction of generation-side defenses without paying the generation-side accuracy cost.

The protocol has four layers, but the empirically load-bearing one against PoisonedRAG-style external poisoning is Layer 1:

1. Layer 1 — Provenance-verified ingestion. Every document must carry an Ed25519 signature from a source registered in the SourceRegistry. Unsigned documents never reach retrieval. This is the layer that drives the headline ASR result in Table 2 of the paper.
2. Layer 2 — Embedding integrity. HMAC-SHA256 commitment binding the embedding to the document. Defense-in-depth against a storage-layer attacker who can swap embeddings without touching content.
3. Layer 3 — Temporal audit. Hash-chained, monotonic-timestamp audit log of every ingest, update, delete, and query operation.
4. Layer 4 — Verified retrieval. Merkle membership proof attached to each retrieved entry; supports public verification by a third-party auditor given only the registry and a Merkle root.

The paper is explicit: a compromised trusted source bypasses Layer 1, and the insider-threat experiment in Section 6.5 shows MemProof degrading to No Defense in that regime. The right tool for that threat is RobustRAG or another content-level defense.

Repository layout

src/memproof/
  crypto/
    attestation.py     Ed25519 source attestation, JSON + binary serialization
    commitment.py      HMAC-SHA256 embedding commitment
    merkle.py          Incremental O(log n) Merkle tree
    source_registry.py SourceRegistry: source -> public key + trust level
  audit/
    log.py             Hash-chained audit log with monotonicity check
  ingestion/
    pipeline.py        Layer 1 + 2 + 3 ingestion path
  retrieval/
    verified.py        Layer 4 query-time verification
  backends/
    chroma.py          ChromaDB-backed vector store wrapper
  store.py             High-level MemProofStore convenience API

tests/
  test_memproof.py     40 unit tests; round-trip, signature, monotonicity,
                       binary encoding regression tests

evaluation/            Scripts + JSON outputs (see "Datasets and outputs")
  composition_eval.py            Main 4-way comparison on NQ (real Ed25519 path)
  composition_eval_realtimeqa.py Same comparison on RealtimeQA
  composition_eval_claude.py     Same on NQ with Claude Haiku 4.5 generator
  insider_threat_eval.py         Insider-threat experiment (signed poison)
  poisonedrag_eval.py            PoisonedRAG attack reproduction
  regrade_semantic.py            GPT-4 + Claude + SBERT re-grading of responses
  nq_target_overlap.py           Measures the 19/100 string-match false-positive count
  tamper_at_scale.py             10K and 100K corpus tamper detection
  tail_latency.py                p50/p95/p99 verification latency at scale
  ed25519_latency.py             Per-operation latency benchmark
  concurrency_eval.py            Lock contention under N writers
  storage_overhead.py            Per-entry storage, JSON + binary attestations
  multi_embedder_latency.py      Crypto overhead across MiniLM, mpnet, BGE
  bm25_retrieval_agnostic.py     BM25 retrieval + admission filter
  ann_rebuild_compat.py          ANN index rebuild from canonical entries
  live_index_embed_resume.py     Resumable embedding of the full NQ corpus
  live_index_eval.py             Full-corpus live-index validation
  generate_real_figures.py       Publication figures from JSON outputs

Quick start

# Python 3.10+, uv recommended
uv sync
uv run --with pytest python -m pytest tests/test_memproof.py -q

# Smoke test the protocol
uv run python -c "
from memproof.store import MemProofStore
from memproof.crypto.attestation import TrustLevel

def embed(text): return [hash(text) % 1000 / 1000.0] * 384
store = MemProofStore(embed_fn=embed)
store.register_source('wiki', TrustLevel.HIGH)
store.ingest('Paris is the capital of France', source_id='wiki')
print('ingested:', store.entry_count, 'entries; root hex:', store.merkle_root.hex()[:16])
"

Datasets and outputs

This repository ships code and experiment outputs, not raw corpora.

• Outputs. Every evaluation/*.json file is a committed experiment output. The reported numbers are read back from these files, so the results are inspectable without rerunning the full pipeline.
• Datasets. The NQ and RealtimeQA question sets and their per-query retrieval contexts come from the RobustRAG preprocessed splits; the PoisonedRAG adversarial passages and the full ~2.68M-passage NQ corpus come from PoisonedRAG. These are large and are not vendored here — fetch them from the baseline repositories below. Large regenerable artifacts (the live-index embedding cache, build outputs, LaTeX intermediates) are git-ignored.

Baselines

The PoisonedRAG, RobustRAG, and AMemGuard baselines are external projects. Clone them into baselines/ if you want to regenerate the attack passages or rerun the baseline defenses:

git clone https://github.com/sleeepeer/PoisonedRAG.git   baselines/PoisonedRAG
git clone https://github.com/inspire-group/RobustRAG.git baselines/RobustRAG
git clone https://github.com/TangciuYueng/AMemGuard.git  baselines/AMemGuard

Reproducing the results

Every reported number traces to a JSON file in evaluation/. Random seeds are fixed at 42 throughout.

# Run the unit tests
uv run --with pytest python -m pytest tests/test_memproof.py -q
# expected: 40 passed

To rerun an end-to-end experiment, set the relevant API key (OpenAI for GPT-3.5, Anthropic for Claude) and run the script. Keys are read from the environment; none are stored in this repository.

export OPENAI_API_KEY=...        # or ANTHROPIC_API_KEY for the Claude runs
uv run --with openai python evaluation/composition_eval.py \
    --num-queries 100 --output evaluation/composition_results_100_real.json

Headline numbers

All measured end-to-end on the real Ed25519 verify pipeline, seeded at 42. NQ and RealtimeQA use 100 queries each (string-match ASR; semantic judges in the paper confirm the ordering). Storage is measured at three corpus sizes from 1,000 to 100,000 synthetic entries.

Configuration	NQ Acc	NQ ASR	RTQA Acc	RTQA ASR
No Defense	51%	17%	41%	44%
RobustRAG Only	14%	2%	22%	1%
MemProof Only	53%	2%	59%	0%
MemProof + RobustRAG	15%	1%	24%	0%

Cross-family validation on Claude Haiku 4.5 (NQ, 100 queries): MemProof 62% / 3% vs RobustRAG 8% / 1%. Same Pareto pattern.

Full-corpus live index (2,681,468 NQ passages, Claude Haiku 4.5; evaluation/live_index_results.json): the attack is real at scale (90/100 queries poisoned, no-defense ASR 47%); MemProof preserves 64% accuracy where RobustRAG collapses to 3%.

Overhead	JSON attestation	Binary attestation
Crypto / ingest	0.077 ms (1.5% of embedding)	(same)
Storage @ 1K	43.4%	32.1%
Storage @ 10K	51.2%	39.9%
Storage @ 100K	57.0%	45.7%

Honest limitations

1. Trust labels are the unsolved problem. MemProof's guarantees are conditional on correct trust labels at ingestion time. The paper does not solve trust-policy management.
2. Insider threat is out of scope by design. A trusted source that publishes adversarial content bypasses Layer 1. Section 6.5 measures this and reports MemProof degrading to No Defense.
3. Bounded-influence theorem retracted. The earlier draft proposed a theorem that does not hold against embeddings optimized for target queries; the 48x empirical gap is reported in Section 4.2.
4. External-poisoning ASR is a deterministic filter result. Because unsigned poison is never admitted, the MemProof ASR reduction is a provable consequence of admission control rather than an empirical defense effect; the paper's Pareto claim rests on accuracy preservation, not on ASR alone.

Citation

@article{memproof2026,
  title  = {MemProof: Cryptographic Admission Control Against External Corpus Poisoning in Retrieval-Augmented Generation},
  author = {Akekudaga, Philip},
  journal = {Submission to ACM TAISAP},
  year   = {2026},
  url    = {https://github.com/akekulip/MemProof}
}

License

Apache License 2.0. See LICENSE.