Skip to content

graphrag.md

Latest commit

 

History

History
214 lines (167 loc) · 9.56 KB

graphrag.md

File metadata and controls

214 lines (167 loc) · 9.56 KB
Raw

GraphRAG architecture (Step 2.9)

Overview

rag-graphrag implements the three primitives from Edge et al. 2024, "From Local to Global: A Graph RAG Approach to Query-Focused Summarization":

  1. Community detection — partition the knowledge graph into densely connected sub-graphs.
  2. Per-community summarisation — produce one LLM-generated "community report" per community, used at retrieval time for query-against-summary scoring.
  3. Graph-aware retrieval — at query time, score the query against community summaries, then expand from the top communities' key entities into the graph.

The package depends only on rag-core and networkx. No new SPIs were added in Step 2.9 — community detection runs over the existing GraphRetrievalBackend SPI (via the NetworkXGraphStore.graph escape hatch for the in-process case), LLM calls run through the existing rag_core.spi.llm.LLM SPI, and the retriever's output is a plain list[ChunkRef] that drops into HybridRetriever.

Why GraphRAG

Hybrid RRF retrieval (Step 2.5) is great when the answer is "a few chunks that locally mention the query." It struggles with two recurring patterns:

Query pattern Why hybrid retrieval misses
"What's the relationship between X and Y?" Vector + keyword surface chunks about X or Y, not the connection. The connection lives in the graph edge, not the chunk text.
"Summarise everything we know about topic Z" The relevant chunks are spread across documents; the topical glue is structural, not textual.

GraphRAG addresses both by making the community — a learned sub-graph cluster — a first-class retrieval anchor. Community summaries serve as topical "table of contents" entries; the graph edges then carry the locality.

Design decisions

1. Louvain by default, Leiden behind a flag

Algorithm Pro Con Default?
Louvain (Blondel et al. 2008) Ships with NetworkX 3.x; no extra; deterministic with a seed. Modularity-optimising but does not guarantee well-connected communities. Yes — pure-Python, zero install cost.
Leiden (Traag et al. 2019) Strictly higher modularity, well-connected communities guaranteed. Pulls python-igraph (C extension) + leidenalg. Behind [leiden] extra.

For most real-world graphs the gap is small; for production-scale graphs (> 100 K nodes) Leiden is noticeably faster and better, so the extra is worth installing. We picked Louvain as default because the demo / laptop / CI install footprint matters more than a few percent of modularity quality at our current scale.

We deliberately do not ship spectral clustering, label propagation, or stochastic block models in V1 — they're either much slower or much weaker than Louvain on the graph shapes we expect (entity-relation graphs with heavy-tailed degree distributions).

2. Hierarchical detection deferred

Leiden supports hierarchical community detection by recursively applying the algorithm to the reduced graph. V1 returns flat partitions only (level=0, parent_id=None) because:

  • The retriever's community scoring + seed-collection logic is much simpler at one level.
  • Hierarchical communities are most valuable for summary-of-summaries workflows, which Phase 3's gateway will own.
  • We can ship hierarchy later without breaking the schema — level and parent_id are already on Community.

3. Strict failure mode

Like RerankPipeline (Step 2.7) and ContextPacker (Step 2.8), every stage that fails raises GraphRAGError rather than silently degrading. The one narrow exception: when a node has no chunk_ids property, the adapter silently skips it — that's a legitimate "isolated entity" state, not an error.

4. The community → chunk join lives in node properties

Each entity node carries properties["chunk_ids"] — a list of the chunks it appears in. The graph-indexer (typically an offline NER / relation-extraction pass) is responsible for stamping this. We don't introduce a separate node_chunks table because:

  • The graph backends we ship (Neo4j, Memgraph, NetworkX) all support list-valued properties natively.
  • A separate table would mean a second backend RPC at retrieval time.
  • Storing the join in the graph means it travels with the node — no consistency issues during graph migration.

5. Token-overlap scoring for community → query

The current scorer is set-overlap between query tokens and (summary, key_entities) tokens, with a 2× weight on key-entity matches. This is deliberately simple:

  • Pure-Python, deterministic, fast.
  • Requires no per-tenant embedder config — rag.yaml corpus routing doesn't land until Step 3.5.
  • Gives us a working pipeline today; the cosine-similarity path arrives with Phase 3 alongside the rest of the per-tenant embedding plumbing.

Phase 3 will swap this for an Embedder-backed cosine scorer behind a constructor argument.

6. Prompt design

The system prompt demands strict JSON to keep parsing deterministic. Real LLMs (GPT-4 class) reliably obey; weaker LLMs may not. Our parser:

  1. Strips ``` fences if present.
  2. Tries json.loads.
  3. On parse failure, falls back to using the raw text as the summary and the first three node IDs as key entities — this is what lets smoke tests work with NoopLLM.
  4. On JSON parse success but schema violation, raises GraphRAGError.

The fallback is documented as "best effort for smoke flows" — real deployments using non-instruction-following LLMs should validate summary quality offline.

Performance & observability

Three OTel spans, one per stage:

Span Key attributes
graphrag.detect algorithm, nodes_in, edges_in, resolution, communities_out, elapsed_ms
graphrag.summarize community_id, nodes_n, edges_n, model, input_tokens, output_tokens, elapsed_ms
graphrag.retrieve query_length, community_top_k, hops, communities_matched, seeds_n, neighbors_n, chunks_out, elapsed_ms

All attributes carry the rag. prefix and follow the same naming conventions as Step 2.7 (rerank) and Step 2.8 (pack) for dashboard consistency.

Tenant isolation

  • Detection: callers pass a nx.Graph already filtered to one tenant (use tenant_subgraph() helper). The resulting Community.tenant_id is set from ctx.tenant_id.
  • Summarisation: cross-tenant calls raise GraphRAGError.
  • Store: every read/write checks ctx.tenant_id matches the community's declared tenant; cross-tenant lookups return None rather than leaking.

PolicyEngine boundary

rag_graphrag does not call PolicyEngine directly:

  • CommunityDetector runs over already-policed data (the caller has filtered the graph to ACL-visible nodes before invoking detection).
  • CommunitySummarizer issues LLM calls at index-build time over already-policed graph nodes — the Phase 3 gateway is the canonical PolicyEngine consumer for LLM-call governance. The summariser is on the tests/policy/coverage.py allowlist with a justifying comment.
  • GraphRAGRetriever returns ChunkRef lists; the downstream HybridRetriever is the canonical read_chunk PolicyEngine consumer.

Composition with HybridRetriever

GraphRAGRetriever returns list[ChunkRef] — exactly the shape that HybridRetriever's graph_seeds + graph_adapter path emits. Two integration modes:

  1. Adapter-only mode — pass graphrag_adapter as the graph_adapter to HybridRetriever and let HybridRetriever drive expand() directly. Best for chunk-graph projections where node_id ≈ chunk_id.
  2. Pre-fan-out mode — call GraphRAGRetriever.retrieve() separately, feed the result to HybridRetriever as a pre-built ranking (forthcoming Phase 3 API). Best for entity-graph projections where community scoring is the primary signal.

V1 supports mode 1. Mode 2 lands with the Phase 3 corpus router (Step 3.5) which will own per-query backend selection.

What's deliberately deferred

  • Hierarchical communities + summary-of-summaries — Phase 3 gateway.
  • Persistent community store (Postgres / Redis) — Step 3.5 corpus router will own backend wiring.
  • Embedder-cosine community scoring — Phase 3 (needs per-tenant embedder config).
  • Drift monitoring for community quality — Step 5.5 drift monitors.
  • /v1/communities REST endpoint + admin UI — Phase 3.
  • LLM contradiction reconciliation across communities — Step 5.x hallucination guard.
  • Edge weights in community detection — V1 treats every edge uniformly; weighted Louvain via the weight parameter is a one-line change once the indexer starts emitting edge weights.

Reviewer checklist

When changing rag-graphrag or its consumers:

  • Did you preserve the frozen-Pydantic invariant on Community / CommunitySummary? These cross package boundaries and downstream consumers may rely on immutability.
  • If you added a new pipeline stage, does it open an OTel span with attributes namespaced under rag.graphrag.*?
  • If the detector / summariser changed, are the tenant isolation checks still in the path (subgraph filter for detection; tenant-mismatch raise for summarisation + store)?
  • If the retriever's seed-collection logic changed, do top-K communities still dedupe shared entities across communities?
  • If you added a new optional dependency, is it behind an extra in pyproject.toml and lazy-imported inside the function that needs it (so import rag_graphrag still succeeds without the extra)?