five-rag-patterns

Companion repo to the Cloud Perspectives post "Five RAG Architectures in Real Azure Code." Five RAG patterns (Hybrid, GraphRAG, Agentic, Corrective, Multimodal) built and measured against one shared corpus and one shared eval harness, so the comparison in the post is a measured table instead of an argument.

Status

Shared infra, corpus, and eval harness are scaffolded and offline-validated. None of the five patterns are implemented yet, each patterns/<nn>-<name>/adapter.py is a stub that documents what it needs to do and raises NotImplementedError until it does it. python eval/harness.py already runs end to end against all five stubs and reports not_implemented, which is the point: the harness, scoring, and reporting shape are locked in before any pattern-specific code gets written.

The infra has not been deployed. This sandbox has no Azure credentials and no az/bicep CLI beyond a standalone Bicep binary pulled just for offline validation, so every .bicep file here has passed bicep build (see below) but has not been provisioned against a live subscription. Treat resource names, API versions, and the APIM policy XML as reviewed-but-unverified until a real azd up confirms them, the same workflow used on the other repos in this series.

Repo layout

five-rag-patterns/
  infra/              # shared Azure resources, one azd environment for all five patterns
  corpus/              # the fictional Zorgverzekeraar Meridiaan corpus, see corpus/README.md
  eval/                # shared query set, scoring, and the harness that runs both
  patterns/
    01-hybrid-rag/
    02-graphrag/
    03-agentic-rag/
    04-corrective-rag/
    05-multimodal-rag/

Why shared infra instead of five separate deployments

Earlier multi-pattern repos in this series (see azure-architecture-patterns) gave each pattern its own independent Bicep, because those patterns were genuinely unrelated architectures. Here the whole point is a fair comparison: all five patterns retrieve from the same Azure AI Search service, generate with the same Azure OpenAI deployments, and get measured by the same harness. One azd up provisions all of it, and the comparison table in the post reads "same substrate, different pattern," not "different substrate, so who knows."

What gets deployed

Resource	Used by	Notes
Azure AI Search (Standard, semantic ranking on)	Hybrid, Corrective, Multimodal	Hybrid query type does RRF fusion natively
Azure OpenAI: text-embedding-3-large, gpt-4o-mini, gpt-4o	All five	gpt-4o is the vision-capable deployment, used by Agentic's planner and Multimodal's captioning step
Azure AI Document Intelligence	Multimodal	Structured table and form extraction before anything gets embedded
Cosmos DB for NoSQL, local auth disabled	GraphRAG, Multimodal	graph_entities, graph_edges, multimodal_assets containers, data-plane RBAC only
Storage account (Blob)	Multimodal, corpus loading	corpus and multimodal containers
APIM (Basicv2) fronting Azure OpenAI, managed-identity auth	Agentic, and any pattern that wants the gateway	See the note below on why this matters specifically for Agentic RAG
Log Analytics + Application Insights	Eval harness	Optional today, the harness currently logs cost and latency to its own JSON report, not to App Insights yet

No compute (App Service, Functions, Container Apps) is deployed yet. Each pattern runs as a local Python script against the shared services above, authenticated with DefaultAzureCredential, no keys anywhere in the repo. If a pattern later needs to be exposed as a real API, add it to azure.yaml as a service and give it a host.

Why APIM matters for Agentic RAG specifically

The Citadel platform series found that the Azure AI Foundry Agent Service SDK bypasses APIM for its LLM calls, which means any gateway-level governance (rate limits, kill switches, cost attribution) silently stops applying the moment an agent framework's own SDK makes the call instead of your code. Pattern 03 is built with the standard OpenAI SDK pointed at OPENAI_GATEWAY_BASE for exactly that reason. See patterns/03-agentic-rag/adapter.py for where this matters in the implementation.

Deploying the shared infra

Prerequisites: az CLI, azd, and bicep (the az bicep install version is fine, you do not need the standalone binary this sandbox used for offline validation).

azd auth login
azd env new <your-environment-name>
azd env set APIM_PUBLISHER_EMAIL you@example.com
azd up

azd up provisions everything in the table above into a new resource group and prints the outputs (SEARCH_ENDPOINT, OPENAI_GATEWAY_BASE, COSMOS_ENDPOINT, and so on) that the pattern scripts read at runtime.

If provisioning fails

• Pinned model versions rot. Confirmed on the first real azd up: InvalidTemplateDeployment / ServiceModelDeprecated on gpt-4o-mini version 2024-07-18 (retired 2026-03-31). Fixed the same way rag-risk-routing fixed it in April: the openai module now leaves version empty by default so Azure resolves the current default instead of a version that will eventually stop accepting new deployments. Default models are now gpt-5-mini and gpt-5. If a model name itself is not offered in your region, override it rather than guessing:

  azd env set CHAT_MINI_MODEL_NAME gpt-4.1-mini
  az cognitiveservices model list -l <region> -o table   # see what your region actually has

• Account kind is AIServices, not OpenAI. Azure now sells OpenAI models through Foundry as "models sold directly by Azure" (aka.ms/foundry-models-sold-by-azure), same deployment mechanism, different account kind and api-version (2024-10-01). Already fixed in infra/modules/openai.bicep, noted here in case a future model or API version change needs the same kind of update again.
• Model availability by region. Even with the kind and version fixed, gpt-5, gpt-5-mini, and text-embedding-3-large are not necessarily available in every region. Check current availability before setting AZURE_LOCATION, and be ready to move the openai module to a different region than the rest of the stack if needed (the pattern used for SQL in azure-architecture-patterns' pattern 01 is a reasonable template for that).
• A malformed APIM policy fails at ARM validation, not at bicep build. Confirmed on the second real azd up, which got all the way through provisioning Search, Storage, Cosmos DB, Foundry, Document Intelligence, and the APIM service itself, then failed on the APIM policy sub-resource with ValidationError: 'openai' is an unexpected token. The expected token is '>'. The cause: the policy's rewrite-uri template attribute had a raw "/openai/" string literal inside a double-quoted XML attribute, which terminated the attribute early. bicep build has no way to know a Bicep string is meant to be XML, so it compiled clean and the break only showed up against the live ARM API. Fixed by removing the rewrite-uri entirely (it was redundant, APIM's default backend routing already forwards to backend.url + operation.urlTemplate, which was already correct for every operation) and by fixing the catch-all operation's wildcard syntax from the invalid /* to APIM's actual /{*path} template parameter syntax. Run python3 infra/scripts/validate_policy_xml.py after editing infra/modules/apim.bicep to catch this class of bug before the next azd up, not during one.
• RBAC role assignments alone do not enable AAD auth on Azure AI Search. Confirmed by scripts/smoke_test.py after a fully successful deployment: get_service_statistics() returned a flat Forbidden despite both the Search Index Data Contributor and Search Service Contributor roles being correctly assigned. The service still required an API key for data-plane calls because nothing had told it to accept AAD tokens at all. Fixed by adding disableLocalAuth: true to the search service's properties in infra/modules/search.bicep, the same approach already used on Cosmos DB. If you see this again on a resource with roles that look correct, check the resource's own auth settings before re-checking the role assignment.
• APIM Basicv2 region and quota availability. Basicv2 is newer than the classic tiers and is not available in every region or every subscription tier yet. If apim fails to provision, check both region support and subscription-level quota before assuming the template is wrong.
• Cosmos DB RBAC propagation delay. disableLocalAuth: true means the sqlRoleAssignments role assignment is the only way in, and Cosmos DB RBAC assignments can take a few minutes to propagate. A "Forbidden" error immediately after azd up completes is more likely a propagation delay than a broken role assignment, retry before changing anything.
• The APIM policy's rewrite-uri expression. This strips the /openai path prefix APIM adds before forwarding to the backend. It has been reviewed by eye, not exercised against a live gateway. If chat completions or embeddings calls 404 through the gateway, check this policy first with APIM's trace tool before checking anything else.

Testing the deployment

Right after azd up succeeds, before writing any pattern code:

pip install -r requirements.txt
az login   # if you have not already
python3 scripts/smoke_test.py

The script pulls azd env get-values automatically if the outputs aren't already in your shell environment. It checks, in order: Azure AI Search, Azure OpenAI directly (embeddings + a one-word chat completion), Azure OpenAI through the APIM gateway with no client-side auth at all (this is the important one, it's the exact managed-identity policy that broke twice during infra provisioning), Cosmos DB (write, read, delete a throwaway item), Blob Storage (upload, download, delete a throwaway blob), and Document Intelligence (a reachability call). Every check authenticates as you, az login's identity, the same principal every RBAC role assignment in infra/ was granted to. If something fails here, it's the infra, not pattern code that doesn't exist yet.

Running the eval harness

pip install -r eval/requirements.txt   # none yet, stdlib only so far
python eval/harness.py

This runs eval/queries.jsonl against every pattern that has an adapter.py under patterns/, scores retrieval precision/recall and a lexical groundedness heuristic (see eval/scoring.py for what that heuristic does and does not catch), estimates cost from eval/pricing.py, and writes a full JSON report to eval/results/. Run --patterns hybrid,corrective to scope it to specific patterns while the others are still stubs.

Build order

1. Shared infra, corpus, eval harness (this scaffold)

2. Pattern 01, Hybrid RAG. Index builder (patterns/01-hybrid-rag/build_index.py) and adapter (patterns/01-hybrid-rag/adapter.py) are implemented, corpus parsing has an offline unit test suite (patterns/01-hybrid-rag/tests/test_corpus_loader.py), and index construction was validated offline (object build + JSON serialization) against the actual installed SDK version before ever touching a live endpoint. Not yet run against the live index or scored with the eval harness for real, that needs your credentials, run:

   python3 -m unittest patterns/01-hybrid-rag/tests/test_corpus_loader.py -v
   python3 eval/harness.py --patterns hybrid

   The first build's setup() call creates the index and embeds all 38 corpus chunks (3 embedding batches), which costs a small amount of Azure OpenAI usage every time it reruns, upload_documents upserts by key so re-running is safe, just not free.

3. Patterns 02 and 03, GraphRAG and Agentic RAG, reusing the APIM/OpenAI SDK governance pattern from Citadel. Pattern 02, GraphRAG is implemented: patterns/02-graphrag/graph_loader.py (pure Python, community detection via networkx Louvain, entity linking, subgraph walking, unit tested in patterns/02-graphrag/tests/) and patterns/02-graphrag/build_graph.py (Cosmos DB sync, community summarization via the chat-mini deployment). Community summaries are stored back in the graph_entities container as synthetic entities (entityType: "community_summary") rather than a fourth Cosmos container, no infra redeploy needed. setup()'s community summarization is a real, one-time Azure OpenAI cost per run that isn't tracked in any single query's model_calls (the harness's cost accounting is per-query by design), watch stdout for that number rather than expecting it in the eval report. Run:

   python3 -m unittest patterns/02-graphrag/tests/test_graph_loader.py -v
   python3 eval/harness.py --patterns graphrag

   Confirmed live: q06-q08 (GraphRAG's own territory) scored recall=1.00, the other 12 shared queries scored recall=0.21, exactly the "different failure mode, not a worse pattern" story the post is built around. The harness's eval/harness.py output now prints this own-target-vs-other breakdown automatically for every pattern, added after this result made it obvious the aggregate numbers alone were misleading.

   Pattern 03, Agentic RAG is implemented: patterns/03-agentic-rag/rules.py (pure Python, the actual dental waiting-period and annual-maximum arithmetic from the policy docs, encoded as code rather than left to an LLM's arithmetic, unit tested including the exact q09 and q10 scenarios) and patterns/03-agentic-rag/tools.py (four tools: search reuses pattern 01's index, provider lookup reuses pattern 02's graph, two rule-check tools call straight into rules.py). patterns/03-agentic-rag/adapter.py runs a standard OpenAI function-calling loop through OPENAI_GATEWAY_BASE (APIM), not the Foundry Agent Service SDK, capped at MAX_ITERATIONS = 5 with the cap-hit outcome recorded separately from a clean answer. Run:

   python3 -m unittest patterns/03-agentic-rag/tests/test_rules.py patterns/03-agentic-rag/tests/test_tools.py -v
   python3 eval/harness.py --patterns agentic

4. Patterns 04 and 05, Corrective RAG and Multimodal RAG, the newest territory. Pattern 04, Corrective RAG is implemented: patterns/04-corrective-rag/grading.py (pure Python: grading prompt construction, tolerant JSON response parsing with a safe "ambiguous" fallback on anything unparseable, and chunk filtering, unit tested against the exact q03 stale-FAQ-vs-current-policy scenario) and patterns/04-corrective-rag/adapter.py (retrieve, grade, then answer with only the grader-preferred chunks / rewrite and retry up to REWRITE_CAP = 2 / decline outright, reusing pattern 01's index). retrieved_doc_ids reflects what the pattern actually used after filtering, not everything it fetched, so a working grader shows distractor_hits=0 on q03-q05 even though the stale FAQ was retrieved. Confirmed live: precision=1.00 and recall=1.00 on q03-q05, versus Hybrid RAG's overall precision of 0.35 with no grading step at all. Run:

   python3 -m unittest patterns/04-corrective-rag/tests/test_grading.py -v
   python3 eval/harness.py --patterns corrective

   Pattern 05, Multimodal RAG is implemented: patterns/05-multimodal-rag/render_assets.py (renders the table CSV and the claim form directly as PNGs with Pillow, no SVG parsing, no native rendering library dependency, portable to Windows without a WSL/apt fallback) and patterns/05-multimodal-rag/build_multimodal_index.py (Document Intelligence table extraction for the table, direct vision captioning for the form, both embedded into the same shared index as new documents, doc_type distinguishes them rather than a new field or a new index). patterns/05-multimodal-rag/validation.py checks both generated captions against corpus/multimodal/captions.json's ground truth at setup() time, printing a warning (not failing the run) if a field goes missing, exactly the check the corpus README promises. run() is identical to Hybrid RAG's retrieve-and-answer once the index has multimodal entries, no separate query-time logic. Run:

   python3 -m unittest patterns/05-multimodal-rag/tests/test_multimodal.py -v
   python3 eval/harness.py --patterns multimodal

5. Run the harness across all five for real, write the post from the numbers it produces. All five patterns now have real setup()/run() implementations, none are stubs anymore, python3 eval/harness.py (no --patterns filter) runs the full comparison in one pass.