MOD-20119: Lens developer docs

content/develop/ai/context-engine/_index.md

@@ -21,6 +21,7 @@ Redis Iris eliminates the infrastructure burden of building context-aware AI age
   {{< tile-card icon="images/dev/icons/icon-redis-agent-memory-64-duotone.png" title="Agent Memory" description="Persistent short-term and long-term memory across agent interactions" url="/develop/ai/context-engine/agent-memory" >}}
   {{< tile-card icon="images/dev/icons/icon-redis-context-retriever-64-duotone.png" title="Context Retriever" description="Governed, schema-first data access tools for agents" url="/develop/ai/context-engine/context-retriever" >}}
   {{< tile-card icon="images/dev/icons/icon-redis-iris-64-duotone.png" title="Agent Playbooks" description="Publish reusable answers and procedures for agents" url="/develop/ai/context-engine/agent-playbook" >}}
+  {{< tile-card icon="images/dev/icons/icon-redis-iris-64-duotone.png" title="Lens" description="Debug agent answers by tracing the context behind each turn" url="/develop/ai/context-engine/lens" >}}
 </div>
 
 ## What is Redis Iris?

content/develop/ai/context-engine/lens/AGENTS.md

@@ -0,0 +1,54 @@
+# Lens docs — conventions for AI-assisted editing
+
+These pages document **Lens**, the Redis Iris observability tool for AI agents. Read the
+repository-root `AGENTS.md` first for style and site mechanics. This file adds terminology,
+disclosure, and framing rules for this directory, and wins where the two conflict.
+
+## Product names
+
+| Use | Not | Note |
+| --- | --- | --- |
+| Lens | Iris Lens, Redis Lens, Agent Lens | Every mention, including titles. |
+| Context Gateway | Context Retriever | In prose. API literals such as `GetContextRetrieverResult` stay as they are. |
+| Agent Memory, Agent Playbooks | RAM, Playbook service | The Iris services Lens traces. |
+| Iris services | Iris products | Agent Memory, Agent Playbooks, and Context Gateway as a group, matching the other Iris pages. |
+| preview | Beta | Lens availability. |
+
+`redis-iris` (`redis_iris.lens`) is the instrumentation package. `redis-agent-lens`
+(`redis_agent_lens`) is the read client. They read different key variables:
+`IRIS_LENS_API_KEY` for export, `LENS_API_KEY` for reads. Never swap them.
+
+## The source is private
+
+Lens source is private, so a file path, repository URL, or "see the implementation"
+pointer is a dead end for readers. Do not link to source code.
+
+Leave out implementation internals: storage design, internal service names, internal span
+and attribute names (`iris.lens.*`, `iris.component.*`), cursor encoding, and design
+history. Document the public SDK calls, settings, and API fields.
+
+## Present what Lens does
+
+Say where a capability is present, not where it's missing. Write "Playbook discovery
+results include their similarity score", not "Agent Memory recalls have no score". Name
+the cases that have a capability, so readers don't assume it applies everywhere.
+
+Leave unshipped and deferred features out entirely. The availability banner is the only
+preview notice; don't add lists of gaps or "coming soon" notes.
+
+## Deployments
+
+Lens runs on Redis Cloud and self-managed. Pages are deployment-neutral: the same steps
+work on both. Name a deployment only where behavior differs.
+
+Readers get endpoints and keys from their administrator. Until the operator pages exist,
+say so without linking to them, and mark each handoff with a TODO comment.
+
+## Terms
+
+- A **turn** is one agent exchange and one trace. Its trace ID is the turn ID.
+- A **session** groups turns, and a **thread** is a conversation within a session. Readers
+  choose session and thread IDs.
+- A **step** is one recorded operation in a turn, such as a memory recall or a model call.
+  Use "step" in prose; "span" only where the reader works with OpenTelemetry directly.
+- **Fetched** is what a source returned. **Used** is what entered the prompt.

content/develop/ai/context-engine/lens/CLAUDE.md

@@ -0,0 +1 @@
+@AGENTS.md

content/develop/ai/context-engine/lens/_index.md

@@ -0,0 +1,61 @@
+---
+Title: Lens
+alwaysopen: false
+categories:
+- docs
+- develop
+- ai
+description: Debug AI agent answers by tracing the context behind each turn, from retrieval to prompt.
+linkTitle: Lens
+hideListLinks: true
+weight: 27
+bannerText: Lens is currently available in preview. Features and behavior are subject to change.
+bannerChildren: true
+---
+
+## What is Lens?
+
+Lens is a Redis Iris observability tool that traces your AI agent's turns. A turn is one agent exchange, such as answering a single message. For each turn, Lens records what each source returned, such as a memory recall or a retrieval, and what entered the prompt. It also records which model and tool calls ran, how long each step took, and how the turn ended.
+
+When your agent gives a wrong answer, Lens shows where it went wrong. For example, you might see that the right memory was recalled but dropped from the prompt, or that a retrieval came back empty.
+
+<div class="grid grid-cols-1 md:grid-cols-3 gap-6 my-8">
+  {{< tile-card icon="images/icon_logo/icon-data-structures-64-midnight.png" title="Concepts" description="Learn how sessions, turns, steps, and fetched and used context work" url="/develop/ai/context-engine/lens/concepts" >}}
+  {{< tile-card icon="images/dev/icons/icon-redis-code-64-duotone.png" title="Quickstart" description="Trace your first turn and read it back" url="/develop/ai/context-engine/lens/quickstart" >}}
+  {{< tile-card icon="images/dev/icons/icon-redis-code-64-duotone.png" title="Agent integration" description="Trace agents built with LangGraph, the OpenAI Agents SDK, and more" url="/develop/ai/context-engine/lens/agent-integration" >}}
+</div>
+
+## Why use Lens?
+
+- **See why an agent answered the way it did:** Compare what each source returned with what entered the prompt, turn by turn.
+- **Iris services trace themselves:** Agent Memory, Agent Playbooks, and Context Gateway calls appear in the turn with no extra code.
+- **Model calls traced automatically:** Calls through OpenAI, Anthropic, Amazon Bedrock, and LangChain clients appear with their prompts, completions, and token counts.
+- **Built on OpenTelemetry:** Lens joins the tracer your application already uses, and Iris service steps reach your existing observability tools as well.
+- **You control content capture:** Capture prompts and retrieved text for debugging, or keep only IDs, hashes, and counts.
+- **Background export:** Lens exports in the background, so your agent never waits for it. If an export fails, the error stays out of your agent's code.
+
+## How it works
+
+1. Your agent opens a turn for each message with the Redis Iris software development kit (SDK).
+1. The SDK records each step in the turn: retrievals, memory recalls, prompt assembly, model calls, tool calls, and Iris service calls.
+1. Lens assembles the turn and groups it into sessions by agent.
+1. You inspect the turn in the Lens user interface (UI) or query it with the read application programming interface (API).
+
+![Your agent and the Iris services it calls record steps through the Redis Iris SDK. The SDK exports each turn to Lens, where you inspect it in the Lens UI or query it with the read API.](/images/ai/context-engine/lens-how-it-works.svg)
+{width="700"}
+
+## Get started
+
+1. Get your Lens endpoints and keys from your administrator.
+   <!-- TODO: link the operator guides for creating an instance and keys when they exist. -->
+1. [Trace your first turn](/content/develop/ai/context-engine/lens/quickstart.md).
+
+## Use Lens
+
+- [Instrument an agent](/content/develop/ai/context-engine/lens/instrument.md): record turns, retrievals, prompt assembly, and custom steps.
+- [Integrate with an agent framework](/content/develop/ai/context-engine/lens/agent-integration.md): trace LangGraph, OpenAI Agents SDK, Google Agent Development Kit (ADK), and your own loop.
+- [Inspect sessions](/content/develop/ai/context-engine/lens/inspect-sessions.md): find a session and read its turns in the Lens UI.
+- [Configure Lens](/content/develop/ai/context-engine/lens/configure.md): settings, keys, content capture, and OpenTelemetry.
+- [Query sessions and turns](/content/develop/ai/context-engine/lens/read-api.md): read turns from code.
+- [Limits and errors](/content/develop/ai/context-engine/lens/limits.md): request limits, export behavior, and troubleshooting.
+- [API reference](/content/develop/ai/context-engine/lens/api-reference.md): the read API operations and data models.

content/develop/ai/context-engine/lens/agent-integration.md

@@ -0,0 +1,101 @@
+---
+alwaysopen: false
+categories:
+- docs
+- develop
+- ai
+description: Trace LangGraph, OpenAI Agents SDK, Google ADK, and your own agent loop with Lens, across Agent Memory, Agent Playbooks, and Context Gateway.
+hideListLinks: true
+linktitle: Agent integration
+title: Integrate Lens with agent frameworks
+weight: 20
+---
+
+To trace an agent run in Lens, open a turn around it. Each step the agent takes inside the run, such as a memory recall or a model call, is recorded in that turn. For more about turns and steps, see [Lens concepts](/content/develop/ai/context-engine/lens/concepts.md).
+
+Before you start, get the Lens connection values from the [quickstart](/content/develop/ai/context-engine/lens/quickstart.md#configure-the-sdk). The framework examples also need a model provider key, such as `OPENAI_API_KEY`.
+
+## Where the turn opens
+
+| Framework | Open the turn around |
+| --------- | -------------------- |
+| LangGraph | `app.invoke()` or `app.ainvoke()`. With an Iris service attached, `attach()` opens the turn for you. |
+| OpenAI Agents SDK | `Runner.run()` |
+| Google Agent Development Kit (ADK) | `runner.run_async()` |
+| Your own loop | The handler for one message |
+
+To trace model calls inside the turn, install `redis-iris[llm-tracing]` and call `lens.init()` at startup. See [Trace model calls](/content/develop/ai/context-engine/lens/instrument.md#trace-model-calls).
+
+## Trace one turn across Iris services
+
+When your agent uses Agent Memory, Agent Playbooks, and Context Gateway, a single Lens turn shows how each Iris service contributed to the agent's response. This example needs the connection values for each Iris service.
+
+`workflows.turn()` runs each service's context step before the model and its write-back after. Every service call joins the turn:
+
+{{< clients-example set="iris_lens_iris_products" step="iris_products_turn" description="" lang_filter="Python (Iris)" footer="hide" />}}
+
+The turn shows the memory recall and the Playbook discovery, then any Context Gateway tool calls your agent makes, and the write-back. In asynchronous code, `workflows.aturn()` runs the services' context steps at the same time. The outer `lens.turn()` sets the outcome. When a turn is already open, `workflows.turn()` and `attach()` add their steps to it rather than starting another.
+
+## LangGraph
+
+Install LangGraph and the LangChain OpenAI integration:
+
+```bash
+python -m pip install 'redis-iris[llm-tracing]' langgraph langchain-openai
+```
+
+Wrap the graph run in `lens.turn()`. LangGraph's thread ID identifies the conversation, so pass it as the Lens session ID:
+
+{{< clients-example set="iris_lens_langgraph" step="langgraph_turn" description="" lang_filter="Python (Iris)" footer="hide" />}}
+
+Each model call in the graph joins the turn with its token counts.
+
+### Attach an Iris service
+
+When the graph uses Agent Memory or Agent Playbooks, `attach()` adds the service's context before your first node and its write-back after your last. With Lens on, every run of an attached graph is one turn. The session comes from `thread_id` and the user from `user_id` in the run's configuration:
+
+{{< clients-example set="iris_lens_langgraph_attach" step="langgraph_attach" description="" lang_filter="Python (Iris)" footer="hide" />}}
+
+`attach()` needs the `langgraph` extra: `python -m pip install 'redis-iris[langgraph]'`.
+
+## OpenAI Agents SDK
+
+Install the OpenAI Agents SDK:
+
+```bash
+python -m pip install 'redis-iris[llm-tracing]' openai-agents
+```
+
+Wrap `Runner.run()` in `lens.turn()`:
+
+{{< clients-example set="iris_lens_openai_agents" step="openai_agents_turn" description="" lang_filter="Python (Iris)" footer="hide" />}}
+
+The OpenAI Agents SDK calls models through the OpenAI client, so each model call joins the turn with its token counts.
+
+## Google ADK
+
+Install the Google ADK:
+
+```bash
+python -m pip install redis-iris google-adk
+```
+
+Wrap the `runner.run_async()` loop in `lens.turn()`:
+
+{{< clients-example set="iris_lens_google_adk" step="google_adk_turn" description="" lang_filter="Python (Iris)" footer="hide" />}}
+
+The turn records the agent run, its outcome, and every Iris service call the agent makes. The example uses a Gemini model and needs `GOOGLE_API_KEY`.
+
+## Your own agent loop
+
+Wrap the handler for one message in `lens.turn()`, and set the outcome on each path:
+
+{{< clients-example set="iris_lens_own_loop" step="own_loop_turn" description="" lang_filter="Python (Iris)" footer="hide" />}}
+
+Each call to `handle()` records one turn under the session and user IDs you pass. The turn is marked `completed` when the model answers, or `failed` when the call raises. Record your retrievals and prompt assembly inside the same turn to see which context reached the model.
+
+## Next steps
+
+- [Instrument an agent](/content/develop/ai/context-engine/lens/instrument.md): record retrievals, memory recalls, prompt assembly, and custom steps.
+- [Inspect sessions](/content/develop/ai/context-engine/lens/inspect-sessions.md): read the turns you traced in the Lens user interface (UI).
+- [Query sessions and turns](/content/develop/ai/context-engine/lens/read-api.md): read turns from code.

content/develop/ai/context-engine/lens/api-reference.md

@@ -0,0 +1,18 @@
+---
+Title: Lens read API
+linkTitle: API reference
+layout: apireference
+type: page
+weight: 80
+params:
+  sourcefile: ./openapi.json
+  sortOperationsAlphabetically: false
+  schemaDefinitionsTagName: Data models
+  showPageContent: true
+---
+
+Use the Lens read application programming interface (API) to list agents and sessions, and to read turns, their captured content, and Context Gateway tool results. Every operation is a `GET` request scoped to your instance.
+
+[Download the OpenAPI contract]({{< relref "/develop/ai/context-engine/lens/api-reference/" >}}openapi.json) to generate clients or inspect schemas.
+
+See [Query sessions and turns](/content/develop/ai/context-engine/lens/read-api.md) for examples, and [Limits and errors](/content/develop/ai/context-engine/lens/limits.md) for request limits and error responses.