Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions content/operate/iris/langcache/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ LangCache is a semantic caching service available as a REST API that stores LLM

For more information about how LangCache works, see the [LangCache overview]({{< relref "/develop/ai/context-engine/langcache" >}}).

To deploy LangCache on your own Kubernetes infrastructure instead of Redis Cloud, see [self-managed LangCache]({{< relref "/operate/iris/langcache/self-managed" >}}).

## LLM cost reduction with LangCache

{{< embed-md "langcache-cost-reduction.md" >}}
Expand Down
84 changes: 84 additions & 0 deletions content/operate/iris/langcache/self-managed/_index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
---
Title: Self-managed LangCache
alwaysopen: false
categories:
- docs
- operate
- iris
description: Deploy, configure, secure, and operate LangCache on a self-managed Kubernetes cluster.
linkTitle: Self-managed
weight: 40
hideListLinks: true
---

LangCache is a semantic caching service that stores LLM responses for faster,
cheaper retrieval. Applications send prompts to LangCache, which either returns a
cached response for a semantically similar prior prompt or calls out to your
embedding provider and stores a new entry when there is no match.

This guide covers deployment, configuration, security, and operations for
self-managed LangCache.

The [LangCache API]({{< relref "/develop/ai/context-engine/langcache/api-reference" >}})
is the shared Data Plane API for Redis Cloud and self-managed deployments. The
[Control Plane API reference]({{< relref "/operate/iris/langcache/self-managed/control-plane-api-reference" >}})
documents the self-managed admin endpoints for caches.

{{< note >}}
Self-managed LangCache is available as a private preview. You need a license
key to deploy it. Contact your Redis representative or
[contact sales](https://redis.io/contact/).
{{< /note >}}

## What you are deploying

One `helm install` of the `langcache` chart always creates the Data Plane and
the Control Plane, plus either a bundled or an external Identity Service.
There is no lighter-weight install of only the Data Plane for self-managed
LangCache. Every cache is created and managed through the Control Plane, and
every Data Plane request is authenticated by the Identity Service.

| Component | Purpose | Default service |
| --- | --- | --- |
| LangCache Data Plane | Cache-scoped runtime API for set, search, and flush. | `langcache:9000` |
| LangCache Control Plane | Admin API for creating and managing caches. | `langcache-controlplane:9100` |
| Identity Service | Issues and validates the agent keys the Data Plane requires. Bundled by the chart (default) or an external instance your suite already runs. | `langcache-identity-service:9200` (bundled mode) |
| Cache Redis | Holds cache entries and RediSearch vector indexes. Registered by ID in the Control Plane's database registry — the Data Plane has no database registry of its own. | Customer-provided |
| Metadata Redis | Holds Control Plane cache records. Can be the same Redis instance as Cache Redis, in a separate keyspace. | Customer-provided |

### How the components work together

1. Platform admins use the Control Plane to create and manage caches,
selecting a Cache Redis target by `databaseId` from the Control Plane's
own database registry.
1. The Control Plane writes cache records to Metadata Redis, including the
resolved Redis URLs for that cache, and synchronously provisions the
RediSearch vector index in Cache Redis.
1. Platform admins mint agent keys through the Identity Service, granting
`lc-cache:<cache-id>` permissions.
1. Agents and applications call the Data Plane with a cache ID and an agent
key.
1. The Data Plane introspects the key against the Identity Service, reads
the cache's metadata (including its Redis URLs) from Metadata Redis, and
reads or writes entries in Cache Redis.

### API surfaces

All Data Plane APIs are scoped to a cache. A cache is the logical isolation
boundary for cached entries.

| API surface | Endpoint prefix | Purpose |
| --- | --- | --- |
| Cache entries | `/v1/caches/{cacheId}/entries` | Set, search, and delete cached entries. |
| Cache flush | `/v1/caches/{cacheId}/flush` | Flush all entries in a cache. |
| Cache health | `/v1/caches/{cacheId}/health` | Cache-scoped health status. |
| Control Plane | `/v1/caches`, `/v1/embedding-providers` | Self-managed administration for caches. |
| Identity Service | `/v1/api-keys` | Mint, list, update, revoke, and rotate agent keys and their cache grants. |

The [LangCache API]({{< relref "/develop/ai/context-engine/langcache/api-reference" >}})
reference does not yet document cache health; for that, use
[API examples]({{< relref "/operate/iris/langcache/self-managed/api-examples" >}})
until the shared schema is updated.

Start with [prerequisites]({{< relref "/operate/iris/langcache/self-managed/prerequisites" >}}),
then follow [Deploy self-managed LangCache]({{< relref "/operate/iris/langcache/self-managed/deploy" >}}).
226 changes: 226 additions & 0 deletions content/operate/iris/langcache/self-managed/api-examples.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,226 @@
---
Title: Self-managed API examples
alwaysopen: false
categories:
- docs
- operate
- iris
description: Use curl examples with the LangCache self-managed Control Plane, Identity Service, and Data Plane APIs.
linkTitle: Self-managed API examples
weight: 50
hideListLinks: true
---

These examples show self-managed Control Plane, Identity Service, and Data
Plane requests. They assume agent-key authentication as described in
[Authentication and authorization]({{< relref "/operate/iris/langcache/self-managed/authentication" >}}).

For the complete shared Data Plane schema, see the
[LangCache API]({{< relref "/develop/ai/context-engine/langcache/api-reference" >}}).
For the self-managed admin schema, see the
[Control Plane API reference]({{< relref "/operate/iris/langcache/self-managed/control-plane-api-reference" >}}).

## Control Plane API examples

Set variables:

```bash
CP_URL="http://localhost:9100"
LC_ADMIN_TOKEN="<admin-token>"
```

List caches:

```bash
curl -sS "$CP_URL/v1/caches" \
-H "Authorization: Bearer $LC_ADMIN_TOKEN"
```

Create a cache:

```bash
curl -sS -X POST "$CP_URL/v1/caches" \
-H "Authorization: Bearer $LC_ADMIN_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"name": "my-cache",
"databaseId": "cache-primary",
"defaultSearchThreshold": 0.9,
"defaultTtlMillis": -1,
"attributes": []
}'
```

Response:

```json
{
"cacheId": "0123456789abcdef0123456789abcdef"
}
```

`databaseId` must match an entry in the Control Plane's configured
`databases` registry. `defaultSearchThreshold` is a float between 0 and 1.
`defaultTtlMillis` accepts `-1` or `0` for no expiration, or a positive
number of milliseconds.

Get a cache:

```bash
curl -sS "$CP_URL/v1/caches/<cache-id>" \
-H "Authorization: Bearer $LC_ADMIN_TOKEN"
```

Response fields include `status` (`PROVISIONING`, `READY`, or
`UNAVAILABLE`), the deployment's `embeddingProvider`/`embeddingModel`/
`embeddingDimensions`, and the resolved `databaseName`.

Update a cache:

```bash
curl -sS -X PATCH "$CP_URL/v1/caches/<cache-id>" \
-H "Authorization: Bearer $LC_ADMIN_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"defaultSearchThreshold": 0.85
}'
```

Flush a cache's entries without deleting the cache:

```bash
curl -sS -X DELETE "$CP_URL/v1/caches/<cache-id>/entries" \
-H "Authorization: Bearer $LC_ADMIN_TOKEN"
```

Delete a cache:

```bash
curl -sS -X DELETE "$CP_URL/v1/caches/<cache-id>?flush=true" \
-H "Authorization: Bearer $LC_ADMIN_TOKEN"
```

List the deployment's configured embedding providers and models:

```bash
curl -sS "$CP_URL/v1/embedding-providers" \
-H "Authorization: Bearer $LC_ADMIN_TOKEN"
```

## Identity Service API examples

Set variables:

```bash
IDS_URL="http://localhost:9200"
IDS_CONTROL_TOKEN="<identity-service-control-token>"
```

Mint an agent key scoped to one cache:

```bash
curl -sS -X POST "$IDS_URL/v1/api-keys" \
-H "Authorization: Bearer $IDS_CONTROL_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"name": "my-agent-key",
"grants": [
{
"product": "langcache",
"resourceType": "lc-cache",
"resourceId": "<cache-id>",
"actions": ["read", "write"]
}
]
}'
```

Response:

```json
{
"keyId": "0123456789abcdef0123456789abcdef",
"token": "<agent-key>",
"createdAt": 1780000000
}
```

Rotate it later:

```bash
curl -sS -X POST "$IDS_URL/v1/api-keys/<key-id>/rotate" \
-H "Authorization: Bearer $IDS_CONTROL_TOKEN"
```

## Data Plane API examples

Set variables:

```bash
DP_URL="http://localhost:9000"
CACHE_ID="<cache-id>"
LC_AGENT_KEY="<agent-key>"
```

### Set a cache entry

```bash
curl -sS -X POST "$DP_URL/v1/caches/$CACHE_ID/entries" \
-H "Authorization: Bearer $LC_AGENT_KEY" \
-H "Content-Type: application/json" \
-d '{
"prompt": "What is the capital of France?",
"response": "The capital of France is Paris."
}'
```

### Search for a cached response

```bash
curl -sS -X POST "$DP_URL/v1/caches/$CACHE_ID/entries/search" \
-H "Authorization: Bearer $LC_AGENT_KEY" \
-H "Content-Type: application/json" \
-d '{
"prompt": "What'"'"'s the capital city of France?"
}'
```

### Delete a specific entry

```bash
curl -sS -X DELETE "$DP_URL/v1/caches/$CACHE_ID/entries/<entry-id>" \
-H "Authorization: Bearer $LC_AGENT_KEY"
```

### Delete entries matching attributes

```bash
curl -sS -X DELETE "$DP_URL/v1/caches/$CACHE_ID/entries" \
-H "Authorization: Bearer $LC_AGENT_KEY" \
-H "Content-Type: application/json" \
-d '{
"attributes": {
"topic": "geography"
}
}'
```

### Flush all entries in a cache

```bash
curl -sS -X POST "$DP_URL/v1/caches/$CACHE_ID/flush" \
-H "Authorization: Bearer $LC_AGENT_KEY"
```

### Check cache health

```bash
curl -sS "$DP_URL/v1/caches/$CACHE_ID/health" \
-H "Authorization: Bearer $LC_AGENT_KEY"
```

For the full request and response schema for cache entries (set, search,
delete, flush), see the
[LangCache API reference]({{< relref "/develop/ai/context-engine/langcache/api-reference" >}}).
That shared reference does not yet cover cache health; the example above
reflects the same Data Plane API.
Loading
Loading