Does context caching cache across conversation turn?

[danhnguyen8391]

I have an agent that call an agent tool to complete task. The agent tool has very long static instructions, if I chat with my agent multiple times, each turn only uses the agent tool, does the context caching benefit across these turn chats?

[adk-bot]

Response from ADK Answering Agent (experimental, answer may be inaccurate)

TLDR: Yes, ADK's context caching is specifically designed to work across multiple conversation turns within the same session, which can significantly reduce latency and cost.

---

Hello! That's a great question. The context caching feature in ADK is indeed designed to persist and provide benefits across multiple turns in a conversation. Here’s a breakdown of how it works and how you can leverage it:

1. Multi-Turn Design: Context caching is enabled at the application level via the ContextCacheConfig class [1]. It is explicitly designed for multi-turn conversations to reduce input token costs and improve latency. The included cache_analysis sample shows typical input cost savings of 30-60% for multi-turn chats [2].

2. Caching Mechanism: Within a session, the ContextCacheRequestProcessor looks for cache metadata from previous events. If a valid, non-expired cache is found from a previous turn, it gets reused for the current turn. This process is repeated for subsequent turns in the same session [3].

3. Controlling Cache Reuse: The ContextCacheConfig includes a cache_intervals parameter, which defines the "maximum number of invocations to reuse the same cache before refreshing it" [1]. This gives you direct control over how many conversational turns the cache will be active for.

4. Optimizing for Static Instructions: For your specific use case with an agent tool that has long, static instructions, you can use the static_instruction field on your LlmAgent [4]. This field is designed for content that does not change between requests. Placing your tool's static instructions here is the ideal way to ensure they are cached effectively, as this static portion of the prompt will remain unchanged across turns.

In summary, by enabling ContextCacheConfig for your app and using static_instruction for your tool's unchanging instructions, you should see the context caching benefits across your multi-turn chats.

---

[1] context_cache_config.py - https://github.com/google/adk-python/blob/main/src/google/adk/agents/context_cache_config.py
[2] cache_analysis Sample README - https://github.com/google/adk-python/blob/main/contributing/samples/cache_analysis/README.md
[3] context_cache_processor.py - https://github.com/google/adk-python/blob/main/src/google/adk/flows/llm_flows/context_cache_processor.py
[4] llm_agent.py - https://github.com/google/adk-python/blob/main/src/google/adk/agents/llm_agent.py

[danhnguyen8391]

At each turn chat, the new user content comes into that make the cache invalid and create a new one, how does it benefit across turn?

[danhnguyen-ct]

I tested many times that the context caching is only apply for the root_agent, not for agent tool's agent or sub agent