Exception Input is too long for requested model due to contex getting bigger

[oavitzur]

Hi
I am using ADK with a very simple scenario with one LlmAgent.

I keep include_contents set to default since I want to keep sending to llm the memory of old messages in the session. I'm using anthropic.claude-haiku-4-5, but this issue happens in all models.

After several messages sent in the same session I get exception : **Context Window Error - {"message":"The model returned the following errors: Input is too long for requested model."}**|

The issue that the messages we send in the LlmResponse keeps on incrementing until the response we send back to the model is too long. I searched for a possible solution to limit this and could not find one. Eventually I added a before_model_callback in which I remove old messages from llm_request.contents

But this seems to me to be such a very basic issue in the current implementation that either there is a built in solution, or I'm doing some thing wrong.

Any help here is appreciated

[adk-bot]

Response from ADK Answering Agent (experimental, answer may be inaccurate)

TLDR: Your approach of using a before_model_callback to remove old messages from llm_request.contents is the correct and intended way to manage the context window size in ADK.

---

Hi there! You've run into a common challenge with conversational AI, and your solution is exactly the pattern that ADK is designed to support.

ADK provides developers with fine-grained control over the LLM lifecycle through callbacks. The issue you're facing, where the context grows too large, is a perfect use case for the before_model_callback. As you've discovered, this callback allows you to inspect and modify the llm_request right before it's sent to the model.

Your implementation of trimming llm_request.contents is the recommended way to prevent context window errors. This approach gives you the flexibility to implement any kind of trimming logic you need—from simple truncation to more complex strategies like summarization or keeping only the first and last few messages.

So, to confirm, you are not doing anything wrong; you've correctly used ADK's callback mechanism to handle context management.

For more details on how to leverage callbacks, you can refer to the design patterns and best practices documentation. [1]

[1] https://google.github.io/adk-docs/callbacks/design-patterns-and-best-practices/

[oavitzur]

getting the context too big for the model is such a basic scenario, how come its not handled by ADK framework ?
maybe a mechanism to use the model max tokens and handle this out of the box by the framework should be added ?
How can I calculate how many tokens a content in LlmRequest.contents has taken ?