FastAPI + Google ADK: Should I reuse a Runner per agent across requests (concurrency/session safety)?

[GanziMan]

Hello,

I’m building a backend service with Google ADK using a custom FastAPI app (not the default CLI app).

Right now, for each incoming request I create a new App and Runner inside the route handler. I’m concerned this per-request initialization may be inefficient and/or increase memory/CPU overhead under high concurrency.

Here’s a simplified version of what I’m doing:

router/writing.py

@router.post("/evaluator/essay")
async def evaluate_essay(request: EssayRequest):
  
  prompt = build_prompt(request)

  sub_agent = some_agent

  result = await run_sub_agent(sub_agent, prompt)

  return json.loads(result["result"])

services/agent_service.py

async def run_sub_agent(sub_agent, prompt: str) -> dict:
session_service = InMemorySessionService()

# created for every request

agent_app = App(name=f"app_{sub_agent.name}", root_agent=sub_agent)

runner = Runner(app=agent_app, session_service=session_service)

# ... runner.run_async(...) ...`

My questions:
1. Is it recommended to reuse a single Runner instance across multiple requests for the same agent (e.g., a singleton or a per-agent cache), or is creating a Runner per request the intended pattern?
2. If reuse is recommended, what is the correct approach regarding session isolation and concurrency safety?
• For example, should Runner be long-lived but create a new session per request?
• Are App/Runner designed to be thread-safe / async-safe for concurrent calls?

I’m considering refactoring to a service that caches Runner instances by agent name (similar to how ADKWebServer appears to manage runners). Any best-practice guidance would be appreciated.

Thanks!

[adk-bot]

Response from ADK Answering Agent (experimental, answer may be inaccurate)

TLDR: Yes, you should reuse a single Runner instance. It is inefficient to create a new App and Runner for each request. Both App and Runner are designed to be stateless, reusable, and safe for concurrent use.

---

Hello!

That's a great question. Your instinct is correct—creating a new App and Runner on every request is inefficient and not the intended pattern.

Here’s a breakdown of the best practice for using ADK in a web service like FastAPI:

1. Reuse App, Runner, and SessionService

The adk.Runner is a stateless orchestration engine. [1] It does not hold conversation history in memory. Instead, it relies on the SessionService to load the state for a given session_id for each invocation. Because of this stateless design, a single Runner instance can and should be reused across all requests.

The same applies to your adk.App and SessionService instances. You should create them once when your FastAPI application starts and reuse them for the lifetime of the application.

2. Session Isolation and Concurrency

• Session Isolation: You achieve session isolation by providing a unique session_id to runner.run_async(). The runner uses this ID to fetch the correct conversation history from the SessionService, ensuring that different user sessions do not interfere with each other. [1]

• Concurrency Safety: Since the Runner is stateless, it is safe to be called concurrently from multiple asynchronous request handlers. Each call to run_async is an independent operation tied to a specific session.

Recommended Refactor

Here’s how you could refactor your code to follow this singleton pattern:

# services/agent_service.py
from adk.services import InMemorySessionService
from adk.api import App, Runner

# --- Create single, long-lived instances ---
# This code should run once at application startup.
# In FastAPI, you can use a startup event handler.

agent_app = App(name="my_app", root_agent=some_agent)
session_service = InMemorySessionService()
runner = Runner(app=agent_app, session_service=session_service)

async def run_sub_agent(prompt: str, session_id: str) -> dict:
    # Reuse the singleton runner and pass the session_id
    # The session_id should be unique per conversation/user.
    # You'll need to manage this (e.g., create a new one for a new user,
    # or get an existing one from the client's request).
    result = await runner.run_async(prompt=prompt, session_id=session_id)
    return result

# router/writing.py
@router.post("/evaluator/essay")
async def evaluate_essay(request: EssayRequest):
    # You need a strategy to manage session IDs. For example:
    # session_id = request.headers.get("X-Session-ID") or str(uuid.uuid4())
    session_id = "some_unique_session_id_per_user" 

    prompt = build_prompt(request)
    result = await run_sub_agent(prompt, session_id)
    return json.loads(result["result"])

By creating these components once, you will significantly reduce the overhead per request, lower memory usage, and improve the overall performance and scalability of your service.

For a complete example, you can refer to the ADK documentation which shows the Runner being initialized once at application startup. [2]

Hope this helps!

---

[1] ADK Agents
[2] ADK Streaming Development Guide