Python: [Bug]: Foundry hosting feeds the model duplicated conversation history when a working session store is combined with service-side conversation storage

[sachinkahawala]

Description

What happened?

When a hosted /responses agent (agent_framework_foundry_hosting.ResponsesHostServer) is configured with a working agent session store and its chat client stores conversation state service-side (the default for FoundryChatClient / OpenAI Responses when store is not False), the model receives the conversation transcript more than once per request, and the duplication compounds every turn. In my repro, by turn 3 the model sees turn 1 three times (23 messages instead of 9).

Observable symptom with a real model: the agent visibly repeats its replies (it mirrors the duplicated transcript), getting worse as the conversation grows — plus the corresponding token overspend. I first hit this in production on a Foundry Hosted Agent (BYO code) using FoundryChatClient, and then reduced it to the fully local, Azure-free repro below.

What did you expect to happen?

The model receives each message of the conversation exactly once per request.

Root cause (from reading the code)

_handle_inner_agent in agent_framework_foundry_hosting/_responses.py gives agent.run(...) two sources of conversation history at once:

1. run_kwargs["messages"] = the full transcript fetched from the platform (await context.get_history()) + the new input, and
2. run_kwargs["session"] = the session loaded from the session store (session_storage.get(context.conversation_id or previous_response_id)).

The handler guards against transcript replay by popping the sentinel history buffer (session.state.pop(_HOSTED_RESPONSES_HISTORY_SOURCE_ID, None)) before the run and before persisting — but that only cleans session.state. The session's top-level service_session_id field survives, and agent-framework-core resumes it ("a live service-managed session id takes precedence over the resolved conversation id", and session.service_session_id = response.conversation_id after each run). So the run continues the service-side conversation thread, which already contains the whole transcript, while messages carries the whole transcript again. The service appends the duplicated request to the thread, so the next turn's thread contains it twice — hence the superlinear growth.

Why this is rarely seen

• The default FoundryAgentSessionStore's reads currently fail on every request ([azure-core] _aiohttp_body_helper does not decompress Content-Encoding: br (Brotli) Azure/azure-sdk-for-python#47186 — the Foundry storage service responds with Content-Encoding: br, which azure-core does not decompress) and return None, so the service thread is never resumed. Any working session store exposes this bug.
• Setting "store": False on the agent would avoid service-side storage, but that currently triggers the ~5 s streaming delay per model call tracked in Python: [Bug]: Foundry Hosted agent - delay before final response when streaming #7487, so real deployments keep store on.

Steps to reproduce

1. pip install --pre agent-framework-foundry-hosting (repro'd with the versions listed below)
2. python repro.py (script in the Code Sample field — starts a local ResponsesHostServer, drives three chained turns with a fake chat client that emulates service-side conversation storage, and prints exactly what the model receives per service call)
3. Observe the duplicated history and the non-zero exit; python repro.py --null-store (session store get returns None — my production workaround) shows the clean, expected input.

Precedent

.NET fixed this exact class of bug in #7525 ("A hosted agent quietly kept a second copy of its conversation … the same conversation could also reach the model twice, once as input from the platform's record and once replayed by the agent itself"). I could not find a Python counterpart issue or port.

Suggested direction

Mirror #7525's single-source-of-history approach on the Python side — e.g. when _uses_hosted_responses_history is active, don't resume/persist service_session_id on the hosted session (or don't replay the full platform history when a service-managed thread is being resumed). Happy to submit a PR with a failing test if a maintainer confirms the preferred direction.

Code Sample

"""Repro: hosted /responses feeds the model the conversation transcript twice
(superlinearly) when a working agent-session store is combined with a chat
client that stores conversation state service-side.

    pip install --pre agent-framework-foundry-hosting
    python repro.py            # buggy: duplicated history, grows every turn
    python repro.py --null-store  # workaround: session store returns None -> clean

No Azure resources needed. The script starts a local ResponsesHostServer on
:8188, drives three chained turns, and prints exactly what the model receives
on each service call. The fake chat client emulates service-side conversation
storage (what FoundryChatClient / OpenAI Responses do when ``store`` is not
False): requests carrying ``conversation_id`` are evaluated against the stored
thread plus the request messages, and responses return the thread id, which
the framework records on the session as ``service_session_id``.
"""

import json
import os
import subprocess
import sys
import time
import urllib.request

LOG = os.path.join(os.path.dirname(os.path.abspath(__file__)), "repro_model_calls.jsonl")
PORT = int(os.environ.get("REPRO_PORT", "8188"))
NULL_STORE = "--null-store" in sys.argv


# --------------------------------------------------------------------------
# server half (run with: python repro.py --serve [--null-store])
# --------------------------------------------------------------------------
def serve() -> None:
    from agent_framework import (
        Agent,
        BaseChatClient,
        ChatResponse,
        ChatResponseUpdate,
        FunctionInvocationLayer,
        ResponseStream,
        SessionStore,
    )
    from agent_framework_foundry_hosting import AgentSessionStoreProvider, ResponsesHostServer

    def log(event, payload):
        with open(LOG, "a") as f:
            f.write(json.dumps({"event": event, "payload": payload}) + "\n")

    def describe(messages):
        out = []
        for m in messages:
            role = getattr(getattr(m, "role", None), "value", getattr(m, "role", None))
            out.append({"role": str(role), "text": getattr(m, "text", None)})
        return out

    class ServiceStorageChatClient(FunctionInvocationLayer, BaseChatClient):
        """Fake model with service-side conversation storage (store=on)."""

        STORES_BY_DEFAULT = True
        _calls = 0
        _threads: dict = {}

        def _inner_get_response(self, *, messages, stream, options, **kwargs):
            ServiceStorageChatClient._calls += 1
            n = ServiceStorageChatClient._calls

            conv = options.get("conversation_id")
            stored = list(self._threads.get(conv, [])) if conv else []
            request = describe(messages)
            # what the real service's model would see: stored thread + request
            log("model_call", {
                "call": n,
                "conversation_id": conv,
                "messages": ([dict(m, origin="service-thread") for m in stored]
                             + [dict(m, origin="request") for m in request]),
            })

            last = messages[-1] if messages else None
            answered = any(getattr(c, "type", None) == "function_result"
                           for c in getattr(last, "contents", []))
            contents = ([{"type": "text", "text": f"ack-{n}"}] if answered else
                        [{"type": "function_call", "call_id": f"call-{n}",
                          "name": "get_time", "arguments": "{}"}])

            tid = conv or "svc-thread-1"
            self._threads[tid] = stored + request + [
                {"role": "assistant", "text": f"ack-{n}" if answered else "(tool call)"}
            ]

            if stream:
                async def _updates():
                    yield ChatResponseUpdate(role="assistant", contents=contents, conversation_id=tid)
                return ResponseStream(_updates(), finalizer=ChatResponse.from_updates)

            async def _response():
                from agent_framework import Message
                return ChatResponse(messages=[Message(role="assistant", contents=contents)],
                                    conversation_id=tid)
            return _response()

    class NullSessionStore(SessionStore):
        async def get(self, session_id):
            return None

        async def set(self, session_id, session):
            return None

    class Provider(AgentSessionStoreProvider):
        def __init__(self):
            # SessionStore() is the framework's own stock in-memory store.
            self._store = NullSessionStore() if NULL_STORE else SessionStore()

        def get_store(self, *, config, platform_context):
            return self._store

    def get_time() -> str:
        """Return the current time."""
        return "noon"

    open(LOG, "w").close()
    agent = Agent(client=ServiceStorageChatClient(), instructions="You are a helpful assistant.",
                  tools=[get_time])
    ResponsesHostServer(agent=agent, agent_session_store_provider=Provider()).run()


# --------------------------------------------------------------------------
# driver half (default entry point)
# --------------------------------------------------------------------------
def post(text, previous_response_id=None):
    body = {"input": text, "stream": False, "store": True, "model": "repro-model"}
    if previous_response_id:
        body["previous_response_id"] = previous_response_id
    req = urllib.request.Request(f"http://127.0.0.1:{PORT}/responses",
                                 data=json.dumps(body).encode(),
                                 headers={"content-type": "application/json"}, method="POST")
    with urllib.request.urlopen(req, timeout=60) as resp:
        return json.loads(resp.read())


def drive() -> None:
    env = dict(os.environ, PORT=str(PORT),
               AGENTSERVER_STATE_ROOT=os.path.join(os.path.dirname(LOG), "repro_state"))
    args = [sys.executable, os.path.abspath(__file__), "--serve"]
    if NULL_STORE:
        args.append("--null-store")
    server = subprocess.Popen(args, env=env, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
    try:
        for _ in range(60):
            try:
                urllib.request.urlopen(f"http://127.0.0.1:{PORT}/readiness", timeout=1)
                break
            except Exception:
                time.sleep(0.5)
        r1 = post("Turn one: my favourite colour is teal.")
        r2 = post("Turn two: I live in Melbourne.", r1["id"])
        post("Turn three: what do you know about me?", r2["id"])
    finally:
        server.terminate()

    calls = [json.loads(l)["payload"] for l in open(LOG) if json.loads(l)["event"] == "model_call"]
    for c in calls:
        print(f"\n--- model call {c['call']} "
              f"(conversation_id={c['conversation_id']}, {len(c['messages'])} messages) ---")
        for m in c["messages"]:
            print(f"  [{m.get('origin', '?'):<14}] {m['role']:<10} {(m['text'] or '')[:70]}")

    last = calls[-1]["messages"]
    counts: dict = {}
    for m in last:
        if m["text"]:
            counts[(m["role"], m["text"])] = counts.get((m["role"], m["text"]), 0) + 1
    dupes = {k: v for k, v in counts.items() if v > 1}
    if dupes:
        print("\nDUPLICATED MESSAGES IN FINAL MODEL INPUT:")
        for (role, text), v in dupes.items():
            print(f"  x{v}  {role}: {text[:60]}")
        sys.exit(1)
    print("\nOK: no duplicates in final model input.")


if __name__ == "__main__":
    if "--serve" in sys.argv:
        PORT = int(os.environ.get("PORT", PORT))
        serve()
    else:
        drive()

Error Messages / Stack Traces

No exception is raised — the failure is silent transcript duplication (plus token overspend). Output of `python repro.py`, third turn (the fake service's model input = stored service thread + request messages; origin labels added by the repro):

    --- model call 5 (conversation_id=svc-thread-1, 21 messages) ---
      [service-thread] user       Turn one: my favourite colour is teal.
      [service-thread] assistant  (tool call)
      [service-thread] tool
      [service-thread] assistant  ack-2
      [service-thread] user       Turn one: my favourite colour is teal.      <-- turn 1 again
      [service-thread] assistant
      [service-thread] tool
      [service-thread] assistant  ack-2
      [service-thread] user       Turn two: I live in Melbourne.
      [service-thread] assistant  (tool call)
      [service-thread] tool
      [service-thread] assistant  ack-4
      [request       ] user       Turn one: my favourite colour is teal.      <-- turn 1 a third time
      [request       ] assistant
      [request       ] tool
      [request       ] assistant  ack-2
      [request       ] user       Turn two: I live in Melbourne.              <-- turn 2 again
      [request       ] assistant
      [request       ] tool
      [request       ] assistant  ack-4
      [request       ] user       Turn three: what do you know about me?

    DUPLICATED MESSAGES IN FINAL MODEL INPUT:
      x3  user: Turn one: my favourite colour is teal.
      x3  assistant: (tool call)
      x3  assistant: ack-2
      x2  user: Turn two: I live in Melbourne.
      x2  assistant: ack-4

With `--null-store` (workaround), the same three turns produce each message exactly once and the script prints `OK: no duplicates in final model input.`

Package Versions

agent-framework-core: 1.16.0, agent-framework-foundry-hosting: 1.0.0b260827, azure-ai-agentserver-core: 2.1.0, azure-ai-agentserver-responses: 2.2.0b1 (also reproduced on agent-framework-core 1.14.0 + agent-framework-foundry-hosting 1.0.0b260813)

Python Version

Python 3.13

Additional Context

• First observed on a deployed Azure AI Foundry Hosted Agent (BYO code, FoundryChatClient, gpt-5.4-mini, platform-managed conversations): replies were visibly emitted twice, worse as the chat grew. Replacing the session store with one whose get always returns None eliminated it; sandbox-filesystem state was unaffected.
• The repro emulates service-side conversation storage in the fake chat client so it runs with no Azure resources; the behaviour matches what I saw with the real FoundryChatClient in production.
• Related: .NET fix .NET: Give a hosted agent a single source of conversation history #7525 (same bug class, no Python port); Python: [Bug]: Foundry Hosted agent - delay before final response when streaming #7487 (why store: False is not a practical avoidance); [azure-core] _aiohttp_body_helper does not decompress Content-Encoding: br (Brotli) Azure/azure-sdk-for-python#47186 (why the default session store currently masks this bug). I'm aware Python: omit failed Foundry turns from conversation chat history #7637 touches _responses.py conversation-history behaviour — it does not change the double-feed path.

[sachinkahawala]

Since the design discussion is happening (thanks Eduard van Valkenburg (@eavanvalkenburg) — closing note on #7957): one measured data point that bears directly on the direction choice discussed there, i.e. forcing store=False on hosted runs versus dropping the stale service_session_id on session load.

Direct HTTPS to my project's /openai/v1/responses (raw httpx SSE — no agent-framework code in the loop), gpt-5.4-mini, reasoning: none, trivial prompt, 5 runs + a discarded warmup per cell:

	time to first event	time to final
stream=true, store=true	min 298ms / med 335ms	min 1095ms / med 1183ms
stream=true, store=false	min 300ms / med 353ms	min 6068ms / med 6100ms
stream=false, either store	—	~1.2s

So the #7487 delay reproduces today, is completely consistent (+4.9s on every streamed call with store=false, ~30ms spread across runs), is service-side (no SDK in this measurement; the first stream event arrives fast either way — the hang is before the final event), and only affects streaming.

Since hosted agents run streamed model calls, a store=False-by-default fix would add ~5s per model call (~10s per tool-using turn) on the project endpoint as it behaves today, while neutralizing the stale service_session_id on load fixes the duplication with store left on, at the cost of an orphan service thread per turn.

Happy to re-run this measurement or the repro benches from this thread against whatever shape the design lands on.