agentling/feature/agent by folathecoder · Pull Request #4 · folathecoder/agentling

folathecoder · 2026-07-06T21:22:00Z

Add agent.py: the async ReAct loop (Agent)
Add end-to-end agent tests with a scripted FakeModel (7 tests)

Assemble Model + Tools + Memory into an async agent. A single async generator (_run_stream) drives the model/tool cycle and yields Events; run() dispatches — stream=True returns that stream, stream=False drains it to the final answer. - forgiving termination (no tool calls) and explicit final_answer - parallel tool execution via asyncio.gather (parallel_tools flag) - tool failures captured as error observations (self-heal), never crashes - ActionStep records model_message/tool_results/usage/timing; step_callbacks fire - max_steps exhausted -> one forced tool-free answer

qodo-code-review · 2026-07-06T21:23:00Z

PR Summary by Qodo

Add async ReAct Agent loop with event streaming and end-to-end tests

✨ Enhancement 🧪 Tests 🕐 20-40 Minutes

AI Description

• Introduce an async Agent ReAct loop that runs model/tool cycles and records memory.
• Support streamed Events (tool calls/results, steps, final answer) or a drained final answer.
• Add end-to-end tests using a scripted FakeModel to cover termination, errors, and max-steps.

Diagram

graph TD
  U([Client]) --> A["Agent.run/_run_stream"] --> M[("Memory")]
  M --> L[["Model.generate"]] --> A
  A --> T{{"Tools (+final_answer)"}} --> A
  A --> E[/"Event stream"/] --> U
  A --> F(["Final answer"]) --> U
  subgraph Legend
    direction LR
    _c([Client]) ~~~ _a[Agent] ~~~ _m[(Memory)] ~~~ _l[[Model]] ~~~ _t{{Tools}} ~~~ _e[/Events/]
  end

High-Level Assessment

The following are alternative approaches to this PR:

1. Require explicit final_answer termination only

➕ Eliminates ambiguity between an assistant message vs a terminal answer
➕ Makes provider behavior more consistent across models
➖ Less ergonomic: simple tasks must always call final_answer
➖ Breaks compatibility with models that don’t reliably produce tool calls

2. Support true model streaming (Model.stream) inside the loop

➕ Lower latency UX (text deltas as they arrive)
➕ Can surface partial tool-call arguments for richer tracing
➖ More complex state machine (merge deltas, tool-call assembly, usage aggregation)
➖ Harder to test deterministically than generate()-only flows

Recommendation: The current hybrid termination (no tool calls implies final, plus explicit final_answer) is a pragmatic default for early framework ergonomics, and the tests exercise both paths. Keep this approach, but consider a future strict-mode flag and a follow-on PR to integrate Model.stream for incremental TextDelta events.

Files changed (2) +376 / -0

Enhancement (1) +185 / -0

agent.pyAdd async Agent ReAct loop with event streaming and robust termination +185/-0

Add async Agent ReAct loop with event streaming and robust termination

• Introduces the Agent orchestrator that combines Model, Tools (including an always-present final_answer), and Memory into a single async generator loop. Supports streaming Event emission, optional parallel tool execution, forgiving termination when no tool calls occur, and a max-steps fallback that forces a tool-free final answer. Tool failures are converted into error ToolResults instead of crashing the run, and ActionStep metadata (usage/timing) is recorded and dispatched to step callbacks.

src/agentling/agent.py

Tests (1) +191 / -0

test_agent.pyAdd end-to-end Agent loop tests with a scripted FakeModel +191/-0

Add end-to-end Agent loop tests with a scripted FakeModel

• Adds async pytest coverage for termination behavior (implicit and explicit final_answer), multi-step tool usage, tool error-to-observation behavior, event streaming, and max_steps forced completion. Uses a deterministic FakeModel that returns predefined ChatMessages to assert on memory steps and emitted Events.

tests/test_agent.py

qodo-code-review · 2026-07-06T21:24:54Z

Code Review by Qodo

🐞 Bugs (4) 📘 Rule violations (0) 📜 Skill insights (0)

1. Final tool runs other tools 🐞 Bug ≡ Correctness

Description

If the model emits final_answer alongside other tool calls, the agent will still execute all tools
(potentially side-effectful) before terminating, because termination is checked only after tool
execution completes.

Code

src/agentling/agent.py[R113-142]

+            if self.parallel_tools:
+                results: list[ToolResult] = await asyncio.gather(
+                    *(self._execute_tool(tc) for tc in response.tool_calls)
+                )
+            else:
+                results = [await self._execute_tool(tc) for tc in response.tool_calls]
+
+            for result in results:
+                yield ToolResultEvent(result=result)
+
+            action = ActionStep(
+                model_message=response,
+                tool_results=results,
+                usage=response.usage,
+                duration=time.monotonic() - started,
+            )
+            self.memory.add(action)
+            for callback in self.step_callbacks:
+                callback(action)
+            yield StepEvent(step=action)
+
+            # Explicit termination: the model called final_answer (successfully).
+            final = next(
+                (r for r in results if r.name == "final_answer" and not r.is_error),
+                None,
+            )
+            if final is not None:
+                self.memory.add(FinalStep(answer=final.content))
+                yield FinalEvent(answer=final.content, usage=response.usage)
+                return

Evidence

The agent executes every tool call via gather/list-comprehension, then searches the results for
final_answer to terminate; therefore final_answer does not prevent other tools in the same
response from running. The built-in tool is explicitly intended to end the run.

src/agentling/agent.py[109-142]
src/agentling/tools.py[358-384]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
When the model requests multiple tool calls in a single turn and one of them is `final_answer`, the agent currently executes *all* tool calls before checking for termination. This can trigger unintended side effects (e.g., sending emails, deleting resources) even though the model has already decided to end the run.

## Issue Context
`final_answer` is explicitly documented as the terminating tool, so it should prevent execution of other tools in the same turn (or at minimum it should be executed last and cancel/skip the rest).

## Fix Focus Areas
- src/agentling/agent.py[109-142]
- src/agentling/tools.py[358-384]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

2. Model exceptions crash agent 🐞 Bug ☼ Reliability

Description

Exceptions from model.generate() propagate and abort the run (including tool-argument parsing
failures in the OpenAI adapter), unlike tool failures which are converted into error observations.

Code

src/agentling/agent.py[R100-102]

+            messages = self.memory.to_messages(self.instructions)
+            response = await self.model.generate(messages, tools=self._tool_schemas)
+

Evidence

The agent makes unguarded calls to model.generate(). The OpenAI adapter can raise ValueError
when tool-call arguments are invalid JSON or not a JSON object, which would currently crash the
agent loop instead of producing a structured step/event.

src/agentling/agent.py[97-107]
src/agentling/agent.py[144-154]
src/agentling/models.py[338-356]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`Agent._run_stream()` awaits `model.generate()` without any exception handling. If the model adapter raises (network failure, provider error, or tool-argument parsing `ValueError`), the agent crashes and yields no structured failure event/step.

This is inconsistent with the tool execution layer, which deliberately converts tool exceptions into recoverable observations.

## Issue Context
The OpenAI adapter explicitly raises `ValueError` on invalid tool-call JSON (“fail loudly so the caller can decide how to handle the error”), so the Agent should decide (e.g., record an ActionStep with error and either retry, request a tool-free response, or terminate with a structured FinalEvent).

## Fix Focus Areas
- src/agentling/agent.py[97-107]
- src/agentling/agent.py[144-154]
- src/agentling/models.py[338-356]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

3. Forced prompt not in memory 🐞 Bug ≡ Correctness

Description

On max-steps fallback, the extra user message (“Step limit reached…”) is appended directly to the
messages list and not recorded as a Step, so Memory/serialization/replay no longer reflects the
exact conversation sent to the model.

Code

src/agentling/agent.py[R144-154]

+        # Step limit reached: force one tool-free answer.
+        messages = self.memory.to_messages(self.instructions)
+        messages.append(
+            ChatMessage(
+                role="user",
+                content="Step limit reached. Give your best final answer now.",
+            )
+        )
+        response = await self.model.generate(messages)
+        self.memory.add(FinalStep(answer=response.content))
+        yield FinalEvent(answer=response.content, usage=response.usage)

Evidence

The agent appends a ChatMessage outside of Memory and then calls the model with that augmented
list. Memory is explicitly the durable record and the sole source for rendering and JSON
round-tripping, so the appended message will not be persisted or replayed.

src/agentling/agent.py[144-154]
src/agentling/memory.py[1-9]
src/agentling/memory.py[125-167]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
When the step limit is reached, the agent appends a synthetic user prompt to the outgoing message list without storing it in `Memory`. This creates divergence between:
- what the model actually saw, and
- what `Memory.to_messages()` / `Memory.dump_json()` can reproduce.

This breaks the stated “persistence/replay” goal of Memory and makes debugging runs harder.

## Issue Context
Memory is designed to be the durable, structured record of a run and is the source of truth for message rendering and JSON round-tripping.

## Fix Focus Areas
- src/agentling/agent.py[144-154]
- src/agentling/memory.py[97-167]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

4. StepEvent contract mismatch 🐞 Bug ◔ Observability

Description

StepEvent is described as “emitted after a step has been recorded to memory,” but the agent only
yields StepEvent for ActionStep and not when recording TaskStep or FinalStep, making the
stream incomplete relative to memory updates.

Code

src/agentling/agent.py[R95-107]

+        self.memory.add(TaskStep(task=task))
+
+        limit = max_steps or self.max_steps
+        for _ in range(limit):
+            started = time.monotonic()
+            messages = self.memory.to_messages(self.instructions)
+            response = await self.model.generate(messages, tools=self._tool_schemas)
+
+            # Forgiving termination: a turn with no tool calls IS the final answer.
+            if not response.tool_calls:
+                self.memory.add(FinalStep(answer=response.content))
+                yield FinalEvent(answer=response.content, usage=response.usage)
+                return

Evidence
The events module frames StepEvent as the hook that connects the live stream to memory for recorded
steps. The agent records TaskStep and FinalStep but only yields StepEvent for the ActionStep, so
consumers relying on StepEvent won’t observe those memory writes in real time.
src/agentling/events.py[7-45]
src/agentling/agent.py[95-107]
src/agentling/agent.py[123-142]
src/agentling/agent.py[144-154]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The agent’s event stream emits `StepEvent` only for `ActionStep`. However, `TaskStep` and `FinalStep` are also added to memory without emitting a corresponding `StepEvent`, which can confuse streaming consumers that use `StepEvent` to keep an in-sync view of Memory.

If the intended contract is “StepEvent == any step recorded,” emit StepEvent for TaskStep/FinalStep too. If the intent is “StepEvent == action steps only,” update the docs/type names to match.

## Issue Context
The events module documents StepEvent as being emitted after a step is recorded, without restricting it to action steps.

## Fix Focus Areas
- src/agentling/agent.py[95-107]
- src/agentling/agent.py[103-107]
- src/agentling/agent.py[139-154]
- src/agentling/events.py[7-45]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

qodo-code-review · 2026-07-06T21:24:54Z

+            if self.parallel_tools:
+                results: list[ToolResult] = await asyncio.gather(
+                    *(self._execute_tool(tc) for tc in response.tool_calls)
+                )
+            else:
+                results = [await self._execute_tool(tc) for tc in response.tool_calls]
+
+            for result in results:
+                yield ToolResultEvent(result=result)
+
+            action = ActionStep(
+                model_message=response,
+                tool_results=results,
+                usage=response.usage,
+                duration=time.monotonic() - started,
+            )
+            self.memory.add(action)
+            for callback in self.step_callbacks:
+                callback(action)
+            yield StepEvent(step=action)
+
+            # Explicit termination: the model called final_answer (successfully).
+            final = next(
+                (r for r in results if r.name == "final_answer" and not r.is_error),
+                None,
+            )
+            if final is not None:
+                self.memory.add(FinalStep(answer=final.content))
+                yield FinalEvent(answer=final.content, usage=response.usage)
+                return


1. Final tool runs other tools 🐞 Bug ≡ Correctness

If the model emits final_answer alongside other tool calls, the agent will still execute all tools (potentially side-effectful) before terminating, because termination is checked only after tool execution completes.

Agent Prompt

## Issue description When the model requests multiple tool calls in a single turn and one of them is `final_answer`, the agent currently executes *all* tool calls before checking for termination. This can trigger unintended side effects (e.g., sending emails, deleting resources) even though the model has already decided to end the run. ## Issue Context `final_answer` is explicitly documented as the terminating tool, so it should prevent execution of other tools in the same turn (or at minimum it should be executed last and cancel/skip the rest). ## Fix Focus Areas - src/agentling/agent.py[109-142] - src/agentling/tools.py[358-384]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

qodo-code-review · 2026-07-06T21:24:54Z

+        # Step limit reached: force one tool-free answer.
+        messages = self.memory.to_messages(self.instructions)
+        messages.append(
+            ChatMessage(
+                role="user",
+                content="Step limit reached. Give your best final answer now.",
+            )
+        )
+        response = await self.model.generate(messages)
+        self.memory.add(FinalStep(answer=response.content))
+        yield FinalEvent(answer=response.content, usage=response.usage)


2. Forced prompt not in memory 🐞 Bug ≡ Correctness

On max-steps fallback, the extra user message (“Step limit reached…”) is appended directly to the messages list and not recorded as a Step, so Memory/serialization/replay no longer reflects the exact conversation sent to the model.

Agent Prompt

## Issue description When the step limit is reached, the agent appends a synthetic user prompt to the outgoing message list without storing it in `Memory`. This creates divergence between: - what the model actually saw, and - what `Memory.to_messages()` / `Memory.dump_json()` can reproduce. This breaks the stated “persistence/replay” goal of Memory and makes debugging runs harder. ## Issue Context Memory is designed to be the durable, structured record of a run and is the source of truth for message rendering and JSON round-tripping. ## Fix Focus Areas - src/agentling/agent.py[144-154] - src/agentling/memory.py[97-167]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

qodo-code-review · 2026-07-06T21:24:54Z

+        self.memory.add(TaskStep(task=task))
+
+        limit = max_steps or self.max_steps
+        for _ in range(limit):
+            started = time.monotonic()
+            messages = self.memory.to_messages(self.instructions)
+            response = await self.model.generate(messages, tools=self._tool_schemas)
+
+            # Forgiving termination: a turn with no tool calls IS the final answer.
+            if not response.tool_calls:
+                self.memory.add(FinalStep(answer=response.content))
+                yield FinalEvent(answer=response.content, usage=response.usage)
+                return


3. Stepevent contract mismatch 🐞 Bug ◔ Observability

StepEvent is described as “emitted after a step has been recorded to memory,” but the agent only yields StepEvent for ActionStep and not when recording TaskStep or FinalStep, making the stream incomplete relative to memory updates.

Agent Prompt

## Issue description The agent’s event stream emits `StepEvent` only for `ActionStep`. However, `TaskStep` and `FinalStep` are also added to memory without emitting a corresponding `StepEvent`, which can confuse streaming consumers that use `StepEvent` to keep an in-sync view of Memory. If the intended contract is “StepEvent == any step recorded,” emit StepEvent for TaskStep/FinalStep too. If the intent is “StepEvent == action steps only,” update the docs/type names to match. ## Issue Context The events module documents StepEvent as being emitted after a step is recorded, without restricting it to action steps. ## Fix Focus Areas - src/agentling/agent.py[95-107] - src/agentling/agent.py[103-107] - src/agentling/agent.py[139-154] - src/agentling/events.py[7-45]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

qodo-code-review · 2026-07-06T21:24:55Z

+            messages = self.memory.to_messages(self.instructions)
+            response = await self.model.generate(messages, tools=self._tool_schemas)
+


4. Model exceptions crash agent 🐞 Bug ☼ Reliability

Exceptions from model.generate() propagate and abort the run (including tool-argument parsing failures in the OpenAI adapter), unlike tool failures which are converted into error observations.

Agent Prompt

## Issue description `Agent._run_stream()` awaits `model.generate()` without any exception handling. If the model adapter raises (network failure, provider error, or tool-argument parsing `ValueError`), the agent crashes and yields no structured failure event/step. This is inconsistent with the tool execution layer, which deliberately converts tool exceptions into recoverable observations. ## Issue Context The OpenAI adapter explicitly raises `ValueError` on invalid tool-call JSON (“fail loudly so the caller can decide how to handle the error”), so the Agent should decide (e.g., record an ActionStep with error and either retry, request a tool-free response, or terminate with a structured FinalEvent). ## Fix Focus Areas - src/agentling/agent.py[97-107] - src/agentling/agent.py[144-154] - src/agentling/models.py[338-356]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

folathecoder added 2 commits July 6, 2026 22:21

Add end-to-end agent tests with a scripted FakeModel (7 tests)

f24ecf2

folathecoder merged commit 2301fe2 into main Jul 6, 2026
2 checks passed

folathecoder deleted the agentling/feature/agent branch July 6, 2026 21:23

qodo-code-review Bot reviewed Jul 6, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

agentling/feature/agent#4

agentling/feature/agent#4
folathecoder merged 2 commits into
mainfrom
agentling/feature/agent

folathecoder commented Jul 6, 2026

Uh oh!

qodo-code-review Bot commented Jul 6, 2026

Uh oh!

Uh oh!

qodo-code-review Bot commented Jul 6, 2026

Uh oh!

qodo-code-review Bot Jul 6, 2026

Uh oh!

qodo-code-review Bot Jul 6, 2026

Uh oh!

qodo-code-review Bot Jul 6, 2026

Uh oh!

qodo-code-review Bot Jul 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

		messages = self.memory.to_messages(self.instructions)
		response = await self.model.generate(messages, tools=self._tool_schemas)

Conversation

folathecoder commented Jul 6, 2026

Uh oh!

qodo-code-review Bot commented Jul 6, 2026

PR Summary by Qodo

Uh oh!

Uh oh!

qodo-code-review Bot commented Jul 6, 2026

Code Review by Qodo

Uh oh!

qodo-code-review Bot Jul 6, 2026

Choose a reason for hiding this comment

Uh oh!

qodo-code-review Bot Jul 6, 2026

Choose a reason for hiding this comment

Uh oh!

qodo-code-review Bot Jul 6, 2026

Choose a reason for hiding this comment

Uh oh!

qodo-code-review Bot Jul 6, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant