Skip to content

fix(tools): Return schema validation feedback to models#6368

Open
faiqali1 wants to merge 5 commits into
google:mainfrom
faiqali1:feature/add-feedback-on-failed-structured-output
Open

fix(tools): Return schema validation feedback to models#6368
faiqali1 wants to merge 5 commits into
google:mainfrom
faiqali1:feature/add-feedback-on-failed-structured-output

Conversation

@faiqali1

Copy link
Copy Markdown

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

1. Link to an existing issue (if applicable):

2. Or, if no issue exists, describe the change:

Problem:
When output_schema is used with tools, ADK adds the internal set_model_response tool so the model can submit its final structured output. If the model calls this tool with invalid structured data, Pydantic validation currently fails without returning the validation details to the model. This prevents the model from learning what was wrong and retrying with corrected arguments.

Solution:
Update SetModelResponseTool to catch Pydantic validation errors and return the validation message as a tool error response. The flow now only promotes set_model_response into the final model response when validation succeeds, so invalid structured output is sent back to the model for correction instead of being treated as final output.

Testing Plan

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass locally.

Passed targeted unit tests:

uv run --with pytest --with pytest-asyncio --with pytest-mock python -m pytest tests/unittests/tools/test_set_model_response_tool.py tests/unittests/flows/llm_flows/test_output_schema_processor.py

Result:

44 passed, 6 warnings

Also ran pre-commit on changed files:

uv run --with pre-commit pre-commit run --files src/google/adk/tools/set_model_response_tool.py src/google/adk/flows/llm_flows/_output_schema_processor.py tests/unittests/tools/test_set_model_response_tool.py tests/unittests/flows/llm_flows/test_output_schema_processor.py

Result:

Passed

Manual End-to-End (E2E) Tests:

Not run yet. Suggested manual test:

  1. Create or use an agent with both output_schema and tools.
  2. Trigger a set_model_response call with invalid schema data, such as a string for an integer field or a missing required field.
  3. Confirm the function response contains the Pydantic validation details.
  4. Confirm ADK does not emit a final structured model response for the invalid tool call.
  5. Retry with valid structured data and confirm ADK emits the final structured response.

Checklist

  • I have read the [CONTRIBUTING.md](https://github.com/google/adk-python/blob/main/CONTRIBUTING.md) document.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.
  • I have manually tested my changes end-to-end.
  • Any dependent changes have been merged and published in downstream modules.

Additional context

This behavior is similar to structured-output retry patterns used by libraries such as Instructor: validation failures are returned to the model with enough detail for the model to correct its next structured-output attempt.

This contribution was developed with AI assistance (Codex GPT-5.5).

@google-cla

google-cla Bot commented Jul 10, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@rohityan rohityan self-assigned this Jul 10, 2026
@rohityan rohityan added the tools [Component] This issue is related to tools label Jul 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

tools [Component] This issue is related to tools

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants