Skip to content

Prefer case-local BFCL tool schemas - #52

Closed
SonAIengine wants to merge 1 commit into
codex/bfcl-argument-failure-tagsfrom
codex/bfcl-schema-only-argument-guidance
Closed

Prefer case-local BFCL tool schemas#52
SonAIengine wants to merge 1 commit into
codex/bfcl-argument-failure-tagsfrom
codex/bfcl-schema-only-argument-guidance

Conversation

@SonAIengine

Copy link
Copy Markdown
Owner

Summary

  • Prefer the current BFCL case row tool schema over the category-wide first definition when presenting retrieved tools to a model.
  • Add a default schema-only argument-key instruction so models avoid undeclared argument names.
  • Document the duplicate calculate_triangle_area schema collision and the Qwen 27B hardcase lift.

Validation

  • poetry run ruff check .
  • poetry run ruff format --check .
  • poetry run pytest tests/test_bfcl_tool_selection_llm_loop.py tests/test_bfcl_tool_selection_sweep.py -q -> 25 passed
  • poetry run pytest tests/ -q -> 961 passed, 3 skipped, 1 warning
  • Qwen 27B cached smoke, simple_python_11: evaluator exact 1.0, predicted args base, height only
  • Qwen 27B cached 6-case argument subset: evaluator exact 0.5, 3 pass, remaining tags optional_value_mismatch, percentage_scale_mismatch

Notes

@SonAIengine

Copy link
Copy Markdown
Owner Author

Closing this internal stacked PR as superseded by the current main branch. The branch is preserved for history.

@SonAIengine SonAIengine closed this Aug 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant