Skip to content

Export BFCL failure tag subsets - #48

Closed
SonAIengine wants to merge 1 commit into
codex/xgen-equivalence-diagnosticsfrom
codex/bfcl-failure-tag-subsets
Closed

Export BFCL failure tag subsets#48
SonAIengine wants to merge 1 commit into
codex/xgen-equivalence-diagnosticsfrom
codex/bfcl-failure-tag-subsets

Conversation

@SonAIengine

Copy link
Copy Markdown
Owner

Summary

  • preserve non-empty BFCL model-loop failure_tags in extracted failure cases
  • emit tag_.txt subset files from hard-case bundles
  • document tag-based hard-case loops for near_duplicate_tool_surface experiments

Validation

  • poetry run pytest tests/test_bfcl_tool_selection_failures.py tests/test_bfcl_tool_selection_hard_cases.py -q
  • poetry run pytest tests/test_bfcl_tool_selection_hard_cases.py tests/test_bfcl_tool_selection_sweep.py tests/test_bfcl_tool_selection_llm_loop.py -q
  • poetry run ruff check .
  • poetry run ruff format --check .
  • poetry run pytest tests/ -q
  • REPORT=/tmp/gtc-bfcl-qwen027-cohesive-guard-limit25-equivalence.json OUT_DIR=/tmp/gtc-bfcl-qwen027-tag-subsets TOOL_SOURCES=retrieved REPORT_TOP_KS=5 TOP_K=5 FAILURE_CATEGORIES=retrieval_miss,candidate_ambiguity,argument_value_mismatch,call_count_mismatch make bfcl-hard-cases

Cached artifact produced tag_near_duplicate_tool_surface.txt with:

  • simple_python_6
  • simple_python_12
  • simple_python_21
  • simple_python_22

@SonAIengine

Copy link
Copy Markdown
Owner Author

Closing this internal stacked PR as superseded by the current main branch. The branch is preserved for history.

@SonAIengine SonAIengine closed this Aug 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant