Tag BFCL near-duplicate tool-surface failures

[SonAIengine]

Summary

• add failure_tags to BFCL model-loop case results
• tag high-confidence candidate ambiguity caused by near-duplicate tool surfaces as near_duplicate_tool_surface
• include failure_tag_breakdown in model-loop and sweep summaries
• include tag breakdowns in row_vs_retrieved_deltas so row-pass/retrieved-fail subsets can be split into duplicate-surface vs other losses
• preserve tags in BFCL-compatible JSONL sidecar metadata

Why

The current 0.27 bottleneck is not just retrieval miss. In the 100-case qwen3.6-27B middle sweep, row-source passes but retrieved-source fails on 11 cases. Several are cases where the model selects a semantically near-equivalent sibling tool (solve_quadratic vs solve_quadratic_equation, math.gcd vs number_theory.gcd).

This change does not change scoring or hide exact-match failures. It marks high-confidence duplicate/equivalent tool surfaces so the next XGEN search work can focus on duplicate grouping, equivalence evidence, and selector rerank instead of blindly tuning BFCL exact names.

Evidence

Cached rerun of the 100-case qwen3.6-27B middle sweep with the new diagnostics:

poetry run python -m benchmarks.bfcl_tool_selection.sweep \
  --categories simple_python,multiple,parallel,parallel_multiple \
  --limit 25 \
  --top-ks 5 \
  --tool-sources row,retrieved \
  --model qwen3.6-27b \
  --llm-url http://127.0.0.1:18000/v1 \
  --disable-thinking \
  --candidate-selection-guidance \
  --cohesive-namespace-candidates \
  --cache-dir /tmp/gtc-bfcl-qwen027-cohesive-guard-limit25-cache \
  --concurrency 6 \
  --progress \
  --progress-every 25 \
  --output /tmp/gtc-bfcl-qwen027-cohesive-guard-limit25-tags.json

Output includes:

row_vs_retrieved k=5 repeat=1 paired=100 row_pass_loss=11 retrieved_on_row_pass=0.883 retrieval_layer_failures=argument_value_mismatch:2,candidate_ambiguity:8,retrieval_miss:1 retrieval_layer_tags=near_duplicate_tool_surface:4

Tagged cases:

• simple_python_6: solve_quadratic vs solve_quadratic_equation
• simple_python_12: geometry.circumference vs calculate_circumference
• simple_python_21: number_theory.gcd vs math.gcd
• simple_python_22: math.gcd vs number_theory.gcd

Validation

• poetry run pytest tests/test_bfcl_tool_selection_llm_loop.py tests/test_bfcl_tool_selection_sweep.py -q
• poetry run ruff check .
• poetry run ruff format --check .
• poetry run pytest tests/ -q (956 passed, 3 skipped)