This directory manages the collection of fundamental un-mutated trajectories. For the first version of raw conversation data, we utilize target frameworks like the Berkeley Function Calling Leaderboard (BFCL) data and securely collect multi-step completion metadata directly through the Langfuse API.
- Fetch and filter exclusively for successfully parsed benchmark tool-calling results.
- Transform and sequence the sequential steps into standard conversational structures suitable for downstream adversarial mutation.
Filters raw observed data against questions to compile the baseline traces.
It requires configuring internal script variables before execution:
QUESTIONS_FILE: Path to the targeted benchmark questionsjson.OBSERVATIONS_FILE: Path to thejsonlcontaining the Langfuse observation dumps.
Run directly via:
python langfuse/filter.py