[Klaud Cold] Delete 53 unused legacy single-node AgentX scripts from benchmarks/single_node/agentic - #3460
Conversation
…benchmarks/single_node/agentic #3428 moved single-node AgentX onto native srt-slurm recipes. Delete every script in benchmarks/single_node/agentic that no master config reaches any more. Keep the nine scripts still reached by search-space entries without an srt-recipe, plus the H100/H200 block32 installers and kernel_configs that the multi-node DeepSeek-V4.1-Flash recipes use. The README now marks the folder as legacy. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
Thanks for the contribution!
中文感谢你的贡献！
|
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.
Beyond the inline findings, I also checked whether any active master-config search-space entry still references one of the 52 deleted scripts or check_dsv4_dspark_checkpoint.py — none do (the only remaining reference to the checkpoint script is a historical perf-changelog.yaml entry), and the nine scripts the PR description says are still reached by unported search-space entries are indeed still present in the directory.
Extended reasoning...
Searched configs/*.yaml for the deleted script and checkpoint-script filenames (no active references found) and confirmed the nine "kept" scripts plus install_h100/h200_block32_configs.py remain in the directory, consistent with the PR description's claim that no active sweep is broken by the deletion.
|
|
||
| **MVP / experimental.** Nothing in this directory is an official InferenceX | ||
| benchmark. Results are not published on https://inferencex.com and are not | ||
| intended to be cited. |
There was a problem hiding this comment.
🟡 (optional) This commit deletes 52 retired agentic scripts and check_dsv4_dspark_checkpoint.py outright instead of moving them to a sibling deprecated/ directory, breaking AGENTS.md's archival convention that the repo already follows elsewhere. AGENTS.md:60 says retired benchmark scripts move into a sibling deprecated/ directory, not get deleted, once no active config uses them; benchmarks/multi_node/deprecated/ and benchmarks/single_node/fixed_seq_len/deprecated/ hold 129+ files under this same rule. Fix: create benchmarks/single_node/agentic/deprecated/ and git-mv the 52 scripts plus check_dsv4_dspark_checkpoint.py there instead of removing them, matching the established pattern and preserving launch-flag/perf history for future reference.
Why this was flagged
AGENTS.md:60 requires retired scripts to move into a sibling deprecated/ directory once no active config uses them, not be deleted; the repo already applies this at benchmarks/multi_node/deprecated/ and benchmarks/single_node/fixed_seq_len/deprecated/. This commit instead git-rm's all 52 scripts and check_dsv4_dspark_checkpoint.py under benchmarks/single_node/agentic/ with no deprecated/ subdirectory created, unlike every other retirement in this repo. A maintainer who later needs the historical launch flags, kernel tuning, or perf context for these configs — preserved for every other retired script in the repo — cannot recover it, only via old git history rather than the browsable deprecated/ location the convention establishes.
Verification: nit. Factual claim holds: AGENTS.md:60 states "Follow the existing benchmark-script archival convention, moving retired scripts into the sibling deprecated/ directory only when no active config still uses them." The sibling deprecated/ directories cited exist and are in active use (benchmarks/multi_node/deprecated/ e.g. dsv4_fp4_gb200_llmd-vllm-disagg.sh;… | nit. Real deviation from a…
| > **Legacy.** Single-node AgentX configs now run as declarative srt-slurm | ||
| > recipes (`benchmarks/single_node/srt-slurm-recipes/<model>/<engine>/<sku>/agentic.yaml`, | ||
| > see #3428). The only bash scripts left here are the ones a master config still | ||
| > reaches through a search-space entry without an `srt-recipe:`; delete each one |
There was a problem hiding this comment.
🟡 nit (optional): the new README note says the kept files are install_h1{0,2}0_block32_configs.py, but that brace expansion actually yields install_h100_block32_configs.py and install_h120_block32_configs.py, not the real install_h200_block32_configs.py. Fix: write install_h{100,200}_block32_configs.py (or spell both names out) so it correctly refers to the two files that actually exist.
Why this was flagged
The README text at benchmarks/single_node/agentic/README.md:6 uses shell brace-expansion notation install_h1{0,2}0_block32_configs.py, which expands to install_h100_block32_configs.py and install_h120_block32_configs.py. The actual second file is install_h200_block32_configs.py, which the pattern never names. A reader following this doc pointer to find the H200 install script will not find one matching the literal notation and may miss it, unlike a correct brace pattern such as install_h{100,200}_block32_configs.py.
Verification: nit: README.md:7 writes install_h1{0,2}0_block32_configs.py. Under shell brace-expansion the {0,2} sits between install_h1 and 0, so it yields install_h100_block32_configs.py and install_h120_block32_configs.py. The real files (confirmed via ls) are install_h100_block32_configs.py and install_h200_block32_configs.py; install_h120... does not exist and the H200 name is never…
1 participant
Summary
#3428 moved single-node AgentX onto native srt-slurm, so AgentX configs are declarative YAML recipes under
benchmarks/single_node/srt-slurm-recipes/*/*/*/agentic.yaml. This deletes the 53 legacy scripts inbenchmarks/single_node/agentic/that no master config reaches any more (−9,528 lines).Kept (still referenced)
The folder is not deleted outright. These nine scripts are still reached by master-config search-space entries that have no
srt-recipe:yet, so deleting them would break those sweeps:dsv41flash_fp4_mi355x_atom_mtp.shdsv41flash-fp4-mi355x-atom-agentic-dsparkglm5.2_fp4_mi355x_atom_mtp.shglm5.2-fp4-mi355x-atom-agentic-mtpkimik3_fp4_mi355x_atom_mtp.shkimik3-fp4-mi355x-atom-agentic-mtpkimik3_fp4_mi355x_mtp.shkimik3-fp4-mi355x-vllm-agentic-mtpminimaxm3_fp4_mi355x_atom_mtp.shminimaxm3-fp4-mi355x-atom-agentic-mtpminimaxm3_fp4_mi355x_mtp.shminimaxm3-fp4-mi355x-vllm-agentic-mtpqwen3.5_fp4_b300_sglang_mtp.shqwen3.5-fp4-b300-sglang-agentic-power-abqwen3.5_fp8_b300_sglang_mtp.shqwen3.5-fp8-b300-sglang-agentic-power-abqwen3.8next_fp8_h100_mtp.shqwen3.8next-fp8-h100-sglang-agentic-mtpAlso kept:
install_h100_block32_configs.py,install_h200_block32_configs.pyandkernel_configs/, which the multi-nodedsv41flash-h{1,2}00-block32.shsrt-slurm configs load; andREADME.md, now marked as legacy and telling porters to delete each remaining script once its config moves to a recipe.How "unused" was determined
For every single-node
agentic-codingsearch-space entry withoutsrt-recipe:, resolve the script the SKU launcher selects ({model-prefix}_{precision}_{sku}_{framework}[_mtp].sh, falling back to the untagged name), plus anything those scripts source from this folder. Everything else in the folder is deleted. The remaining references to deleted names are code comments, docs, andconfigs/deprecated/; none are executed.Test plan
utils/test_srt_single_node.py+utils/matrix_logic: 382 passed, 14 failed — identical failure set onmain(allmapfile: command not foundfrom macOS bash 3.2 in the pool-launcher tests; unrelated).perf-changelog.yamlchange, so no sweep is triggered.🤖 Generated with Claude Code