Paper review 2026-08-04: 25 candidates

Reply with commands, one per line:
`/approve all` · `/approve 1,3-5` · `/reject 2` · `/edit 3 category=world_terminal tags=benchmark` (edit implies approve; `tags=-` clears tags). Valid category keys: see taxonomy.json.

### 1. Coding Agents as Test-Suite Auditors: Finding What Official Suites Miss While Approaching What They Catch
_Shuyang Xie, Shuxiao Xie, Feng Zhu, et al._ · arXiv 2026/08 · [paper](https://arxiv.org/abs/2608.01715)
**proposed: `software_testing`** · tags: none
> This paper uses off-the-shelf coding agents as test-suite auditors that build adversarial test suites to expose buggy accepted submissions missed by official online-judge test suites, with a certification chain to verify true bugs without relying on judge verdicts.
Agentic test generation for proactive correctness assurance independent of a specific reported bug fits software_testing.

### 2. LoopsBench: From Harness Engineering to Loop Engineering in Benchmarking Coding Agent
_Han Li, Zhemin Fang, Rili Feng, et al._ · arXiv 2026/07 · [paper](https://arxiv.org/abs/2608.00267)
**proposed: `software_development`** · tags: `benchmark`
> Introduces LOOPSBENCH, a long-horizon benchmark of 112 tasks structured as dependency DAGs of testable development units across 8 languages and 9 domains, evaluating coding agents' sustained iterative software development with regression tracking.
Benchmark tasks require agents to incrementally build out functionality with tracked prerequisites and regressions, matching feature/iterative development rather than debugging or from-scratch generation.

### 3. CADIR: A Cross-Backend Editable Intermediate Representation for Agentic CAD Generation
_Yu Liu, Jingzhe Ni, Yiming Chen, et al._ · arXiv 2026/08 · [paper](https://arxiv.org/abs/2608.00891)
**proposed: `cad`** · tags: none
> CADIR is a cross-backend editable intermediate representation for agentic CAD generation that preserves construction history and topological references, enabling reliable reconstruction and editing across FreeCAD, SolidWorks, and Fusion 360.
The task's ultimate purpose is producing editable CAD programs, matching the artifact/cad leaf.

### 4. WebCoderBench: Benchmarking Web Application Generation with Comprehensive and Interpretable Evaluation Metrics
_Chenxu Liu, Yingjie Fu, Wei Yang, et al._ · arXiv 2026/07 · [paper](https://arxiv.org/abs/2601.02430)
**proposed: `web`** · tags: `benchmark`
> WebCoderBench is a benchmark of 1,572 real-world user requirements for evaluating LLM-generated web applications, using 24 interpretable, ground-truth-free metrics across 9 perspectives. It benchmarks 12 LLMs and 2 code agents, finding no single dominant model.
Benchmark for generating web application code, matching the web leaf.

### 5. EcoTable: Cost-effective Table Integration in Data Lakes for Natural Language Queries
_Yuhui Wang, Jinqi Liu, Chengliang Chai, et al._ · arXiv 2026/08 · [paper](https://arxiv.org/abs/2606.26613)
**proposed: `database`** · tags: none
> EcoTable is a natural-language-driven data integration framework that uses a join-likelihood graph and LLMs to identify relevant tables, discover join paths, and generate transformation code to satisfy user SQL queries over data lakes.
The system generates database transformation/join code to serve SQL query answering, matching the database leaf's query/schema synthesis scope.

### 6. SERL-SQL: Selective Hindsight Distillation for Text-to-SQL Reinforcement Agentic Learning
_Tao Liu, Tao Feng, Xiangheng Li, et al._ · arXiv 2026/08 · [paper](https://arxiv.org/abs/2608.00485)
**proposed: `database`** · tags: none
> SERL-SQL is a reinforcement learning framework for multi-turn Text-to-SQL agents that uses execution-grounded hindsight distillation to reweight token-level rewards, improving SQL generation accuracy on BIRD and Spider.
An RL-trained agent that executes SQL and iterates on execution feedback to produce queries; the deliverable is SQL code, so it routes to database.

### 7. Agentic Harness for Real-World Compilers
_Yingwei Zheng, Cong Li, Shaohua Li, et al._ · arXiv 2026/07 · [paper](https://arxiv.org/abs/2603.20075)
**proposed: `systems`** · tags: `benchmark`
> llvm-harness provides agent-friendly tools, a 334-bug benchmark (llvm-bench), and a specialized mini-agent (llvm-autofix-mini) to help LLM agents understand and automatically fix bugs in the LLVM middle-end compiler. Experiments show substantial performance gains from the specialized harness over generic state-of-the-art agents.
An agent fixing bugs in compiler infrastructure delivers systems-level code, placing it in the non-expanded systems leaf per its own scope boundary.

### 8. Benchmarking LLMs on File System Design and Implementation: The Good, The Bad, and The Ugly
_Yuqi Xue, Daixuan Li, Jian Huang_ · arXiv 2026/07 · [paper](https://arxiv.org/abs/2608.00280)
**proposed: `systems`** · tags: `benchmark`
> Presents φ-Bench, a 505-task benchmark evaluating LLMs on file-system design and implementation across understanding, implementation, performance modeling, debugging, optimization, and new-feature tasks, with an AI-assisted task generation pipeline.
Tasks target file-system (systems-level) code, so the benchmark belongs to the systems domain rather than general software.

### 9. Kernel Foundry: A Diagnosis-driven Evolutionary Kernel Optimizer with Multi-Experts
_Zixuan Huang, Da Chen, Kecheng Huang, et al._ · arXiv 2026/08 · [paper](https://arxiv.org/abs/2605.30359)
**proposed: `systems`** · tags: `benchmark`
> Kernel Foundry is a diagnosis-driven evolutionary framework that uses expert-guided retrieval and multi-island search with structured feedback to optimize GPU kernel code, improving correctness and performance on KernelBench.
Produces systems-level performance-critical GPU kernel code, matching the systems leaf's kernel/runtime code generation focus.

### 10. Rethinking Agentic Kernel Generation for Emerging Accelerators
_Ruijie Gao, Jirong Yang, Barry Lyu, et al._ · arXiv 2026/08 · [paper](https://arxiv.org/abs/2608.00894)
**proposed: `systems`** · tags: none
> Zomboss is a compiler-mediated agentic framework that compiles workload-invariant accelerator machine semantics into a reusable interface, letting a neural agent search only workload-specific mapping decisions to generate correct, optimized kernels for emerging accelerators.
The deliverable is systems-level accelerator kernel code, placing it in the systems leaf.

### 11. LEAP: Lean Environment-Feedback via Adaptive Pruning for Code RL in GPU Kernel Generation
_Tankun Li, Zhi Chen, Yaohua Tang_ · arXiv 2026/08 · [paper](https://arxiv.org/abs/2608.01804)
**proposed: `systems`** · tags: none
> LEAP is a multi-turn RL framework with difficulty-conditioned pruning and rank-based rewards to train LLMs to generate CUDA/GPU kernels more efficiently, addressing sparse rewards and compilation latency.
Kernel/systems-level code generation training method; per benchmark-routing precedent (KernelBench-style tasks) this routes to systems.

### 12. AgentCompile: An LLM-Guided Compiler for Direct CUDA Inference
_Xuanzhe Li, Ziyan Weng, Zhiyu Zhu, et al._ · arXiv 2026/08 · [paper](https://arxiv.org/abs/2606.07665)
**proposed: `systems`** · tags: none
> AgentCompile is an LLM-guided CUDA inference compiler that combines LLM-provided advisory metadata for template-based kernel candidate generation with direct LLM generation of decode-critical kernels, validated for correctness and selected by measured latency. Integrated into a serving runtime, it achieves substantial speedups over PyTorch eager and vLLM.
An LLM-guided system generates and validates systems-level GPU kernel code, so it serves producing systems code (artifact/systems).

### 13. Open-Source LLM-Driven Formal Verification: A Multi-Agent Pipeline for RTL Repair
_Ha Trung Tran_ · arXiv 2026/07 · [paper](https://arxiv.org/abs/2607.28877)
**proposed: `hardware`** · tags: none
> An open-source multi-agent pipeline couples an LLM with Yosys, SymbiYosys, and Z3 to repair RTL designs through counterexample-guided iteration, proving correctness via k-induction. A case study repairs a real ALU bug, and a six-benchmark suite characterizes failure modes in fully open-source formal RTL repair.
Repairs and formally verifies hardware description (RTL) code, matching the hardware artifact leaf.

### 14. RTLCurator: Label-Efficient Data Curation for RTL Generation
_Siyang Cai, Cangyuan Li, Wenjing Chang, et al._ · arXiv 2026/07 · [paper](https://arxiv.org/abs/2607.29283)
**proposed: `hardware`** · tags: `training-data`
> RTLCurator is a label-efficient data curation method that learns a behavior-aware compatibility prior to select high-quality specification-code pairs from noisy synthesized RTL (Verilog) training corpora. Curating 80% of the corpus this way improves downstream RTL-generation model performance over training on the full or naively-filtered corpus.
Resource paper whose downstream task is hardware description code (RTL) generation.

### 15. FinHardBench: Can LLMs Generate Latency-Aware Hardware for Financial Computing?
_Weimin Fu, Hejia Zhang, Minghao Shao, et al._ · arXiv 2026/08 · [paper](https://arxiv.org/abs/2608.00909)
**proposed: `hardware`** · tags: `benchmark`
> FinHardBench is a 33-task benchmark evaluating whether LLMs can generate latency-aware FPGA hardware for financial computing, covering module generation, system-level design-space exploration, and specification-change adaptation.
The deliverable is hardware description code for FPGA financial systems, matching the artifact/hardware leaf.

### 16. PICopilot: An LLM-based Agentic Framework for Assisting Photonic Integrated Circuit Design via Script Generation
_Xiaohan Jiang, Zeyu Li, Wei Zhang, et al._ · arXiv 2026/08 · [paper](https://arxiv.org/abs/2608.01791)
**proposed: `hardware`** · tags: none
> PICopilot is a multi-agent LLM framework with retrieval-augmented generation that assists photonic integrated circuit design by generating design scripts from natural language instructions, outperforming baselines on a benchmark of PIC scripting tasks.
The agent produces hardware/chip design scripts (code) as the deliverable, fitting the hardware artifact leaf.

### 17. Automated Table Reproduction via Code Generation
_Doehyun Baek, Michael Pradel_ · arXiv 2026/07 · [paper](https://arxiv.org/abs/2602.10046)
**proposed: `world_research`** · tags: `benchmark`
> Artisan is an LLM agent that reproduces numeric result tables from software engineering papers by generating auditable, re-runnable reproduction scripts, using automated judging to prevent shortcuts. The accompanying Artisan-Bench benchmark spans 60 tasks from 23 papers and helped uncover 20 previously unknown errors.
The agent uses code as the instrument to validate and reproduce published research findings, matching the research/discovery world rather than delivering code as the end product.

### 18. LabGuard: Grounding Natural-Language Laboratory Rules into Runtime Guards for Embodied Laboratory Agents
_Jingpu Yang, Fengxian Ji, Zhengzhao Lai, et al._ · arXiv 2026/07 · [paper](https://arxiv.org/abs/2606.31045)
**proposed: `world_research`** · tags: `benchmark`
> LabGuard grounds natural-language laboratory safety rules into executable runtime guards for embodied laboratory agents, using a typed IR and a grounding pipeline. It introduces LabGuard-Bench and shows reduced unsafe events while preserving task success in self-driving-lab settings.
Targets embodied self-driving-lab agents executing procedures, matching the world_research scientific-discovery/self-driving-lab boundary.

### 19. CyberNeuro: A Privacy-Preserving Agentic Workbench for Cohort-Scale Neuroimage and Clinical Data Analysis
_Ran Ren, Junhong Tong, Yunxi Kong, et al._ · arXiv 2026/07 · [paper](https://arxiv.org/abs/2607.28841)
**proposed: `world_research`** · tags: `model`
> CyberNeuro is a multi-agent (Planner, Validator, Dispatcher, Reporter) agentic workbench with a tailored local LLM, WandaMind, that automates cohort-scale neuroimaging and clinical data analysis pipelines from natural language while preserving privacy. It substantially improves domain accuracy and reduces token cost versus cloud baselines.
An agent driving data-analysis pipelines toward research insight is world_research, not a case study of a generic technique.

### 20. Scaling Scientific Discovery Environments for Turn-Level Agentic RL
_Yucheng Xu, Keyi Zhang, Yuyang Yu, et al._ · arXiv 2026/07 · [paper](https://arxiv.org/abs/2607.28990)
**proposed: `world_research`** · tags: `training-data` `model`
> SciDisco trains scientific-discovery agents via process-verifiable environments (SciThèque) that compile hypotheses, datasets, and hidden evidence graphs, using DAG-grounded trajectory synthesis and turn-level RL (DiscoPO) to reward verifiable analytical progress. The resulting SciDisco-14B achieves state-of-the-art on hypothesis-driven scientific data analysis benchmarks.
An agent interacts with an execution environment to produce data-driven scientific claims, matching world_research's data-analysis/discovery agent scope.

### 21. SeekBrain: An Autonomous Multi-Agent System for Accelerating Neuroscience Discovery
_Jiamin Wu, Peishan Xiang, Jingyang Chen, et al._ · arXiv 2026/07 · [paper](https://arxiv.org/abs/2607.29347)
**proposed: `world_research`** · tags: none
> SeekBrain is an autonomous multi-agent framework that constructs and executes analysis pipelines from code-paper pairs to accelerate neuroscience discovery across multimodal datasets. It outperforms baselines on the BrainArena benchmark and produced real scientific findings in zebrafish and mouse studies.
Agent writes and executes analysis code as the instrument for scientific discovery, matching world_research.

### 22. MOT-SR: Multi-Objective Tool-Augmented Scientific Equation Discovery with Large Language Models
_Boxiao Wang, Runxiang Wang, Kai Li, et al._ · arXiv 2026/07 · [paper](https://arxiv.org/abs/2607.29561)
**proposed: `world_research`** · tags: none
> MOT-SR is a tool-augmented, multi-objective LLM framework for symbolic regression that uses collaborative agent modules to discover interpretable scientific equations, validated on gravitational-wave orbital modeling tasks.
Agentic, tool-using LLM system performing scientific discovery/data analysis, matching world_research's AI-scientist style boundary.

### 23. AgentHPOBench: A Benchmark For Evaluating LLM Agents as Sequential Hyperparameter Optimizers
_Tianyu Huai, Tingshuo Fan, Xinchi Chen, et al._ · arXiv 2026/07 · [paper](https://arxiv.org/abs/2607.29626)
**proposed: `world_research`** · tags: `benchmark`
> AgentHPOBench is a sequential benchmark of 30 executable ML tasks evaluating LLM agents' ability to interpret experimental logs and iteratively propose hyperparameter configurations, testing 12 agents against conventional HPO baselines.
MLE-bench style benchmark of agents conducting iterative ML experiments, matching world_research's ML-engineering interaction focus.

### 24. Learning to Coordinate Symbolic Tools: LLM Agents for Verified Sum-of-Squares Certificates
_Bohan Chen, Shivam N. Patel, Richard Hoffmann, et al._ · arXiv 2026/07 · [paper](https://arxiv.org/abs/2608.00326)
**proposed: `world_research`** · tags: none
> An LLM agent trained via SFT and GRPO with verifier-grounded rewards coordinates symbolic SymPy tool calls to find and verify weighted sum-of-squares decompositions proving polynomial nonnegativity. It combines algebraic skill training with executable symbolic tools for exactly-checkable math outputs.
An agent uses code/tool execution to pursue verified mathematical knowledge, matching the theorem-proving/math-discovery boundary of world_research.

### 25. VFEAgent: A Multimodal Agent Framework for End-to-End Automated Finite Element Analysis
_Jiachen Zhang, Junyi Lao, Chenghao Liu, et al._ · arXiv 2026/08 · [paper](https://arxiv.org/abs/2605.28978)
**proposed: `world_research`** · tags: none
> VFEAgent is a multimodal multi-agent system that automates finite element analysis by extracting FEA specifications from images/text and synthesizing verified, self-debugged simulation code. It achieves high success rates in generating physically valid engineering simulations across mechanics scenarios.
Generated code is the instrument to run engineering simulations and obtain valid analysis results, matching world_research's operation of scientific/simulation software for discovery-like outcomes.

<details><summary>machine payload (do not edit)</summary>

```json
[{"paper": {"id": "2608.01715", "title": "Coding Agents as Test-Suite Auditors: Finding What Official Suites Miss While Approaching What They Catch", "authors": ["Shuyang Xie", "Shuxiao Xie", "Feng Zhu", "Yanli Ji", "Wangmeng Zuo"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-03", "links": {"paper": "https://arxiv.org/abs/2608.01715", "github": "", "website": ""}}, "category": "software_testing", "tags": [], "summary": "This paper uses off-the-shelf coding agents as test-suite auditors that build adversarial test suites to expose buggy accepted submissions missed by official online-judge test suites, with a certification chain to verify true bugs without relying on judge verdicts.", "reason": "Agentic test generation for proactive correctness assurance independent of a specific reported bug fits software_testing.", "source": "crawl"}, {"paper": {"id": "2608.00267", "title": "LoopsBench: From Harness Engineering to Loop Engineering in Benchmarking Coding Agent", "authors": ["Han Li", "Zhemin Fang", "Rili Feng", "Yingqi Zhao", "Jiaheng Liu", "Pengfei Gao", "He Ye", "Dayi Lin", "Qingwei Lin", "Saravan Rajmohan", "Dongmei Zhang"], "venue": "arXiv 2026/07", "category": "", "published": "2026-07-31", "links": {"paper": "https://arxiv.org/abs/2608.00267", "github": "", "website": ""}}, "category": "software_development", "tags": ["benchmark"], "summary": "Introduces LOOPSBENCH, a long-horizon benchmark of 112 tasks structured as dependency DAGs of testable development units across 8 languages and 9 domains, evaluating coding agents' sustained iterative software development with regression tracking.", "reason": "Benchmark tasks require agents to incrementally build out functionality with tracked prerequisites and regressions, matching feature/iterative development rather than debugging or from-scratch generation.", "source": "crawl"}, {"paper": {"id": "2608.00891", "title": "CADIR: A Cross-Backend Editable Intermediate Representation for Agentic CAD Generation", "authors": ["Yu Liu", "Jingzhe Ni", "Yiming Chen", "Junqi Huang", "Ruofeng Tong", "Min Tang", "Peng Du"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-01", "links": {"paper": "https://arxiv.org/abs/2608.00891", "github": "", "website": ""}}, "category": "cad", "tags": [], "summary": "CADIR is a cross-backend editable intermediate representation for agentic CAD generation that preserves construction history and topological references, enabling reliable reconstruction and editing across FreeCAD, SolidWorks, and Fusion 360.", "reason": "The task's ultimate purpose is producing editable CAD programs, matching the artifact/cad leaf.", "source": "crawl"}, {"paper": {"id": "2601.02430", "title": "WebCoderBench: Benchmarking Web Application Generation with Comprehensive and Interpretable Evaluation Metrics", "authors": ["Chenxu Liu", "Yingjie Fu", "Wei Yang", "Ying Zhang", "Tao Xie"], "venue": "arXiv 2026/07", "category": "", "published": "2026-07-31", "links": {"paper": "https://arxiv.org/abs/2601.02430", "github": "", "website": ""}}, "category": "web", "tags": ["benchmark"], "summary": "WebCoderBench is a benchmark of 1,572 real-world user requirements for evaluating LLM-generated web applications, using 24 interpretable, ground-truth-free metrics across 9 perspectives. It benchmarks 12 LLMs and 2 code agents, finding no single dominant model.", "reason": "Benchmark for generating web application code, matching the web leaf.", "source": "crawl"}, {"paper": {"id": "2606.26613", "title": "EcoTable: Cost-effective Table Integration in Data Lakes for Natural Language Queries", "authors": ["Yuhui Wang", "Jinqi Liu", "Chengliang Chai", "Hangyu Zhao", "Yuhao Deng", "Yuyu Luo", "Xin Tang", "Ye Yuan", "Guoren Wang", "Fengjin Wang", "Lei Cao"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-01", "links": {"paper": "https://arxiv.org/abs/2606.26613", "github": "", "website": ""}}, "category": "database", "tags": [], "summary": "EcoTable is a natural-language-driven data integration framework that uses a join-likelihood graph and LLMs to identify relevant tables, discover join paths, and generate transformation code to satisfy user SQL queries over data lakes.", "reason": "The system generates database transformation/join code to serve SQL query answering, matching the database leaf's query/schema synthesis scope.", "source": "crawl"}, {"paper": {"id": "2608.00485", "title": "SERL-SQL: Selective Hindsight Distillation for Text-to-SQL Reinforcement Agentic Learning", "authors": ["Tao Liu", "Tao Feng", "Xiangheng Li", "Jinwang Song", "Yifan Li", "Xiaoqing Cheng", "Dixuan Zhang", "Siquan Li", "Lin Lan", "Hongying Zan", "Kunli Zhang", "Chao Wu"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-01", "links": {"paper": "https://arxiv.org/abs/2608.00485", "github": "", "website": ""}}, "category": "database", "tags": [], "summary": "SERL-SQL is a reinforcement learning framework for multi-turn Text-to-SQL agents that uses execution-grounded hindsight distillation to reweight token-level rewards, improving SQL generation accuracy on BIRD and Spider.", "reason": "An RL-trained agent that executes SQL and iterates on execution feedback to produce queries; the deliverable is SQL code, so it routes to database.", "source": "crawl"}, {"paper": {"id": "2603.20075", "title": "Agentic Harness for Real-World Compilers", "authors": ["Yingwei Zheng", "Cong Li", "Shaohua Li", "Yuqun Zhang", "Zhendong Su"], "venue": "arXiv 2026/07", "category": "", "published": "2026-07-31", "links": {"paper": "https://arxiv.org/abs/2603.20075", "github": "", "website": ""}}, "category": "systems", "tags": ["benchmark"], "summary": "llvm-harness provides agent-friendly tools, a 334-bug benchmark (llvm-bench), and a specialized mini-agent (llvm-autofix-mini) to help LLM agents understand and automatically fix bugs in the LLVM middle-end compiler. Experiments show substantial performance gains from the specialized harness over generic state-of-the-art agents.", "reason": "An agent fixing bugs in compiler infrastructure delivers systems-level code, placing it in the non-expanded systems leaf per its own scope boundary.", "source": "crawl"}, {"paper": {"id": "2608.00280", "title": "Benchmarking LLMs on File System Design and Implementation: The Good, The Bad, and The Ugly", "authors": ["Yuqi Xue", "Daixuan Li", "Jian Huang"], "venue": "arXiv 2026/07", "category": "", "published": "2026-07-31", "links": {"paper": "https://arxiv.org/abs/2608.00280", "github": "", "website": ""}}, "category": "systems", "tags": ["benchmark"], "summary": "Presents φ-Bench, a 505-task benchmark evaluating LLMs on file-system design and implementation across understanding, implementation, performance modeling, debugging, optimization, and new-feature tasks, with an AI-assisted task generation pipeline.", "reason": "Tasks target file-system (systems-level) code, so the benchmark belongs to the systems domain rather than general software.", "source": "crawl"}, {"paper": {"id": "2605.30359", "title": "Kernel Foundry: A Diagnosis-driven Evolutionary Kernel Optimizer with Multi-Experts", "authors": ["Zixuan Huang", "Da Chen", "Kecheng Huang", "Lihao Yin", "Xing Li", "Huiling Zhen", "Mingxuan Yuan", "Zili Shao"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-03", "links": {"paper": "https://arxiv.org/abs/2605.30359", "github": "", "website": ""}}, "category": "systems", "tags": ["benchmark"], "summary": "Kernel Foundry is a diagnosis-driven evolutionary framework that uses expert-guided retrieval and multi-island search with structured feedback to optimize GPU kernel code, improving correctness and performance on KernelBench.", "reason": "Produces systems-level performance-critical GPU kernel code, matching the systems leaf's kernel/runtime code generation focus.", "source": "crawl"}, {"paper": {"id": "2608.00894", "title": "Rethinking Agentic Kernel Generation for Emerging Accelerators", "authors": ["Ruijie Gao", "Jirong Yang", "Barry Lyu", "Haoran Jin", "Nathan Bleier"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-01", "links": {"paper": "https://arxiv.org/abs/2608.00894", "github": "", "website": ""}}, "category": "systems", "tags": [], "summary": "Zomboss is a compiler-mediated agentic framework that compiles workload-invariant accelerator machine semantics into a reusable interface, letting a neural agent search only workload-specific mapping decisions to generate correct, optimized kernels for emerging accelerators.", "reason": "The deliverable is systems-level accelerator kernel code, placing it in the systems leaf.", "source": "crawl"}, {"paper": {"id": "2608.01804", "title": "LEAP: Lean Environment-Feedback via Adaptive Pruning for Code RL in GPU Kernel Generation", "authors": ["Tankun Li", "Zhi Chen", "Yaohua Tang"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-03", "links": {"paper": "https://arxiv.org/abs/2608.01804", "github": "", "website": ""}}, "category": "systems", "tags": [], "summary": "LEAP is a multi-turn RL framework with difficulty-conditioned pruning and rank-based rewards to train LLMs to generate CUDA/GPU kernels more efficiently, addressing sparse rewards and compilation latency.", "reason": "Kernel/systems-level code generation training method; per benchmark-routing precedent (KernelBench-style tasks) this routes to systems.", "source": "crawl"}, {"paper": {"id": "2606.07665", "title": "AgentCompile: An LLM-Guided Compiler for Direct CUDA Inference", "authors": ["Xuanzhe Li", "Ziyan Weng", "Zhiyu Zhu", "Junhui Hou"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-03", "links": {"paper": "https://arxiv.org/abs/2606.07665", "github": "", "website": ""}}, "category": "systems", "tags": [], "summary": "AgentCompile is an LLM-guided CUDA inference compiler that combines LLM-provided advisory metadata for template-based kernel candidate generation with direct LLM generation of decode-critical kernels, validated for correctness and selected by measured latency. Integrated into a serving runtime, it achieves substantial speedups over PyTorch eager and vLLM.", "reason": "An LLM-guided system generates and validates systems-level GPU kernel code, so it serves producing systems code (artifact/systems).", "source": "crawl"}, {"paper": {"id": "2607.28877", "title": "Open-Source LLM-Driven Formal Verification: A Multi-Agent Pipeline for RTL Repair", "authors": ["Ha Trung Tran"], "venue": "arXiv 2026/07", "category": "", "published": "2026-07-30", "links": {"paper": "https://arxiv.org/abs/2607.28877", "github": "", "website": ""}}, "category": "hardware", "tags": [], "summary": "An open-source multi-agent pipeline couples an LLM with Yosys, SymbiYosys, and Z3 to repair RTL designs through counterexample-guided iteration, proving correctness via k-induction. A case study repairs a real ALU bug, and a six-benchmark suite characterizes failure modes in fully open-source formal RTL repair.", "reason": "Repairs and formally verifies hardware description (RTL) code, matching the hardware artifact leaf.", "source": "crawl"}, {"paper": {"id": "2607.29283", "title": "RTLCurator: Label-Efficient Data Curation for RTL Generation", "authors": ["Siyang Cai", "Cangyuan Li", "Wenjing Chang", "Kun Wang", "Haoyu Gao", "Yinhe Han", "Ying Wang"], "venue": "arXiv 2026/07", "category": "", "published": "2026-07-31", "links": {"paper": "https://arxiv.org/abs/2607.29283", "github": "", "website": ""}}, "category": "hardware", "tags": ["training-data"], "summary": "RTLCurator is a label-efficient data curation method that learns a behavior-aware compatibility prior to select high-quality specification-code pairs from noisy synthesized RTL (Verilog) training corpora. Curating 80% of the corpus this way improves downstream RTL-generation model performance over training on the full or naively-filtered corpus.", "reason": "Resource paper whose downstream task is hardware description code (RTL) generation.", "source": "crawl"}, {"paper": {"id": "2608.00909", "title": "FinHardBench: Can LLMs Generate Latency-Aware Hardware for Financial Computing?", "authors": ["Weimin Fu", "Hejia Zhang", "Minghao Shao", "Zeng Wang", "Johann Knechtel", "Ozgur Sinanoglu", "Muhammad Shafique", "Ramesh Karri", "Xiaolong Guo"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-02", "links": {"paper": "https://arxiv.org/abs/2608.00909", "github": "", "website": ""}}, "category": "hardware", "tags": ["benchmark"], "summary": "FinHardBench is a 33-task benchmark evaluating whether LLMs can generate latency-aware FPGA hardware for financial computing, covering module generation, system-level design-space exploration, and specification-change adaptation.", "reason": "The deliverable is hardware description code for FPGA financial systems, matching the artifact/hardware leaf.", "source": "crawl"}, {"paper": {"id": "2608.01791", "title": "PICopilot: An LLM-based Agentic Framework for Assisting Photonic Integrated Circuit Design via Script Generation", "authors": ["Xiaohan Jiang", "Zeyu Li", "Wei Zhang", "Jiang Xu"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-03", "links": {"paper": "https://arxiv.org/abs/2608.01791", "github": "", "website": ""}}, "category": "hardware", "tags": [], "summary": "PICopilot is a multi-agent LLM framework with retrieval-augmented generation that assists photonic integrated circuit design by generating design scripts from natural language instructions, outperforming baselines on a benchmark of PIC scripting tasks.", "reason": "The agent produces hardware/chip design scripts (code) as the deliverable, fitting the hardware artifact leaf.", "source": "crawl"}, {"paper": {"id": "2602.10046", "title": "Automated Table Reproduction via Code Generation", "authors": ["Doehyun Baek", "Michael Pradel"], "venue": "arXiv 2026/07", "category": "", "published": "2026-07-31", "links": {"paper": "https://arxiv.org/abs/2602.10046", "github": "", "website": ""}}, "category": "world_research", "tags": ["benchmark"], "summary": "Artisan is an LLM agent that reproduces numeric result tables from software engineering papers by generating auditable, re-runnable reproduction scripts, using automated judging to prevent shortcuts. The accompanying Artisan-Bench benchmark spans 60 tasks from 23 papers and helped uncover 20 previously unknown errors.", "reason": "The agent uses code as the instrument to validate and reproduce published research findings, matching the research/discovery world rather than delivering code as the end product.", "source": "crawl"}, {"paper": {"id": "2606.31045", "title": "LabGuard: Grounding Natural-Language Laboratory Rules into Runtime Guards for Embodied Laboratory Agents", "authors": ["Jingpu Yang", "Fengxian Ji", "Zhengzhao Lai", "Zhexuan Cui", "Guangxian Ouyang", "Qian Jiang", "Fan Zhang", "Min Peng", "Qianqian Xie", "Preslav Nakov", "Zhuohan Xie"], "venue": "arXiv 2026/07", "category": "", "published": "2026-07-31", "links": {"paper": "https://arxiv.org/abs/2606.31045", "github": "", "website": ""}}, "category": "world_research", "tags": ["benchmark"], "summary": "LabGuard grounds natural-language laboratory safety rules into executable runtime guards for embodied laboratory agents, using a typed IR and a grounding pipeline. It introduces LabGuard-Bench and shows reduced unsafe events while preserving task success in self-driving-lab settings.", "reason": "Targets embodied self-driving-lab agents executing procedures, matching the world_research scientific-discovery/self-driving-lab boundary.", "source": "crawl"}, {"paper": {"id": "2607.28841", "title": "CyberNeuro: A Privacy-Preserving Agentic Workbench for Cohort-Scale Neuroimage and Clinical Data Analysis", "authors": ["Ran Ren", "Junhong Tong", "Yunxi Kong", "Yiyao Chen", "Yucheng Li", "Kunhao Zhou", "Shaoqi Wang", "Yuxiang Tao", "Shuheng Cao", "Zhihao Fan", "Marissa DiPiero", "Tingting Dan", "Guorong Wu"], "venue": "arXiv 2026/07", "category": "", "published": "2026-07-30", "links": {"paper": "https://arxiv.org/abs/2607.28841", "github": "", "website": ""}}, "category": "world_research", "tags": ["model"], "summary": "CyberNeuro is a multi-agent (Planner, Validator, Dispatcher, Reporter) agentic workbench with a tailored local LLM, WandaMind, that automates cohort-scale neuroimaging and clinical data analysis pipelines from natural language while preserving privacy. It substantially improves domain accuracy and reduces token cost versus cloud baselines.", "reason": "An agent driving data-analysis pipelines toward research insight is world_research, not a case study of a generic technique.", "source": "crawl"}, {"paper": {"id": "2607.28990", "title": "Scaling Scientific Discovery Environments for Turn-Level Agentic RL", "authors": ["Yucheng Xu", "Keyi Zhang", "Yuyang Yu", "Min Zhang", "Shiyuan Meng", "Pei Chu", "Zhongying Tu"], "venue": "arXiv 2026/07", "category": "", "published": "2026-07-31", "links": {"paper": "https://arxiv.org/abs/2607.28990", "github": "", "website": ""}}, "category": "world_research", "tags": ["training-data", "model"], "summary": "SciDisco trains scientific-discovery agents via process-verifiable environments (SciThèque) that compile hypotheses, datasets, and hidden evidence graphs, using DAG-grounded trajectory synthesis and turn-level RL (DiscoPO) to reward verifiable analytical progress. The resulting SciDisco-14B achieves state-of-the-art on hypothesis-driven scientific data analysis benchmarks.", "reason": "An agent interacts with an execution environment to produce data-driven scientific claims, matching world_research's data-analysis/discovery agent scope.", "source": "crawl"}, {"paper": {"id": "2607.29347", "title": "SeekBrain: An Autonomous Multi-Agent System for Accelerating Neuroscience Discovery", "authors": ["Jiamin Wu", "Peishan Xiang", "Jingyang Chen", "Yuqing Zhu", "Yuxi Li", "Ling Luo", "Qihao Zheng", "Jialiang Zu", "Yongchao Wu", "Mindong Liu", "Haitao Wu", "Chaofan Hu", "Yijie Sun", "Yuqi Hang", "Yu Zhu", "Shuo Li", "Yue Fan", "Shiyang Feng", "Wanghan Xu", "Tianlei Zhang", "Jie Zhang", "Wenlong Zhang", "Bo Zhang", "Kai Wang", "Lei Bai", "Mianxin Liu", "Wanli Ouyang", "Jiulin Du", "Chunfeng Song"], "venue": "arXiv 2026/07", "category": "", "published": "2026-07-31", "links": {"paper": "https://arxiv.org/abs/2607.29347", "github": "", "website": ""}}, "category": "world_research", "tags": [], "summary": "SeekBrain is an autonomous multi-agent framework that constructs and executes analysis pipelines from code-paper pairs to accelerate neuroscience discovery across multimodal datasets. It outperforms baselines on the BrainArena benchmark and produced real scientific findings in zebrafish and mouse studies.", "reason": "Agent writes and executes analysis code as the instrument for scientific discovery, matching world_research.", "source": "crawl"}, {"paper": {"id": "2607.29561", "title": "MOT-SR: Multi-Objective Tool-Augmented Scientific Equation Discovery with Large Language Models", "authors": ["Boxiao Wang", "Runxiang Wang", "Kai Li", "Chongming Li", "Zhiwei Chen", "Yifan Zhang", "Jian Cheng"], "venue": "arXiv 2026/07", "category": "", "published": "2026-07-31", "links": {"paper": "https://arxiv.org/abs/2607.29561", "github": "", "website": ""}}, "category": "world_research", "tags": [], "summary": "MOT-SR is a tool-augmented, multi-objective LLM framework for symbolic regression that uses collaborative agent modules to discover interpretable scientific equations, validated on gravitational-wave orbital modeling tasks.", "reason": "Agentic, tool-using LLM system performing scientific discovery/data analysis, matching world_research's AI-scientist style boundary.", "source": "crawl"}, {"paper": {"id": "2607.29626", "title": "AgentHPOBench: A Benchmark For Evaluating LLM Agents as Sequential Hyperparameter Optimizers", "authors": ["Tianyu Huai", "Tingshuo Fan", "Xinchi Chen", "Yining Zheng", "Yuxin Wang", "Shuang Chen", "Jie Zhou", "Xuanjing Huang"], "venue": "arXiv 2026/07", "category": "", "published": "2026-07-31", "links": {"paper": "https://arxiv.org/abs/2607.29626", "github": "", "website": ""}}, "category": "world_research", "tags": ["benchmark"], "summary": "AgentHPOBench is a sequential benchmark of 30 executable ML tasks evaluating LLM agents' ability to interpret experimental logs and iteratively propose hyperparameter configurations, testing 12 agents against conventional HPO baselines.", "reason": "MLE-bench style benchmark of agents conducting iterative ML experiments, matching world_research's ML-engineering interaction focus.", "source": "crawl"}, {"paper": {"id": "2608.00326", "title": "Learning to Coordinate Symbolic Tools: LLM Agents for Verified Sum-of-Squares Certificates", "authors": ["Bohan Chen", "Shivam N. Patel", "Richard Hoffmann", "Sam Looi", "Tony Yue Yu"], "venue": "arXiv 2026/07", "category": "", "published": "2026-07-31", "links": {"paper": "https://arxiv.org/abs/2608.00326", "github": "", "website": ""}}, "category": "world_research", "tags": [], "summary": "An LLM agent trained via SFT and GRPO with verifier-grounded rewards coordinates symbolic SymPy tool calls to find and verify weighted sum-of-squares decompositions proving polynomial nonnegativity. It combines algebraic skill training with executable symbolic tools for exactly-checkable math outputs.", "reason": "An agent uses code/tool execution to pursue verified mathematical knowledge, matching the theorem-proving/math-discovery boundary of world_research.", "source": "crawl"}, {"paper": {"id": "2605.28978", "title": "VFEAgent: A Multimodal Agent Framework for End-to-End Automated Finite Element Analysis", "authors": ["Jiachen Zhang", "Junyi Lao", "Chenghao Liu", "Siyuan Liu", "Shixin Wu", "Linsen Zhang", "Boyu Wang", "Songfang Huang"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-03", "links": {"paper": "https://arxiv.org/abs/2605.28978", "github": "", "website": ""}}, "category": "world_research", "tags": [], "summary": "VFEAgent is a multimodal multi-agent system that automates finite element analysis by extracting FEA specifications from images/text and synthesizing verified, self-debugged simulation code. It achieves high success rates in generating physically valid engineering simulations across mechanics scenarios.", "reason": "Generated code is the instrument to run engineering simulations and obtain valid analysis results, matching world_research's operation of scientific/simulation software for discovery-like outcomes.", "source": "crawl"}]
```

</details>

Navigation Menu

Uh oh!

Paper review 2026-08-04: 25 candidates #352

1. Coding Agents as Test-Suite Auditors: Finding What Official Suites Miss While Approaching What They Catch

2. LoopsBench: From Harness Engineering to Loop Engineering in Benchmarking Coding Agent

3. CADIR: A Cross-Backend Editable Intermediate Representation for Agentic CAD Generation

4. WebCoderBench: Benchmarking Web Application Generation with Comprehensive and Interpretable Evaluation Metrics

5. EcoTable: Cost-effective Table Integration in Data Lakes for Natural Language Queries

6. SERL-SQL: Selective Hindsight Distillation for Text-to-SQL Reinforcement Agentic Learning

7. Agentic Harness for Real-World Compilers

8. Benchmarking LLMs on File System Design and Implementation: The Good, The Bad, and The Ugly

9. Kernel Foundry: A Diagnosis-driven Evolutionary Kernel Optimizer with Multi-Experts

10. Rethinking Agentic Kernel Generation for Emerging Accelerators

11. LEAP: Lean Environment-Feedback via Adaptive Pruning for Code RL in GPU Kernel Generation

12. AgentCompile: An LLM-Guided Compiler for Direct CUDA Inference

13. Open-Source LLM-Driven Formal Verification: A Multi-Agent Pipeline for RTL Repair

14. RTLCurator: Label-Efficient Data Curation for RTL Generation

15. FinHardBench: Can LLMs Generate Latency-Aware Hardware for Financial Computing?

16. PICopilot: An LLM-based Agentic Framework for Assisting Photonic Integrated Circuit Design via Script Generation

17. Automated Table Reproduction via Code Generation

18. LabGuard: Grounding Natural-Language Laboratory Rules into Runtime Guards for Embodied Laboratory Agents

19. CyberNeuro: A Privacy-Preserving Agentic Workbench for Cohort-Scale Neuroimage and Clinical Data Analysis

20. Scaling Scientific Discovery Environments for Turn-Level Agentic RL

21. SeekBrain: An Autonomous Multi-Agent System for Accelerating Neuroscience Discovery

22. MOT-SR: Multi-Objective Tool-Augmented Scientific Equation Discovery with Large Language Models

23. AgentHPOBench: A Benchmark For Evaluating LLM Agents as Sequential Hyperparameter Optimizers

24. Learning to Coordinate Symbolic Tools: LLM Agents for Verified Sum-of-Squares Certificates

25. VFEAgent: A Multimodal Agent Framework for End-to-End Automated Finite Element Analysis

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Uh oh!

Paper review 2026-08-04: 25 candidates #352

Description

1. Coding Agents as Test-Suite Auditors: Finding What Official Suites Miss While Approaching What They Catch

2. LoopsBench: From Harness Engineering to Loop Engineering in Benchmarking Coding Agent

3. CADIR: A Cross-Backend Editable Intermediate Representation for Agentic CAD Generation

4. WebCoderBench: Benchmarking Web Application Generation with Comprehensive and Interpretable Evaluation Metrics

5. EcoTable: Cost-effective Table Integration in Data Lakes for Natural Language Queries

6. SERL-SQL: Selective Hindsight Distillation for Text-to-SQL Reinforcement Agentic Learning

7. Agentic Harness for Real-World Compilers

8. Benchmarking LLMs on File System Design and Implementation: The Good, The Bad, and The Ugly

9. Kernel Foundry: A Diagnosis-driven Evolutionary Kernel Optimizer with Multi-Experts

10. Rethinking Agentic Kernel Generation for Emerging Accelerators

11. LEAP: Lean Environment-Feedback via Adaptive Pruning for Code RL in GPU Kernel Generation

12. AgentCompile: An LLM-Guided Compiler for Direct CUDA Inference

13. Open-Source LLM-Driven Formal Verification: A Multi-Agent Pipeline for RTL Repair

14. RTLCurator: Label-Efficient Data Curation for RTL Generation

15. FinHardBench: Can LLMs Generate Latency-Aware Hardware for Financial Computing?

16. PICopilot: An LLM-based Agentic Framework for Assisting Photonic Integrated Circuit Design via Script Generation

17. Automated Table Reproduction via Code Generation

18. LabGuard: Grounding Natural-Language Laboratory Rules into Runtime Guards for Embodied Laboratory Agents

19. CyberNeuro: A Privacy-Preserving Agentic Workbench for Cohort-Scale Neuroimage and Clinical Data Analysis

20. Scaling Scientific Discovery Environments for Turn-Level Agentic RL

21. SeekBrain: An Autonomous Multi-Agent System for Accelerating Neuroscience Discovery

22. MOT-SR: Multi-Objective Tool-Augmented Scientific Equation Discovery with Large Language Models

23. AgentHPOBench: A Benchmark For Evaluating LLM Agents as Sequential Hyperparameter Optimizers

24. Learning to Coordinate Symbolic Tools: LLM Agents for Verified Sum-of-Squares Certificates

25. VFEAgent: A Multimodal Agent Framework for End-to-End Automated Finite Element Analysis

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions