Skip to content

Add SLURM executor - #11

Merged
krokicki merged 16 commits into
mainfrom
slurm-executor
Oct 21, 2026
Merged

krokicki merged 16 commits into
mainfrom
slurm-executor

Conversation

@krokicki

@krokicki krokicki commented Oct 21, 2026 •

Copy link
Copy Markdown
Member

Summary

  • New SlurmExecutor (cluster_api/executors/slurm.py) behind the existing Executor interface: sbatch --parsable submission, sacct status polling, scancel cancellation, squeue+sacct reconnect.
  • Every directive and flag is sourced from submitit, clusterfutures, dask-jobqueue, Nextflow or the SLURM man pages; see docs/superpowers/specs/2026-10-18-slurm-executor-design.md and the design review alongside it.
  • --no-requeue on every job so preempted/node-failed jobs are terminal; --dependency=afterok --kill-on-invalid-dep=yes for depends_on; --export=NONE + in-script exports for inherit_env=False; bash -l shebang for login_shell.
  • Array elements are re-keyed from 123_4 to the base class's 123[4] form so poll() aggregation is unchanged; compressed pending ranges are expanded with SLURM_BITSTR_LEN=0.
  • Signal-terminated jobs report exit_code = -signal, matching the local executor.
  • New CI job slurm-integration runs tests/test_slurm_integration.py on a real single-node SLURM with slurmdbd (koesterlab/setup-slurm-action); CLUSTER_API_REQUIRE_SLURM=1 turns the module's skip into a failure so the job cannot pass green by skipping.
  • Docs: README SLURM notes and profile example, API/Development/CLAUDE updates. Version 0.9.0.

Test plan

  • pixi run check (267 passed)
  • slurm-integration CI job green

🤖 Generated with Claude Code

krokicki and others added 16 commits October 21, 2026 12:49
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…d of sacct --array

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…, array headers

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…t codes

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… SLURM

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… for non-terminal states

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…y state and trace polls

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… pending ranges

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…de poll()

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@krokicki
krokicki merged commit 00eb5d5 into main Oct 21, 2026
2 checks passed
@krokicki
krokicki deleted the slurm-executor branch October 21, 2026 19:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant