Add SLURM executor

.github/workflows/ci.yml

@@ -31,3 +31,37 @@ jobs:
 
     - name: Test
       run: pixi run test
+
+  slurm-integration:
+    runs-on: ubuntu-latest
+
+    services:
+      mysql:
+        image: mysql:8.0
+        env:
+          MYSQL_ROOT_PASSWORD: root
+        ports:
+          - "8888:3306"
+        options: --health-cmd="mysqladmin ping" --health-interval=10s --health-timeout=5s --health-retries=3
+
+    steps:
+    - uses: actions/checkout@v5
+
+    - name: Set up SLURM (single node with slurmdbd accounting)
+      uses: koesterlab/setup-slurm-action@v1
+
+    - name: Set up Pixi
+      uses: prefix-dev/setup-pixi@v0.9.0
+      with:
+        pixi-version: v0.70.2
+        cache: true
+
+    - name: Smoke-test scheduler and accounting
+      run: |
+        sinfo
+        sacct -n -X -j 1 -o JobID
+
+    - name: SLURM integration tests
+      env:
+        CLUSTER_API_REQUIRE_SLURM: "1"
+      run: pixi run test-integration

CLAUDE.md

@@ -30,20 +30,22 @@ pixi run check        # lint + test
   - `monitor.py` — async polling loop + callback dispatch
   - `exceptions.py` — `ClusterAPIError`, `CommandTimeoutError`, `CommandFailedError`, `SubmitError`
   - `executors/lsf.py` — LSF executor (bsub/bjobs/bkill)
+  - `executors/slurm.py` — SLURM executor (sbatch/sacct/scancel; squeue for reconnect discovery)
   - `executors/local.py` — local subprocess executor (for testing), including array job simulation
 - `tests/` — pytest tests; all async tests use pytest-asyncio with `asyncio_mode = "auto"`
 
 ## Log File Naming
 
 Default log files include the job ID for uniqueness:
 - **LSF**: `stdout.%J.log` / `stderr.%J.log` (`%J` = job ID); array jobs use `stdout.%J.%I.log` (`%I` = array index)
+- **SLURM**: `stdout.%j.log` / `stderr.%j.log`; array jobs use `stdout.%A_%a.log` (`$SLURM_ARRAY_TASK_ID` per element)
 - **Local**: `stdout.{job_id}.log` / `stderr.{job_id}.log`; array jobs use `stdout.{job_id}.{index}.log` (`$ARRAY_INDEX` env var per element)
 
 Explicit `stdout_path` / `stderr_path` in `ResourceSpec` override these defaults.
 
 ## Testing
 
-All tests mock `Executor._call()` to avoid needing a real scheduler (except `test_local.py` which runs real subprocesses, and `test_integration.py` which requires a live LSF cluster and is skipped by default). Use `unittest.mock.patch` with `AsyncMock` for async method mocking.
+All tests mock `Executor._call()` to avoid needing a real scheduler (except `test_local.py` which runs real subprocesses, and `test_integration.py` / `test_slurm_integration.py` which require a live cluster and are skipped by default). Use `unittest.mock.patch` with `AsyncMock` for async method mocking.
 
 ## Style
 

README.md

@@ -8,7 +8,8 @@ A Python library for submitting and monitoring jobs on HPC clusters. Supports ru
 
 * Local Subprocess
 * IBM Platform LSF
-* We will accept PRs that implement and test additional executors (SLURM, etc.)
+* SLURM (requires slurmdbd accounting; see [SLURM notes](#slurm-notes))
+* We will accept PRs that implement and test additional executors (SGE, PBS, etc.)
 
 ## Features
 
@@ -85,7 +86,7 @@ async def run_array():
     await monitor.stop()
 ```
 
-The array index environment variable depends on the executor: LSF uses `$LSB_JOBINDEX`, while the local executor uses `$ARRAY_INDEX`.
+The array index environment variable depends on the executor: LSF uses `$LSB_JOBINDEX`, SLURM uses `$SLURM_ARRAY_TASK_ID`, and the local executor uses `$ARRAY_INDEX`.
 
 ### Job Dependencies
 
@@ -104,6 +105,7 @@ pyramid = await executor.submit(
 and LSF terminates dependents whose upstream failed — no jobs stuck
 pending forever. When a dependent is cancelled this way,
 `record.metadata["dependency_failed"]` lists the failed upstream job ids.
+On SLURM the same chain uses `sbatch --dependency=afterok:... --kill-on-invalid-dep=yes`.
 
 ### Reconnecting After Restart
 
@@ -188,6 +190,13 @@ profiles:
     script_prologue:
       - "module load java/11"
 
+  slurm_cluster:
+    executor: slurm
+    queue: compute          # SLURM partition
+    cpus: 2
+    memory: "8 GB"
+    walltime: "04:00:00"    # HH:MM:SS — "04:00" would mean four minutes on SLURM
+
   local_dev:
     executor: local
     poll_interval: 2
@@ -197,11 +206,11 @@ profiles:
 
 | Option | Default | Description |
 |---|---|---|
-| `executor` | `"local"` | Backend: `lsf` or `local` |
+| `executor` | `"local"` | Backend: `lsf`, `slurm` or `local` |
 | `cpus` | `None` | Default CPU count |
 | `gpus` | `None` | Default GPU count |
 | `memory` | `None` | Default memory (e.g. `"8 GB"`) |
-| `walltime` | `None` | Default wall time (e.g. `"04:00"`) |
+| `walltime` | `None` | Default wall time (LSF `"04:00"` = 4 h; SLURM `"04:00:00"` = 4 h) |
 | `queue` | `None` | Default queue/partition |
 | `poll_interval` | `10.0` | Seconds between status polls |
 | `job_name_prefix` | `None` | Optional prefix prepended to job names. When set, polling filters by `{prefix}-*` and `reconnect()` is available; when unset, the user controls the full job name and polling queries all jobs |
@@ -217,6 +226,28 @@ profiles:
 | `zombie_timeout_minutes` | `30.0` | Mark jobs as failed if unseen for this long |
 | `completed_retention_minutes` | `10.0` | Keep finished jobs in memory for this long |
 
+### SLURM notes
+
+- **Accounting required.** Status comes from `sacct`, so the cluster must run
+  slurmdbd. History is kept subject to the site's `PurgeJobAfter` setting.
+- **No requeue.** Every job is submitted with `--no-requeue` so preempted or
+  node-failed jobs are reported as terminal. Sites that set
+  `PrologFlags=ForceRequeueOnFail` or `requeue_on_resume_failure` can still
+  requeue a job behind our back; the rerun is not tracked.
+- **Exit codes.** `exit_code` is the script's return code, or `-signal` when
+  a signal terminated the job (e.g. `-9`), as for the local executor.
+- **`max_mem`** is not reported (sacct only has it on step rows).
+- **`cancel_by_name`** matches an exact job name; `scancel` has no globbing.
+- **`inherit_env=False`** uses `--export=NONE`, which SLURM documents as
+  still loading the user's login environment on the node; `env` values are
+  exported inside the script and `SLURM_EXPORT_ENV=ALL` is set so `srun`
+  steps inside your command see them.
+- **`login_shell=True`** runs the script with `#!/bin/bash -l`; shebangs of
+  the form `#!/usr/bin/env bash` are left unchanged with a warning.
+- **GPUs** are requested with `--gres=gpu:N`; use `extra_directives` for
+  `--gpus-per-node`, `--mem-per-cpu`, `--account` and similar.
+- **Multi-cluster** (`-M`, `SLURM_CLUSTERS`) is unsupported.
+
 ## API Reference
 
 See [docs/API.md](docs/API.md) for the full API reference and error handling guide.

cluster_api/__init__.py

@@ -16,13 +16,15 @@
 from .executors import get_executor_class
 from .executors.local import LocalExecutor
 from .executors.lsf import LSFExecutor
+from .executors.slurm import SlurmExecutor
 from .monitor import JobMonitor
 
 __all__ = [
     "create_executor",
     "ArrayElement",
     "Executor",
     "LSFExecutor",
+    "SlurmExecutor",
     "LocalExecutor",
     "JobRecord",
     "JobStatus",

cluster_api/executors/__init__.py

@@ -12,13 +12,15 @@
 
 def _ensure_builtins() -> None:
     """Lazily register built-in executors."""
-    if "lsf" in _REGISTRY and "local" in _REGISTRY:
+    if {"lsf", "local", "slurm"} <= _REGISTRY.keys():
         return
     from .lsf import LSFExecutor
     from .local import LocalExecutor
+    from .slurm import SlurmExecutor
 
     _REGISTRY.setdefault("lsf", LSFExecutor)
     _REGISTRY.setdefault("local", LocalExecutor)
+    _REGISTRY.setdefault("slurm", SlurmExecutor)
 
 
 def get_executor_class(name: str) -> type[Executor]: