Repository navigation
Ship parallel engines on every platform at a fixed -j 8 (macOS static libomp, Windows vcomp140 beside the engine) (#1923) - #1925
Merged
Conversation
macOS: build a static libomp from the pinned LLVM release with a macOS 12 deployment target and link it by path (-Xclang -fopenmp), so an engine loads only /usr/lib/libc++ and libSystem; write the .parallel marker. A new guard fails the build on anything outside /usr/lib and /System, a minos above 12.0, or a .parallel engine without the OpenMP runtime in it. Windows: compile the language engines with /openmp again and copy vcomp140.dll (plus any VC runtime DLL it imports) from the MSVC redist folder beside each engine; write the .exe.parallel marker. The guard now requires every DLL an engine or a shipped DLL imports to be part of Windows or beside it. The engine package's files list carries */*.dll. run-souffle.sh: AXIOM_SOLVE_PARALLEL=0 now holds a packaged parallel engine to -j 1 too, and the PARALLEL SOLVE comments describe every platform. Co-authored-by: axiomcode-bot[bot] <334110751+axiomcode-bot[bot]@users.noreply.github.com>
A packaged engine is the parallel flavor on every published platform, so run-souffle.sh passes a fixed -j 8 whenever the .parallel marker is beside it: no AXIOM_SOLVE_PARALLEL or thread-count variable changes the shipped path, and every user's solve is the configuration the release validated. A local compile keeps its OpenMP probe, AXIOM_SOLVE_PARALLEL and AXIOMCODE_SOLVE_THREADS. Co-authored-by: axiomcode-bot[bot] <334110751+axiomcode-bot[bot]@users.noreply.github.com>
swapnilpaliwal-sd
requested review from
JaredHLZhang,
Whua689 and
suyashpaliwal26
as code owners
October 11, 2026 04:33
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1923 — every platform ships PARALLEL (OpenMP) engines, solving at a fixed
-j 8, with no switch.Before this, only Linux shipped parallel: macOS engines were built without OpenMP, and Windows was made serial in #1914 because
/openmpneedsvcomp140.dll, absent without the Visual C++ Redistributable.What changes (
.github/workflows/build-engines.yml,graph/pipeline/run-souffle.sh,packaging/engine-package.json)libomp.a(built for macOS 12) linked statically;otool -Lshows only/usr/lib/libc++andlibSystem. A guard fails the build if an engine links anything outside/usr/liband/System./openmpagain, withvcomp140.dll(and whatever VC runtime DLL it imports; none for MSVC 14.44) copied beside each engine, so a machine with no VC++ Redistributable runs it. The guard now requires every DLL an engine (or a shipped DLL) imports to be part of Windows or shipped beside it; the engine package carries the DLLs.-j 8; no environment variable changes it. A local source compile keeps its probe andAXIOM_SOLVE_PARALLEL/AXIOMCODE_SOLVE_THREADS.Validation (packaged engines + packed CLI,
npm install -g, clean environment)Solve time, serial →
-j 8(avg of 5):Known trade-off: on a very small input (6 MB of facts) macOS TypeScript is slower at
-j 8than serial; accepted — the gain is on large repositories.darwin-x64 (x64 Node under Rosetta on Apple Silicon, clean environment): e2e 222/227, 0 failed · MCP 40/40 · extras 80/81, 0 failed · crashscan 0 crash markers. Parallel = serial: 0 differing relations at index level (java 39, python 95, ts 58, js 51, c# 44 relations) and in all 5 direct parallel re-solves per language; 0 crashes. Timing under Rosetta, with the case suites running on the same machine, so it is noisy: serial build →
-j 8median: java 2.1 → 2.5 s, python 2.7 → 4.2 s, ts 1.8 → 9.3 s, js 7.8 → 8.7 s, c# 2.6 → 1.1 s. On these small inputs-j 8under translation does not pay; native Intel Macs were not available to measure.Case suites on this branch (macOS arm64, local compile): java (
--oracle --no-torture), typescript (--oracle), python, javascript (--oracle), csharp: all pass. Query cases (tests/run.py): all five languages pass. The first local run had 2 java checks fail inlibrary-auto-discovers-imported-dependenciesbecause no Vineflower decompiler was installed locally (CI installs it). The csharp suite needs the Roslyn oracle built. With both in place everything passes, andengine-package-test.shpasses too. A wrapper around the installed engine confirmed the packaged path passes-j 8whateverAXIOM_SOLVE_PARALLEL(unset/0/1) orAXIOMCODE_SOLVE_THREADSsays.