engine: generate the Souffle program once and hash that file for the engine id (#1593)

[swapnilpaliwal-sd]

Fixes #1593

What was wrong

graph/pipeline/run-souffle.sh generated the Soufflé program twice: once into the file souffle compiles, and once more inside engine_id() only to hash it. Both renderings went through printf loops piped into sort, and no write inside those pipelines was checked. Under load, bash 3.2 (macOS /bin/bash) does not restart a pipe write interrupted by a signal, so a line was sometimes lost with exit 0 (printf: write error: Interrupted system call). Lost from the hashed copy, it gave a different engine id, the cached engine was missed, and a warm rebuild became a full C++ compile. Lost from the compiled copy, a program missing an .input would have been compiled and cached under the correct id.

The change

• The program text is assembled once, in memory, in the running shell: string appends, globs, and read straight from the map files, sorted in the shell. No subprocess and no pipe carries a line of it.
• It is written with one checked write to a temp file, read back and compared byte for byte, and checked against an independent derivation with awk from the maps and the export manifest (every .input and .output line exactly once, nothing missing, nothing extra). Only then is it renamed onto the destination. A failed attempt is retried; three failures abort the run with an error.
• The engine id is the sha256 of that exact file plus the files it includes, in include order, with the pinned Soufflé version first: the same definition as before, so every id is unchanged. The hash input is written with checked writes, its size is checked against the source files, the digest is read from a file argument, and the result must be 64 hex digits.
• Compile and cache insertion: the binary is built under a temp name, and before the rename it must be a non-empty executable and the program must still hash to the id it was built for. A partial or wrong engine cannot land under a good id.
• New test graph/test/tools/engine-id-lost-line-test.sh (wired into the Java suite beside engine-id): a sort or cat on PATH that silently drops a line must leave the id and the emitted program unchanged, or fail the run. It has a control that the shim was reached.
• The id's definition (path-independent, rule-sensitive, hashed from the compiled file) is written down in .github/RELEASING.md and in the script.

Measurement

macOS, /bin/bash 3.2.57, Soufflé 2.5.

	before	after
issue repro script	exit 1: 33a53d0f… intact, c9a07c91… with a lost line, exit 0	exit 0: same id
engine-id-lost-line-test.sh	6 failures	11 checks pass
300 --print-engine-id runs (python, java, csharp) under yes x8, load average 150 to 220	1 wrong id with exit 0 (java, after the printf write error)	0 wrong, 0 refused
50 warm rebuilds of the 7-file python, java and csharp fixtures, one-line edit before each, under yes x8, load average about 175	not run	0 recompiles, 0 failures; mean 12.4 s python, 11.3 s java, 11.3 s csharp; max 30 s

• Identical rules give identical ids: all five languages print the same id before and after the change, and the emitted java and python programs are byte-identical to the old ones. A rule change still moves the id: engine-id-test.sh mutates a rule file, a map, the manifest and the pinned version, and each moves it.
• engine-id, engine-id-locale and engine-package pass.
• Suites: being run locally for all five languages; this line is updated with the results.