nginx-systemd units: fix service start and socket removal, so CI runs test_nginx_dispatch - #225
Merged
ThomasWaldmann merged 2 commits intoSep 19, 2026
Conversation
… test_nginx_dispatch test_nginx_dispatch (and all generic backend tests for the 2 nginx-proxied REST backends) were silently skipped in all Linux CI jobs, because the REST servers behind nginx never worked there: - borgstore@.service had ExecStart=/usr/bin/borgstore-server-rest, but "pip install" as root puts the script into /usr/local/bin on Debian/Ubuntu, so the service failed with status=203/EXEC in a restart loop and the first request hung until the client timed out. Give the executable without a path, systemd then finds it in /usr/local/bin or /usr/bin. - both units declared RuntimeDirectory=borgstore. /run/borgstore/ is shared by the sockets of all instances, but systemd removes a runtime directory when the unit declaring it stops or fails - together with the sockets of all other instances (nginx: "connect() to unix:/run/borgstore/repo1.sock failed (2: No such file or directory)"). Also, starting a service chown'ed the directory recursively to borgstore:borgstore, so the sockets lost SocketGroup=www-data. systemd creates the parent directory of a ListenStream= socket by itself, so just do not declare a RuntimeDirectory. CI: - create and destroy a store in both repos via nginx at the end of the setup step, so a broken setup fails the job instead of skipping the tests. - do not add www-data to the borgstore group: not needed, the sockets now keep their www-data group. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
They run in the system Python and compute the hashes server-side, so test_hash_blake3 failed on the rest1/rest2 backends as soon as these backends were not skipped any more. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
test_nginx_dispatch(and all the generic backend tests for the two nginx-proxied REST backends) were silently skipped in every Linux CI job, e.g. "SKIPPED [1] tests/test_backends.py:984: REST1 and REST2 backends not available" in this run on main. The REST servers behind nginx never worked there, because of two bugs in thecontrib/server/nginx-systemd/unit files.Reproduced by running the CI setup step verbatim in an Ubuntu 24.04 systemd container.
Causes
Wrong executable path.
borgstore@.servicehadExecStart=/usr/bin/borgstore-server-rest, butpip installas root puts the script into/usr/local/binon Debian/Ubuntu. The service failed withstatus=203/EXECin a restart loop and the first request hung until the client's 30 s timeout (that is most of the 33 s test collection takes in the CI log).RuntimeDirectory=borgstorein both units./run/borgstore/is shared by the sockets of all instances, but systemd removes a runtime directory when the unit declaring it stops or fails, together with the sockets of all other instances:So the failing repo1 service also deleted
repo2.sock, and restarting one.socketunit deletes its sibling's socket the same way. This also affects real deployments with more than one repo. Additionally, starting a service chown'ed the directory recursively toborgstore:borgstore, so the sockets lostSocketGroup=www-data- CI worked around that withusermod -aG borgstore www-data.Changes
borgstore@.service: give the executable without a path, systemd then finds it in/usr/local/binor/usr/bin. NoRuntimeDirectory=.borgstore@.socket: noRuntimeDirectory=, systemd creates the parent directory of aListenStream=socket by itself.README.md: updated to match.usermodworkaround is removed, the sockets now keep theirwww-datagroup.blake3extra into the system Python the socket-activated servers run in. They compute hashes server-side, sotest_hash_blake3failed on rest1/rest2 as soon as these backends were not skipped any more (first CI run of this PR).Testing
In a fresh container with the new units and the new step: the step passes with
www-datain no extra group, the sockets stayborgstore:www-dataacross service starts and sibling stops/restarts,test_nginx_dispatchpasses and 46 parametrized tests additionally run on the rest1/rest2 backends.Negative control: main's units with the new step make the step fail (curl exit 28) instead of going green.
CI of this PR:
test_nginx_dispatch PASSEDin all Linux py3xx jobs, 390 passed / 1 skipped (main: 330 passed / 2 skipped).🤖 Generated with Claude Code