feat(extract): extract systemd units so a repo's scheduled-job topology reaches the graph (#2848) - #3088
feat(extract): extract systemd units so a repo's scheduled-job topology reaches the graph (#2848)#3088abhay-codes07 wants to merge 2 commits into
Conversation
…gy reaches the graph (Graphify-Labs#2848) `.service`/`.timer` were in no extension set, so a repo that keeps its units under version control had its whole OS-level job topology missing from the graph — silently: "what scheduled jobs run" was answered with confidence from the application's in-process scheduler alone. Units are INI, so a regex pass (modelled on extractors/sln.py) covers .service .timer .socket .target .path .mount .slice. Every unit is a file node; edges land on nodes the AST pass already creates: activates timer/socket/path -> unit [Timer] Unit= / [Socket] Service= / [Path] Unit=, else the same-stem .service runs service -> script Exec*= after stripping the -@:+! prefixes, /usr/bin/env and the interpreter; `python -m` is not a file documented_by unit -> doc Documentation=file:// after/before/wants/requires/binds_to/part_of/conflicts/wanted_by/required_by unit -> unit [Unit] and [Install] keys Unit -> unit edges only target a unit file beside this one, so the host's network-online.target / timers.target are never fabricated into a phantom hub. A template instance (backup@nightly.service) resolves to its template (backup@.service). Exec/Documentation values are deployment paths (/opt/app/bin/run.py), so resolution tries the literal path and then walks up from the unit's directory looking for the same tail (bin/run.py, then run.py) — the usual units/ beside bin/ layout; deployment paths are parsed as POSIX so they are absolute on every host graphify runs on. `x.service` and `x.timer` share the extension-less file-node id, so the corpus-level collision remap — which resolves an edge target by the edge's own file — turned the timer's `activates` edge into a self-loop. The edges now carry the transient `target_file` stamp import edges already use, and the disambiguator's whitelist of stamped relations is named and widened.
![graphify-labs[bot]](https://avatars.githubusercontent.com/in/4356270?s=60&v=4){kind=small}
There was a problem hiding this comment.
Graphify reviewed this change.
Worth a look — the grounded gate found no coupling regressions or blocking issues, but 3 advisory finding(s) below merit a look before merge.
Formal verification. No changes could be formally verified in this run.
Graphify review — findings
Adds a systemd unit extractor (extract_systemd) that parses .service .timer .socket .target .path .mount .slice INI files without a grammar, emitting activates (timer/socket/path → unit, falling back to the same-stem .service), runs (service → the script named by ExecStart= after stripping systemd prefixes and interpreters), documented_by (from Documentation=file://), and the [Unit]/[Install] ordering/dependency keys as unit→unit edges — the latter only when the target unit is a file beside this one, so host units like network.target are never fabricated. Wires the extension into CODE_EXTENSIONS and the _DISPATCH table, resolving script/doc targets by literal path then by walking up for a matching path tail, and skipping any that don't resolve. Extends _TARGET_FILE_RELATIONS so the new edges honor the transient target_file stamp during id disambiguation, preventing a timer's activates edge from collapsing into a self-loop against its own extension-less file id.
Worth a look
- Valid systemd file: documentation URIs are ignored —
graphify/extractors/systemd.py· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- Exec parser splits quoted script paths on whitespace —
graphify/extractors/systemd.py:191· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- Exec parser treats shell -c command text as a script path —
graphify/extractors/systemd.py:211· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 2445 functions depend on the 524 functions this change touches.
Health — this change adds coupling hotspots:
- new:
extract()— 500 callers, 42 callees - new:
_rebuild_code()— 98 callers, 50 callees - new:
detect()— 108 callers, 15 callees - new:
save_manifest()— 40 callers, 11 callees - new:
_extract_generic()— 18 callers, 24 callees - new:
extract_files_direct()— 17 callers, 20 callees - new:
extract_xaml()— 19 callers, 17 callees - new:
extract_corpus_parallel()— 26 callers, 11 callees - …and 58 more — each is listed as a finding
Verification — 2445 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 2218 function(s) in the blast radius were not formally verified this run
Formal verification
Could not verify: Could not verify \_disambiguate\_colliding\_node\_ids.
The verifier did not have enough to check \_disambiguate\_colliding\_node\_ids, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set
· 1 grounded finding(s) anchored inline below; 65 more finding(s) on lines outside this diff (see the check run).
| return None | ||
|
|
||
|
|
||
| def extract_systemd(path: Path) -> dict: |
There was a problem hiding this comment.
extract_systemd()
fans out to 6 callees (efferent coupling); 11 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
…can root On a Linux host /usr/bin/mkdir exists, so ExecStartPre=-/usr/bin/mkdir resolved to the host binary and the tail walk could climb to / and match /usr/bin/python3.12. An absolute Exec/Documentation path is a HOST path: it may only resolve inside the corpus, and the ancestor walk now stops at the scan root (or a few levels up when called outside a scan).
There was a problem hiding this comment.
Graphify reviewed this change.
Worth a look — the grounded gate found no coupling regressions or blocking issues, but 5 advisory finding(s) below merit a look before merge.
Formal verification. No changes could be formally verified in this run.
Graphify review — findings
Adds a systemd unit extractor that treats .service/.timer/.socket/.target/.path/.mount/.slice files as INI (no grammar), making each unit a file node and emitting activates (timer/socket/path→unit), runs (service→the script from ExecStart= after stripping systemd prefixes and env/interpreter wrappers), documented_by (from Documentation=file://), and the [Unit]/[Install] ordering/dependency edges (after, wants, requires, wanted_by, etc.). Unit→unit edges are only created when the target unit is a file in the same directory, so host units like network.target aren't fabricated into phantom hubs, and template instances resolve to their @-template file; script/doc targets try the literal path then walk up for a matching tail (bin/run.py), and unresolvable targets are skipped rather than minted. Extends _TARGET_FILE_RELATIONS in resolution.py so these edges carry the transient target_file stamp during id disambiguation — without it a timer sharing its .service's extension-less file id would resolve its activates edge into a self-loop.
Worth a look
- Incomplete test function leaves file syntactically invalid —
tests/test_systemd_units.py:267· Escalate · high- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- Accept=yes socket units imply @.service, not .service —
graphify/extractors/systemd.py· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- _resolve_path_target absolute-path host escape via bounded bases still reaches ancestors above unit_dir —
graphify/extractors/systemd.py:200· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- Exec command parsing breaks quoted script paths with spaces —
graphify/extractors/systemd.py:234· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- Combined '-' and '@' Exec prefixes do not skip argv0 —
graphify/extractors/systemd.py:239· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 2450 functions depend on the 529 functions this change touches.
Health — this change adds coupling hotspots:
- new:
extract()— 500 callers, 42 callees - new:
_rebuild_code()— 98 callers, 50 callees - new:
detect()— 108 callers, 15 callees - new:
save_manifest()— 40 callers, 11 callees - new:
_extract_generic()— 18 callers, 24 callees - new:
extract_files_direct()— 17 callers, 20 callees - new:
extract_xaml()— 19 callers, 17 callees - new:
extract_corpus_parallel()— 26 callers, 11 callees - …and 59 more — each is listed as a finding
Verification — 2450 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 2223 function(s) in the blast radius were not formally verified this run
Formal verification
Could not verify: Could not verify \_disambiguate\_colliding\_node\_ids.
The verifier did not have enough to check \_disambiguate\_colliding\_node\_ids, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set
· 2 grounded finding(s) anchored inline below; 65 more finding(s) on lines outside this diff (see the check run).
| return None | ||
|
|
||
|
|
||
| def _resolve_path_target(unit_dir: Path, raw: str) -> Path | None: |
There was a problem hiding this comment.
_resolve_path_target()
6 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| return None | ||
|
|
||
|
|
||
| def extract_systemd(path: Path) -> dict: |
There was a problem hiding this comment.
extract_systemd()
fans out to 6 callees (efferent coupling); 11 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
Closes #2848.
The problem
.serviceand.timerwere in no extension set, soclassify_file()returnedNoneand systemd units were never collected. For a repo that keeps its units under version control, the whole OS-level scheduled-job topology was absent from the graph — and silently so:graphify query "what scheduled jobs run"answered with 59 nodes from the application's in-process scheduler and zero timers, looking complete.The change
A deterministic INI pass,
extractors/systemd.py(modelled onextractors/sln.py, no grammar), covering.service .timer .socket .target .path .mount .slice. Every unit is a file node; edges land on nodes the AST pass already creates:activates[Timer] Unit=/[Socket] Service=/[Path] Unit=, else the same-stem.serviceby systemd's own conventionrunsExecStart=and the otherExec*=keys, after stripping the-@:+!prefixes,/usr/bin/env(with itsVAR=and-Sarguments) and the interpreter (python3 -u,bash,node,uv run,npx tsx, …);python -m pkgis a module, not a filedocumented_byDocumentation=file://…afterbeforewantsrequiresbinds_topart_ofconflictswanted_byrequired_by[Unit]and[Install]keysTwo resolution rules keep it honest:
After=network-online.targetandWantedBy=timers.targetname the host's units; manufacturing a node for each would put a phantom hub in every repo. A template instance (backup@nightly.service) resolves to its template (backup@.service)./opt/app/bin/run.py) that rarely exist at that path in the repo. Resolution tries the literal path, then walks up from the unit's directory looking for the same tail (bin/run.py, thenrun.py) — the usualunits/besidebin/layout. A target that resolves is minted the way every other extractor mints a file reference (_make_id(str(resolved))), whichextract()rewires onto the real file node; one that does not is skipped, not fabricated. Deployment paths are parsed as POSIX so they are absolute on every host graphify runs on.One interaction worth calling out
x.serviceandx.timerare the most common pair a repo has, and they share the extension-less file-node id. The corpus-level collision remap (_disambiguate_colliding_node_ids) resolves an edge's target by the edge's own file, so the timer'sactivatesedge came out as a self-loop on the timer. Import edges already solve this with a transienttarget_filestamp naming the file the target id was minted from; systemd edges now carry it too, and the disambiguator's whitelist of stamped relations is named (_TARGET_FILE_RELATIONS) and widened. The stamp is popped before anything ships, as before.What it looks like
with
After=network-online.target,WantedBy=timers.targetand a nonexistentbackup.servicecorrectly producing nothing.Tests
tests/test_systemd_units.py— 40 tests: every unit type classified and dispatched; the per-file extractor (script, doc, implied and explicit activation, template instances, ordering keys, host units never fabricated,-mmodules, continuation lines, comments, unreadable files); 14Exec=parsing cases; deployment-path resolution including the POSIX-on-Windows case; and corpus-level assertions that the edges land on the real script/doc nodes, that the timer/service pair links correctly with no self-loop, that nothing dangles, and that the stamp does not leak.With the wiring reverted and only the module kept, 10 of them fail.
test_extractors_registry,test_detect,test_dotnetand the disambiguation suites are unchanged (304 passed); the full suite matches thev8baseline.README gains a row in the file-types table.