Skip to content

graphify update deletes every hyperedge whose source_file is in the corpus, even on a no-op run #1755

Description

@AstroMined

Version: graphifyy 0.9.11. Also present on v8 HEAD (graphify/watch.py is byte-identical to the installed 0.9.11 copy, 1238 lines).

Summary

graphify update <path> unconditionally drops every hyperedge whose source_file points at a file in the current corpus. It happens on every run, including a run where nothing changed on disk. The AST pass never emits hyperedges, so nothing replaces them: the graph's hyperedge set is permanently lost on the first graphify update after a full build.

This is the watch._rebuild_code path, not the build_merge path fixed in #1574 (4f40967). The closing note on #1574 said the fix "mirrors what watch.py already did", but watch.py has the inverse problem.

Root cause

In _reconcile_existing_graph (graphify/watch.py):

  • graphify update sets extract_targets = code_files (watch.py:858), i.e. the entire corpus.
  • rebuilt_source_identities is built from every extract_targets entry (watch.py:406-408).
  • edge_evicted_source_identities = node_evicted_source_identities | rebuilt_source_identities (watch.py:412-414).
  • A hyperedge is skipped when source_paths.is_evicted(edge, edge_evicted_source_identities) (watch.py:475-478), and is_evicted is just self.identity(item.get("source_file")) in identities (watch.py:350-351).

So on graphify update, every corpus file is "rebuilt", therefore every hyperedge anchored to a corpus file is evicted. Because the AST result carries no hyperedges, preserved_hyperedges ends up empty and merged["hyperedges"] == []. build() then only assigns G.graph["hyperedges"] when the list is truthy (build.py:654-662), so to_json writes [].

Markdown is AST-extractable (_get_extractor(Path("doc.md")) is not None), so doc-sourced hyperedges, which are exactly the ones semantic extraction produces, are the ones destroyed.

Minimal repro (0.9.11)

mkdir -p /tmp/hetest && cd /tmp/hetest
printf 'def foo():\n    return 1\n\ndef bar():\n    return foo()\n' > a.py
printf '# Design\n\nThe `foo` helper backs `bar`.\n' > doc.md

graphify update .    # baseline: 5 nodes, 0 hyperedges; node ids include `doc` and `doc_design`

python3 - <<'PY'
import json
p = 'graphify-out/graph.json'
d = json.load(open(p))
he = {"id": "g", "label": "G", "nodes": ["doc", "doc_design"], "relation": "implements",
      "confidence": "EXTRACTED", "confidence_score": 1.0, "source_file": "doc.md"}
d["hyperedges"] = [he]
d.setdefault("graph", {})["hyperedges"] = [he]
json.dump(d, open(p, 'w'))
PY

graphify update .    # nothing changed on disk since the injection
python3 -c "import json; print(len(json.load(open('graphify-out/graph.json')).get('hyperedges', [])))"
# prints 0   (expected 1)

Varying only source_file on the injected hyperedge isolates the trigger:

source_file in corpus? hyperedges after graphify update
omitted (cross-file) n/a 1 (preserved)
absent.md (not on disk) no 1 (preserved)
doc.md yes 0 (dropped)

The member-resolution guard at watch.py:479 is not what fires here: in the doc.md case both members (doc, doc_design) are present in the merged node set.

Real-world impact

On a 7935-node repository graph, one graphify update . took hyperedges from 3 to 0. All three were doc-sourced. Two of the three had every member node still present in the merged graph and had completely unmodified source files. Hyperedges are the highest-signal part of the semantic layer, so query and explain quality degrades on the first incremental update after any full build.

Suggested fix

Evict a hyperedge only when its source was genuinely deleted, or when the re-extraction actually produced replacement hyperedges for that source. Since the AST pass never produces hyperedges, graphify update should preserve existing hyperedges whose members survive. The any(member not in all_ids ...) check already at watch.py:479 correctly handles the dangling-member case on its own.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions