Skip to content

update_index records a genuine child dream as a root, and crashes on parent_branch: null #27

Description

Summary

dream-reconcile.py derives the parent column of _dreams/_index.md from a single manifest key:

https://github.com/microsoft/ShadowFrog/blob/main/skills/shadow-frog-dream/dream-reconcile.py#L850

parent = manifest.get('parent_branch', 'main').strip()

This has two distinct failure modes:

  1. Silent lineage corruption. If a manifest records its lineage under any other key, the dream is written as parent: main — a foundation. The chain silently splits at that dream and its depth is understated.
  2. Hard crash. If parent_branch is present but null, .get(key, default) returns None (the default only applies when the key is absent), so .strip() raises AttributeError: 'NoneType' object has no attribute 'strip' and reconciliation aborts.

meditate-repair.py cannot recover from either case, so today there is no supported repair path.

Why this matters

_index.md is the canonical lineage table. dream-lineage.py keys its graph on it, and downstream exports derive parent relationships from it. A wrong parent cell is not cosmetic — it changes the measured shape of the dream forest, and nothing downstream can detect it, because a mis-rooted row is indistinguishable from a real foundation.

Evidence

Observed while reconciling a large compounding-dream campaign. Across 1113 real manifests:

condition count effect on public main
parent_branch present but null 14 AttributeError, reconciliation aborts
parent_branch key absent 26 silently indexed as parent: main

Of the affected dreams, 16 were genuine children indexed as roots. These were not bad manifests — each named a real parent, and git confirmed the ancestry independently:

$ git merge-base --is-ancestor \
    origin/dream/<ns>/20260810-150810Z-s17-b1-topn-tuple \
    origin/dream/<ns>/20260810-152614Z-s18-b1-topn-n-capped
$ echo $?
0

$ git diff --stat <parent> <child> -- ':!.shadow'
 lib/matplotlib/cbook/__init__.py   | 8 +++++---
 lib/matplotlib/tests/test_cbook.py | 32 ++++++++++++++++++++++++++++----

So the child truly descends from the parent and carries product changes on top; the index was wrong, not the manifests.

Repairing those 16 rows changed the measured structure substantially in one repo:

before after
leaf chains 70 55
compounding 70% 76%
depth histogram (d1/d2/d3/d4/d5) 14/13/2/17/24 3/3/0/20/29

Eleven of the depth-1 leaves and ten of the depth-2 leaves were not roots at all; once the 15 mis-rooted children in this repo were re-linked, those shallow chains collapsed into eight additional depth-4/5 chains.

Reproduction

Crash:

>>> {"parent_branch": None}.get('parent_branch', 'main').strip()
AttributeError: 'NoneType' object has no attribute 'strip'

Silent mis-rooting — reconcile a dream whose manifest is:

{
  "dream_id": "20260101-000000Z-child",
  "branch": "dream/proj/20260101-000000Z-child",
  "base_branch": "dream/proj/20251231-000000Z-parent"
}

Expected parent cell: dream/proj/20251231-000000Z-parent
Actual: main

Related: meditate-repair.py cannot repair it

repair_parent is meant to be the recovery path for exactly this, but it reads only parent_branch and the report frontmatter:

https://github.com/microsoft/ShadowFrog/blob/main/skills/shadow-frog-meditate/meditate-repair.py#L209

pb = mdata.get('parent_branch', '').strip()

So /shadow-frog-meditate leaves these rows mis-rooted, and it inherits the same None.strip() crash on a null value.

Suggested fix

Resolve the parent through a fallback chain instead of assuming one key, mirroring the fallback update_index already applies to title a few lines below:

  1. parent_branch
  2. base_branch
  3. parent_dream_id / builds_on, resolved to a branch via
    git for-each-ref refs/remotes/origin/dream/*/<id> (a dream branch ends with its dream id)
  4. report.md frontmatter parent_branch
  5. only then main

Use (manifest.get(k) or '') rather than manifest.get(k, '') throughout so an explicit null is treated as absent instead of crashing. Apply the same chain to meditate-repair.repair_parent so existing corrupted indexes can actually be repaired.

Two properties worth preserving in tests:

  • a dream with no lineage anywhere still records main (no invented lineage)
  • a mis-rooted row is only rewritten when the parent is independently confirmed, so a correct row is never overwritten on a manifest's say-so

Notes

I have a working patch with regression tests for both files (the tests fail without the fix, reproducing the exact | ... | main | ... row, and the full suite passes with it). Happy to open a PR if that is useful.

Environment

  • microsoft/ShadowFrog @ 3419480627 (default branch)
  • Python 3.9 / Linux

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions