Summary
dream-reconcile.py derives the parent column of _dreams/_index.md from a single manifest key:
https://github.com/microsoft/ShadowFrog/blob/main/skills/shadow-frog-dream/dream-reconcile.py#L850
parent = manifest.get('parent_branch', 'main').strip()
This has two distinct failure modes:
- Silent lineage corruption. If a manifest records its lineage under any other key, the dream is written as
parent: main — a foundation. The chain silently splits at that dream and its depth is understated.
- Hard crash. If
parent_branch is present but null, .get(key, default) returns None (the default only applies when the key is absent), so .strip() raises AttributeError: 'NoneType' object has no attribute 'strip' and reconciliation aborts.
meditate-repair.py cannot recover from either case, so today there is no supported repair path.
Why this matters
_index.md is the canonical lineage table. dream-lineage.py keys its graph on it, and downstream exports derive parent relationships from it. A wrong parent cell is not cosmetic — it changes the measured shape of the dream forest, and nothing downstream can detect it, because a mis-rooted row is indistinguishable from a real foundation.
Evidence
Observed while reconciling a large compounding-dream campaign. Across 1113 real manifests:
| condition |
count |
effect on public main |
parent_branch present but null |
14 |
AttributeError, reconciliation aborts |
parent_branch key absent |
26 |
silently indexed as parent: main |
Of the affected dreams, 16 were genuine children indexed as roots. These were not bad manifests — each named a real parent, and git confirmed the ancestry independently:
$ git merge-base --is-ancestor \
origin/dream/<ns>/20260810-150810Z-s17-b1-topn-tuple \
origin/dream/<ns>/20260810-152614Z-s18-b1-topn-n-capped
$ echo $?
0
$ git diff --stat <parent> <child> -- ':!.shadow'
lib/matplotlib/cbook/__init__.py | 8 +++++---
lib/matplotlib/tests/test_cbook.py | 32 ++++++++++++++++++++++++++++----
So the child truly descends from the parent and carries product changes on top; the index was wrong, not the manifests.
Repairing those 16 rows changed the measured structure substantially in one repo:
|
before |
after |
| leaf chains |
70 |
55 |
| compounding |
70% |
76% |
| depth histogram (d1/d2/d3/d4/d5) |
14/13/2/17/24 |
3/3/0/20/29 |
Eleven of the depth-1 leaves and ten of the depth-2 leaves were not roots at all; once the 15 mis-rooted children in this repo were re-linked, those shallow chains collapsed into eight additional depth-4/5 chains.
Reproduction
Crash:
>>> {"parent_branch": None}.get('parent_branch', 'main').strip()
AttributeError: 'NoneType' object has no attribute 'strip'
Silent mis-rooting — reconcile a dream whose manifest is:
{
"dream_id": "20260101-000000Z-child",
"branch": "dream/proj/20260101-000000Z-child",
"base_branch": "dream/proj/20251231-000000Z-parent"
}
Expected parent cell: dream/proj/20251231-000000Z-parent
Actual: main
Related: meditate-repair.py cannot repair it
repair_parent is meant to be the recovery path for exactly this, but it reads only parent_branch and the report frontmatter:
https://github.com/microsoft/ShadowFrog/blob/main/skills/shadow-frog-meditate/meditate-repair.py#L209
pb = mdata.get('parent_branch', '').strip()
So /shadow-frog-meditate leaves these rows mis-rooted, and it inherits the same None.strip() crash on a null value.
Suggested fix
Resolve the parent through a fallback chain instead of assuming one key, mirroring the fallback update_index already applies to title a few lines below:
parent_branch
base_branch
parent_dream_id / builds_on, resolved to a branch via
git for-each-ref refs/remotes/origin/dream/*/<id> (a dream branch ends with its dream id)
report.md frontmatter parent_branch
- only then
main
Use (manifest.get(k) or '') rather than manifest.get(k, '') throughout so an explicit null is treated as absent instead of crashing. Apply the same chain to meditate-repair.repair_parent so existing corrupted indexes can actually be repaired.
Two properties worth preserving in tests:
- a dream with no lineage anywhere still records
main (no invented lineage)
- a mis-rooted row is only rewritten when the parent is independently confirmed, so a correct row is never overwritten on a manifest's say-so
Notes
I have a working patch with regression tests for both files (the tests fail without the fix, reproducing the exact | ... | main | ... row, and the full suite passes with it). Happy to open a PR if that is useful.
Environment
microsoft/ShadowFrog @ 3419480627 (default branch)
- Python 3.9 / Linux
Summary
dream-reconcile.pyderives theparentcolumn of_dreams/_index.mdfrom a single manifest key:https://github.com/microsoft/ShadowFrog/blob/main/skills/shadow-frog-dream/dream-reconcile.py#L850
This has two distinct failure modes:
parent: main— a foundation. The chain silently splits at that dream and its depth is understated.parent_branchis present butnull,.get(key, default)returnsNone(the default only applies when the key is absent), so.strip()raisesAttributeError: 'NoneType' object has no attribute 'strip'and reconciliation aborts.meditate-repair.pycannot recover from either case, so today there is no supported repair path.Why this matters
_index.mdis the canonical lineage table.dream-lineage.pykeys its graph on it, and downstream exports derive parent relationships from it. A wrongparentcell is not cosmetic — it changes the measured shape of the dream forest, and nothing downstream can detect it, because a mis-rooted row is indistinguishable from a real foundation.Evidence
Observed while reconciling a large compounding-dream campaign. Across 1113 real manifests:
mainparent_branchpresent butnullAttributeError, reconciliation abortsparent_branchkey absentparent: mainOf the affected dreams, 16 were genuine children indexed as roots. These were not bad manifests — each named a real parent, and
gitconfirmed the ancestry independently:So the child truly descends from the parent and carries product changes on top; the index was wrong, not the manifests.
Repairing those 16 rows changed the measured structure substantially in one repo:
14/13/2/17/243/3/0/20/29Eleven of the depth-1 leaves and ten of the depth-2 leaves were not roots at all; once the 15 mis-rooted children in this repo were re-linked, those shallow chains collapsed into eight additional depth-4/5 chains.
Reproduction
Crash:
Silent mis-rooting — reconcile a dream whose manifest is:
{ "dream_id": "20260101-000000Z-child", "branch": "dream/proj/20260101-000000Z-child", "base_branch": "dream/proj/20251231-000000Z-parent" }Expected
parentcell:dream/proj/20251231-000000Z-parentActual:
mainRelated:
meditate-repair.pycannot repair itrepair_parentis meant to be the recovery path for exactly this, but it reads onlyparent_branchand the report frontmatter:https://github.com/microsoft/ShadowFrog/blob/main/skills/shadow-frog-meditate/meditate-repair.py#L209
So
/shadow-frog-meditateleaves these rows mis-rooted, and it inherits the sameNone.strip()crash on a null value.Suggested fix
Resolve the parent through a fallback chain instead of assuming one key, mirroring the fallback
update_indexalready applies totitlea few lines below:parent_branchbase_branchparent_dream_id/builds_on, resolved to a branch viagit for-each-ref refs/remotes/origin/dream/*/<id>(a dream branch ends with its dream id)report.mdfrontmatterparent_branchmainUse
(manifest.get(k) or '')rather thanmanifest.get(k, '')throughout so an explicitnullis treated as absent instead of crashing. Apply the same chain tomeditate-repair.repair_parentso existing corrupted indexes can actually be repaired.Two properties worth preserving in tests:
main(no invented lineage)Notes
I have a working patch with regression tests for both files (the tests fail without the fix, reproducing the exact
| ... | main | ...row, and the full suite passes with it). Happy to open a PR if that is useful.Environment
microsoft/ShadowFrog@3419480627(default branch)