Skip to content

fix(learnings): publish what an older import --from-mr left in the checkout (#823) - #838

Open
SaulMoro wants to merge 6 commits into
Tencent:mainfrom
SaulMoro:fix-823-data-layout-followups
Open

SaulMoro wants to merge 6 commits into
Tencent:mainfrom
SaulMoro:fix-823-data-layout-followups

Conversation

@SaulMoro

@SaulMoro SaulMoro commented Sep 25, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

The learning an older import --from-mr (0.25.0 to 0.26.0-beta.3) left uncommitted in the learnings checkout now reaches the team, and it no longer keeps a single-repo project's old .teamai/learnings-wt from being removed. Maintenance publishes only what it changed. pull --dry-run no longer publishes the queue, and the leftover's duplicate check sees what a teammate published since the old checkout last synced. This also includes #814's follow-ups and items 15, 17, 20 and 21.

 publishQueuedLearnings                                   # learnings-publish.ts
+  take the sync lock (unless pull holds it)
+  queueImportRemnants                                    # item 7, skipped on --dry-run
+    checkout = the one this repo registers for teamai-learnings (shared, or old .teamai/learnings-wt)
+    untracked learnings/<YYYY-MM-DD>-<title>.md with source_mr (title part may be empty)
+      on origin's branch or in the queue already (source_mr or content) -> delete, name the one that has it
+        branch = the tree of origin/teamai-learnings, fetched first, never the checkout's files  # item 21; review 5: a stale tracked copy lost the learning
+        fetch fails -> leave every remnant for a later run   # review 3; a stale ref queued a duplicate
+          ls-remote answers with no such branch -> origin adds nothing   # review 4; it stayed forever
+    a remnant it cannot read or remove -> skip it, not the rest   # review 4
+      else -> queue in the active namespace, contribute's name; then delete the original
+  publish recorded maintenance changes, queue empty or not   # review 3
   list queue -> publish
+  --dry-run: list queue -> "[dry-run] Would publish N queued learning(s)"   # item 20; it published before

 recall maintenance / promote -> publishLearningsMaintenance
-  git add learnings                                      # swept any uncommitted file in
+  git add <files the maintenance step wrote or removed, minus removed files git never tracked>
+  that ls-files lookup fails (HTTP cache, no git) -> failed publish, as commitAndPush gives  # review 2
+  record the paths (checkout's git dir) -> publish every record -> drop them only on published / already-present  # review 3
+    record written atomically; an unreadable one -> removed with a warning, the rest published  # review 4
+      read fails (permission, I/O) -> kept for the next publish, with a warning  # review 5; it was removed
 commitAndPushAt
-  git add <files>                                        # `[` or `*` in a name is a pattern
+  git --literal-pathspecs add -- <files>                 # review; also the ls-files above
-  staged = status.staged                                 # a rename (prune --archive) is not in it
+  staged = status.staged + status.renamed

 readableReportsWorktree, refresh failed                  # item 15
-  ensure()                                               # no lock
+  refresh() again, under the lock; throw its cause if it fails again

 init, about to clone another owner's team repo           # item 17
+  settle the old install now (queue aside, indexes dropped); config.yaml -> config.yaml.previous
   clone ... prompts ... save new config
+  rerun, no config.yaml: carry agents, toolRoots, inheritUserScope from the last config.yaml.previous  # review
+  no --agent: keep the carried enabledAgents / disabledAgents  # review 2; they were dropped

 contribute / import --from-mr
-  drainCheckoutQueue                                     # the preAction migration already does it
 retire-only
-  "Finished an interrupted migration: retired X to Y"
+  "Retired X to Y: this project's data already lives in <partition>"
 .gitignore (single-repo, its self-heal, project scope)
+  config.yaml.*.tmp                                      # an interrupted save's temp (#831)

Type of Change

  • Bug fix (non-breaking change that fixes an issue)

Causes:

  • Item 7. import --from-mr in 0.25.0 to 0.26.0-beta.3 (feat(learnings): write learnings to the teamai-learnings branch #616 until fix(import): publish the learning import --from-mr extracts (#823) #825) wrote the learning into the checkout and never committed it. Git will not remove a checkout with untracked files without --force, so in single-repo mode every pull and contribute stopped on CheckoutRefusedError.
  • Maintenance. publishLearningsMaintenance staged all of learnings/. Confidence write-back of an inherited learning, prune --archive and the promote mark create new files, so git add -u alone was not enough.
  • prune --archive (found here, also broken on main): the archive move stages as a rename, which simple-git does not list in staged. With nothing else staged it counted as "nothing to commit", and the archive never left the machine.
  • Item 21 (in this PR's item 7 code): the duplicate check read the checkout's tracked files only. In single-repo mode that checkout is often the old .teamai/learnings-wt, which nothing syncs any more, so a teammate's later import of the same MR was missed and the leftover went out as a duplicate.
  • Item 20. publishQueuedLearnings honoured dryRun only for the leftover adoption, so pull --dry-run published the whole queue (confirmed with the real CLI below).
  • Item 15. A reports reader whose refresh failed called ensure without the lock, so it could create the checkout while a writer that had just taken the lock created it too.
  • Item 17. init removed the old clone and cloned the new team repo long before it saved the new config, so an init that stopped in between left team A's config.yaml beside team B's clone.

Test Plan

  • npx tsc --noEmit passes
  • npx vitest run passes: 328 files, 5067 passed, 1 skipped
  • Added/updated tests for the change

New tests, red on origin/main (7c834ce) unless marked guard:

unit learnings-import-remnants   remnant, empty queue -> published, checkout clean      red
                                 same source_mr on branch (namespace) -> deleted        red
                                 same content in queue -> published once                red
                                 tracked edit / subdir / no date / no source_mr -> kept guard
                                 sync lock held elsewhere -> untouched                  guard
                                 dry run -> untouched                                   guard
                                 `<date>-.md` (a title with no Latin character) -> published  red
                                 teammate published same source_mr, checkout not fetched -> deleted  red (item 21)
unit dry run                     queued learning, dryRun -> nothing published, still queued  red (item 20)
unit maintenance publish         tracked rewrite + removed never-tracked file -> published     guard
                                 removed `a[1].md` + written `b*.md`, strays a1.md/bee.md -> only b*.md  red on b018180f (review)
                                 HTTP install, prune removed a file -> failed result, no throw  red on c23302d8 (review 2)
                                 lock held -> busy; next publish, empty queue -> published      red on 8523c0d3 (review 3)
                                 prune, origin offline -> failed; next publish, empty queue -> pushed  red on 8523c0d3 (review 3)
unit learnings-import-remnants   teammate published same source_mr, origin offline -> remnant kept; online -> removed, not published  red on 8523c0d3 (review 3)
                                 origin has no teamai-learnings -> remnant queued and published  red on 331d9296 (review 4)
                                 a dangling-link remnant beside a real one -> the real one published  red on 331d9296 (review 4)
unit maintenance publish         busy record + truncated record -> busy one published, truncated removed and named  red on 331d9296 (review 4)
unit learnings-import-remnants   origin deleted the tracked learning with the same source_mr -> remnant queued and published  red on 4defc3ee (review 5)
unit maintenance publish         busy record + a record whose read fails (EACCES) -> busy one published, the other kept and named  red on 4defc3ee (review 5)
e2e  learnings-remnants-823      git pull -> learnings/alpha/quokka-...md, checkout clean red
                                 confidence write-back -> stray not published           red
                                 prune --archive -> _archive/ published, stray not      red
                                 pull --dry-run with one queued -> "Would publish 1", nothing pushed, still queued  red (item 20)
e2e  self-mode-worktrees-808     remnant in old .teamai/learnings-wt, contribute -> both published, old checkout gone  red
                                 init B --role ghost after init A -> no config naming A, queue aside  red
                                 init A --agent claude, init B clone fails, rerun --agent codex -> claude, codex  red on b018180f (review)
                                 same, disabledAgents codex, rerun with no --agent -> both lists kept  red on c23302d8 (review 2)
                                 remnant in old checkout, teammate already published its MR -> removed, not republished  red on 14bd21ba (item 21)
                                 import --from-mr in an unmigrated linked worktree -> old queue published  guard (Q9)
unit reports-reader-lock         failed refresh + writer takes lock -> no checkout created red
unit migrate / single-repo-mode  retire-only message; config.yaml.*.tmp in 3 lists       red

Ablations: each part was reverted alone and its test failed at the expected assertion. That covers adoption, the sync-lock guard, the dry-run guard, dedup (and source_mr-only), each signature check, the registered-checkout lookup, the namespace, staging the exact list, rename counting, item 15, item 17, Q10, the three Q11 lists, item 20 (the publish gate, the Would publish line, the warning gate), item 21 (the origin read, and the fetch alone), and for Q9 the preAction migration that now owns the linked-worktree queue.

Updated tests: writeBackConfidence returns the paths it wrote, and executePrune/executePromotion return theirs, so their assertions check those paths now. The import-mr-learnings-checkout case for a linked worktree's old queue called importCmd directly, so it bypassed the preAction migration that owns that job since Q9; it moved to the e2e above. In reports-branch-readonly, the "lock is free" case passed only because its mocked refresh failed and fell through to the unlocked ensure (item 15). Its mock now has the real status() shape.

E2E run (npm run build, --retry 0): the new files, plus self-mode-worktrees-808, import-mr-publish-823, data-layout-migration, learnings-sync-704-705, init-project-all, init-unattended, reports-writer-sync-561, members-legacy-roster, dashboard-unified, deleted-worktree-scope-810, pull-new-worktree-807, scope-isolation-issue85 and multi-project: 14 files, 80 passed. The other files that run pull --dry-run (e2e, managed-resources-uninstall, push-stale-worktree-812, push-sync-followups-823, role-scoped-agents): 5 files, 34 passed, 22 skipped. The full e2e suite was not run. After the review fixes: self-mode-worktrees-808, learnings-remnants-823, init-project-all, init-unattended, data-layout-migration, multi-project: 6 files, 40 passed. After review 2: self-mode-worktrees-808, learnings-remnants-823: 2 files, 32 passed. After review 3: those two plus learnings-sync-704-705: 3 files, 33 passed. After review 4: the same 3 files, 33 passed. After review 5: the same 3 files, 33 passed.

Before: single-repo contribute with the remnant in the old checkout: ⚠ ... still has teamai-learnings checked out, and git will not remove it, Saved locally, nothing on origin.
After: ⚠ Queued 1 learning(s) an older teamai import --from-mr left unpublished: …, ✔ Contributed, the old checkout is gone, and both learnings are on teamai-learnings.

Real CLI, #823 item 7 step 5: before (origin/main 7c834ce) and after (this branch)
===== Before (origin/main 7c834ce4)
$ git -C .teamai/learnings-wt status --short
  ?? learnings/2026-09-20-Quokka-cache-warmup-before-deploy.md
$ teamai contribute --title zebrafish --file note.md
✔ Slimmed single-repo .teamai/: relocated 1 machine-data item(s) to <sandbox>/home/.teamai/projects/<slug> (team knowledge stays in the repo).
- Contributing session knowledge...
ℹ Updated .teamai/.gitignore for current machine-local files — please `git add .teamai/.gitignore` and commit it.
⚠ <sandbox>/project/.teamai/learnings-wt still has teamai-learnings checked out, and git will not remove it (fatal: '<sandbox>/project/.teamai/learnings-wt' contains modified or untracked files, use --force to delete it). teamai now keeps that checkout at <sandbox>/home/.teamai/projects/<slug>/learnings-wt, shared by every checkout of this repo. Commit or move the uncommitted changes in <sandbox>/project/.teamai/learnings-wt (or unlock it, if git says it is locked), or delete it by hand, which loses those uncommitted changes, then run the command again.
⚠ Saved locally (an old teamai-learnings checkout is in the way; see the warning above). It stays queued and recallable here, but no `teamai pull` can publish it until that checkout is dealt with: do what the refusal says, then run `teamai pull`.
[exit 0]
$ git worktree list
  <sandbox>/project 7b219ce [main]
  <sandbox>/project/.teamai/learnings-wt 39864b6 [teamai-learnings]
old checkout .teamai/learnings-wt exists: yes
$ git -C origin.git ls-tree -r --name-only teamai-learnings
  .gitignore
  learnings/.gitkeep

===== After (this branch, 14bd21ba)
$ git -C .teamai/learnings-wt status --short
  ?? learnings/2026-09-20-Quokka-cache-warmup-before-deploy.md
$ teamai contribute --title zebrafish --file note.md
✔ Slimmed single-repo .teamai/: relocated 1 machine-data item(s) to <sandbox>/home/.teamai/projects/<slug> (team knowledge stays in the repo).
- Contributing session knowledge...
ℹ Updated .teamai/.gitignore for current machine-local files — please `git add .teamai/.gitignore` and commit it.
⚠ Queued 1 learning(s) an older teamai import --from-mr left unpublished: <sandbox>/project/.teamai/learnings-wt/learnings/2026-09-20-Quokka-cache-warmup-before-deploy.md
✔ Contributed: learnings/zebrafish-2026-09-25-1ceqvc.md
ℹ Your session knowledge has been shared with the team.
[exit 0]
$ git worktree list
  <sandbox>/project 50d95ef [main]
  <sandbox>/home/.teamai/projects/<slug>/learnings-wt 808637a [teamai-learnings]
old checkout .teamai/learnings-wt exists: no
$ git -C origin.git ls-tree -r --name-only teamai-learnings
  .gitignore
  learnings/.gitkeep
  learnings/quokka-cache-warmup-before-deploy-2026-09-25-4hdqk0.md
  learnings/zebrafish-2026-09-25-1ceqvc.md

recall maintenance --prune --archive alone, git mode (before / after): origin keeps learnings/low-note.md and the checkout keeps a staged R learnings/low-note.md -> learnings/_archive/low-note.md / ✔ Published maintenance changes to the learnings branch, origin has learnings/_archive/low-note.md.

Real CLI, #823 item 20: pull --dry-run with one queued learning, before (14bd21b) and after
===== Before (14bd21ba; origin/main behaves the same on this path)
initial pull exit 0
$ teamai contribute --title otter --file note.md   (remote unreachable)
- Contributing session knowledge...
⚠ Saved locally (Pushing to <sandbox>/team.git
fatal: '<sandbox>/team.git' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
). It stays recallable here and the next `teamai pull` publishes it.
--- queue before:
<sandbox>/home/.teamai/pending-learnings/otter-2026-09-25-t8ky0j.md
--- branch before:
.gitignore
learnings/.gitkeep
$ teamai pull --dry-run
- [user] Pulling team repo...
✔ [user] Team repo: already up to date
✔ Published 1 queued learning(s)
ℹ [user] No resources to sync
[exit 0]
--- queue after:
--- branch after:
.gitignore
learnings/.gitkeep
learnings/otter-2026-09-25-t8ky0j.md

===== After
initial pull exit 0
$ teamai contribute --title otter --file note.md   (remote unreachable)
- Contributing session knowledge...
⚠ Saved locally (Pushing to <sandbox>/team.git
fatal: '<sandbox>/team.git' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
). It stays recallable here and the next `teamai pull` publishes it.
--- queue before:
<sandbox>/home/.teamai/pending-learnings/otter-2026-09-25-vuscid.md
--- branch before:
.gitignore
learnings/.gitkeep
$ teamai pull --dry-run
- [user] Pulling team repo...
✔ [user] Team repo: already up to date
ℹ [user] [dry-run] Would publish 1 queued learning(s)
ℹ [user] No resources to sync
[exit 0]
--- queue after:
<sandbox>/home/.teamai/pending-learnings/otter-2026-09-25-vuscid.md
--- branch after:
.gitignore
learnings/.gitkeep
Real CLI, #823 item 21: single-repo, leftover in the old checkout, a teammate already published its MR; before (14bd21b) and after
===== Before (14bd21ba)
$ git -C .teamai/learnings-wt status --short
  ?? learnings/2026-09-20-Quokka-cache-warmup-before-deploy.md
$ teamai contribute --title zebrafish --file note.md
✔ Slimmed single-repo .teamai/: relocated 1 machine-data item(s) to <sandbox>/home/.teamai/projects/<slug> (team knowledge stays in the repo).
- Contributing session knowledge...
ℹ Updated .teamai/.gitignore for current machine-local files — please `git add .teamai/.gitignore` and commit it.
⚠ Queued 1 learning(s) an older teamai import --from-mr left unpublished: <sandbox>/project/.teamai/learnings-wt/learnings/2026-09-20-Quokka-cache-warmup-before-deploy.md
✔ Contributed: learnings/zebrafish-2026-09-25-s39657.md
ℹ Your session knowledge has been shared with the team.
[exit 0]
old checkout .teamai/learnings-wt exists: no
$ git -C origin.git ls-tree -r --name-only teamai-learnings
  .gitignore
  learnings/.gitkeep
  learnings/quokka-cache-warmup-2026-09-21-ttt111.md
  learnings/quokka-cache-warmup-before-deploy-2026-09-25-y34jpn.md
  learnings/zebrafish-2026-09-25-s39657.md

===== After
$ git -C .teamai/learnings-wt status --short
  ?? learnings/2026-09-20-Quokka-cache-warmup-before-deploy.md
$ teamai contribute --title zebrafish --file note.md
✔ Slimmed single-repo .teamai/: relocated 1 machine-data item(s) to <sandbox>/home/.teamai/projects/<slug> (team knowledge stays in the repo).
- Contributing session knowledge...
ℹ Updated .teamai/.gitignore for current machine-local files — please `git add .teamai/.gitignore` and commit it.
⚠ Removed <sandbox>/project/.teamai/learnings-wt/learnings/2026-09-20-Quokka-cache-warmup-before-deploy.md, which an older teamai import --from-mr left unpublished: learnings/quokka-cache-warmup-2026-09-21-ttt111.md already has it.
✔ Contributed: learnings/zebrafish-2026-09-25-0gtcnh.md
ℹ Your session knowledge has been shared with the team.
[exit 0]
old checkout .teamai/learnings-wt exists: no
$ git -C origin.git ls-tree -r --name-only teamai-learnings
  .gitignore
  learnings/.gitkeep
  learnings/quokka-cache-warmup-2026-09-21-ttt111.md
  learnings/zebrafish-2026-09-25-0gtcnh.md
Real CLI, review fix for item 17: init rerun after the replacement clone failed, before (b018180) and after
===== Before (b018180f)
$ teamai init https://git.example.com/team/team-a.git --scope project --force --agent claude   (exit 0)
--- enabledAgents
  - claude
(team-b.git taken offline)
$ teamai init https://git.example.com/team/team-b.git --scope project --force
⚠ Moved <partition>/config.yaml to <partition>/config.yaml.previous: ...
✖ Clone failed: git clone failed: Cloning into '<partition>/team-repo'...
[exit 1]
config.yaml exists: no
$ teamai init https://git.example.com/team/team-b.git --scope project --force --agent codex
✔ Local config saved to <partition>/config.yaml
[exit 0]
--- config.yaml after init B --agent codex
  remote: https://git.example.com/team/team-b.git
enabledAgents:
  - codex

===== After (c23302d8)
$ teamai init https://git.example.com/team/team-a.git --scope project --force --agent claude   (exit 0)
--- enabledAgents
  - claude
(team-b.git taken offline)
$ teamai init https://git.example.com/team/team-b.git --scope project --force
⚠ Moved <partition>/config.yaml to <partition>/config.yaml.previous: ...
✖ Clone failed: git clone failed: Cloning into '<partition>/team-repo'...
[exit 1]
config.yaml exists: no
$ teamai init https://git.example.com/team/team-b.git --scope project --force --agent codex
✔ Local config saved to <partition>/config.yaml
[exit 0]
--- config.yaml after init B --agent codex
  remote: https://git.example.com/team/team-b.git
enabledAgents:
  - claude
  - codex
Real CLI, review 2: init rerun with no --agent after the replacement clone failed, before (c23302d) and after
===== Before (c23302d8)
(config edited as uninstall --agent codex records it)
--- after init A: agent lists
enabledAgents:
  - claude
disabledAgents:
  - codex
(team-b.git taken offline)
$ teamai init https://git.example.com/team/team-b.git --scope project --force
⚠ Moved <partition>/config.yaml to <partition>/config.yaml.previous: ...
✖ Clone failed: git clone failed: Cloning into '<partition>/team-repo'...
[exit 1]
config.yaml exists: no
$ teamai init https://git.example.com/team/team-b.git --scope project --force
✔ Local config saved to <partition>/config.yaml
[exit 0]
--- config.yaml after init B (no --agent)
  remote: https://git.example.com/team/team-b.git

===== After (8523c0d3)
(config edited as uninstall --agent codex records it)
--- after init A: agent lists
enabledAgents:
  - claude
disabledAgents:
  - codex
(team-b.git taken offline)
$ teamai init https://git.example.com/team/team-b.git --scope project --force
⚠ Moved <partition>/config.yaml to <partition>/config.yaml.previous: ...
✖ Clone failed: git clone failed: Cloning into '<partition>/team-repo'...
[exit 1]
config.yaml exists: no
$ teamai init https://git.example.com/team/team-b.git --scope project --force
✔ Local config saved to <partition>/config.yaml
[exit 0]
--- config.yaml after init B (no --agent)
  remote: https://git.example.com/team/team-b.git
enabledAgents:
  - claude
disabledAgents:
  - codex
Real CLI, review 3: recall maintenance --prune --archive with origin offline, then pull; before (8523c0d) and after
===== Before (8523c0d3)
(same steps and maintenance output as below)
$ teamai pull
- [user] Pulling team repo...
✔ [user] Team repo: already up to date
✔ [user] Already synced at bb7e2b7, skipping
✔ Synced 1 learnings (index: 1ms)
[exit 0]
$ git -C team.git ls-tree -r --name-only teamai-learnings
  .gitignore
  learnings/.gitkeep
  learnings/low-note.md
$ git -C learnings-wt status --short; unpushed commits
  1 ahead of origin

===== After (331d9296)
initial pull exit 0
(team.git taken offline)
$ teamai recall maintenance --prune --archive
ℹ Found 1 candidate(s) for pruning:
ℹ   - low-note.md (confidence: 0.04, reason: confidence 0.04 < 0.15)
✔ Archived 1 learning(s)
⚠ Maintenance changes stay local for now: Pushing to <sandbox>/team.git
fatal: '<sandbox>/team.git' does not appear to be a git repository
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
(team.git back online)
$ teamai pull
- [user] Pulling team repo...
✔ [user] Team repo: already up to date
✔ Published earlier maintenance changes to the learnings branch
✔ [user] Already synced at 55ba79e, skipping
✔ Synced 1 learnings (index: 1ms)
[exit 0]
$ git -C team.git ls-tree -r --name-only teamai-learnings
  .gitignore
  learnings/.gitkeep
  learnings/_archive/low-note.md
$ git -C learnings-wt status --short; unpushed commits
  0 ahead of origin
Real CLI, review 4: origin without teamai-learnings, a leftover in the checkout, then pull; before (331d929) and after
===== Before (331d9296)
initial pull exit 0
teamai-learnings
(origin has no teamai-learnings, as after an offline first publish)
(planted learnings-wt/learnings/2026-09-20-Quokka-cache-warmup.md, untracked)
$ teamai pull
- [user] Pulling team repo...
✔ [user] Team repo: already up to date
✔ [user] Already synced at f1bbba1, skipping
✔ Synced 1 learnings (index: 1ms)
[exit 0]
$ git -C team.git ls-tree -r --name-only teamai-learnings
  fatal: Not a valid object name teamai-learnings
$ git -C learnings-wt status --short
  ?? learnings/2026-09-20-Quokka-cache-warmup.md

===== After (this branch)
initial pull exit 0
teamai-learnings
(origin has no teamai-learnings, as after an offline first publish)
(planted learnings-wt/learnings/2026-09-20-Quokka-cache-warmup.md, untracked)
$ teamai pull
- [user] Pulling team repo...
✔ [user] Team repo: already up to date
⚠ Queued 1 learning(s) an older teamai import --from-mr left unpublished: <sandbox>/home/.teamai/learnings-wt/learnings/2026-09-20-Quokka-cache-warmup.md
✔ Published 1 queued learning(s)
✔ [user] Already synced at 1e14dc2, skipping
✔ Synced 1 learnings (index: 1ms)
[exit 0]
$ git -C team.git ls-tree -r --name-only teamai-learnings
  .gitignore
  learnings/.gitkeep
  learnings/quokka-cache-warmup-2026-09-26-tj2ob3.md
$ git -C learnings-wt status --short
  (clean)
Real CLI, review 5: origin deleted the learning the checkout still tracks, a leftover from the same MR, then pull; before (4defc3e) and after
===== Before (4defc3ee)
initial pull exit 0
$ git -C learnings-wt ls-files learnings
  learnings/alpha/quokka-2026-09-21-aaa111.md
(a teammate deleted learnings/alpha/quokka-2026-09-21-aaa111.md on origin)
(planted learnings-wt/learnings/2026-09-20-Quokka-cache-warmup.md, untracked, same source_mr)
$ teamai pull
- [user] Pulling team repo...
✔ [user] Team repo: already up to date
⚠ Removed <sandbox>/home/.teamai/learnings-wt/learnings/2026-09-20-Quokka-cache-warmup.md, which an older teamai import --from-mr left unpublished: learnings/alpha/quokka-2026-09-21-aaa111.md already has it.
✔ [user] Already synced at 61d7615, skipping
[exit 0]
$ git -C team.git ls-tree -r --name-only teamai-learnings
  .gitignore
$ ls learnings-wt/learnings/2026-09-20-Quokka-cache-warmup.md
  ls: <sandbox>/home/.teamai/learnings-wt/learnings/2026-09-20-Quokka-cache-warmup.md: No such file or directory
===== After
initial pull exit 0
$ git -C learnings-wt ls-files learnings
  learnings/alpha/quokka-2026-09-21-aaa111.md
(a teammate deleted learnings/alpha/quokka-2026-09-21-aaa111.md on origin)
(planted learnings-wt/learnings/2026-09-20-Quokka-cache-warmup.md, untracked, same source_mr)
$ teamai pull
- [user] Pulling team repo...
✔ [user] Team repo: already up to date
⚠ Queued 1 learning(s) an older teamai import --from-mr left unpublished: <sandbox>/home/.teamai/learnings-wt/learnings/2026-09-20-Quokka-cache-warmup.md
✔ Published 1 queued learning(s)
✔ [user] Already synced at ceb258b, skipping
✔ Synced 1 learnings (index: 1ms)
[exit 0]
$ git -C team.git ls-tree -r --name-only teamai-learnings
  .gitignore
  learnings/quokka-cache-warmup-2026-09-26-cxzf0f.md
$ ls learnings-wt/learnings/2026-09-20-Quokka-cache-warmup.md
  ls: <sandbox>/home/.teamai/learnings-wt/learnings/2026-09-20-Quokka-cache-warmup.md: No such file or directory

Related Issues

Part of #823 (item 7 and #814 follow-ups; also items 15, 17, 20 and 21).

Notes for Reviewers

Door: two-way. Blast radius: learnings publishing and maintenance, init replacing a team repo, reports readers.

  • Invariant (review 5): a remnant or a maintenance record is deleted only after positive proof that its content is safe, never because information is missing. For a remnant, safe means its copy is in the durable queue (which drops an entry only once origin has it), or origin's freshly fetched branch, or the queue, has the same source_mr or content. For a record, it means commitAndPush returned published, or already-present (nothing staged and nothing past origin/teamai-learnings), or a successful read found it malformed. Audit of every delete site in the new code: the covered-remnant rm now checks origin's tree, not the checkout; the queued-remnant rm follows a saved queue write; the malformed-record rm now needs a successful read; the published-record rm is gated on the two statuses above. When origin cannot be fetched or read, or a queue entry or record cannot be read, nothing is deleted. Any other untracked file in the checkout is left alone.
  • publishQueuedLearnings now takes the sync lock before it lists the queue (sync, then queue lock, the same order as the migration), so two commands cannot adopt one remnant twice. pull still passes holdsSyncLock.
  • Item 17 moves the old config aside rather than deleting it. A rerun that finds no config.yaml reads the last config.yaml.previous[.<n>] for the settings a re-init carries forward (enabledAgents, disabledAgents, toolRoots, inheritUserScope), and only those. With no --agent, both agent lists are kept as they were; before, any re-init without --agent dropped them, which contradicted the documented durable uninstall --agent exclusion. With --agent, behavior is unchanged. Queue ownership and the settle still go by the live config. initHttp never moves a config aside and is unchanged.
  • Item 21 fetches origin/teamai-learnings only when there is a leftover to check, so ordinary runs make no extra network call. When the fetch fails, the leftover stays where it is until a run can fetch (review 3). When origin answers ls-remote without the branch (an offline first publish never pushed it), origin adds nothing and the leftover is queued (review 4); only a failed ls-remote keeps it. Review 5: the duplicate check reads the fetched tree itself (ls-tree plus one git show per learning), so a copy the checkout still tracks after origin deleted or replaced it no longer counts. That costs one git call per learning on origin, only in a run that finds a leftover.
  • Not changed: pull --dry-run does not preview leftovers a real run would queue or delete, and it still takes the queue lock briefly to list the queue (both P3 from the local review). Item 16 (viz) is left for [bug] Follow-ups to the worktree fixes: import --from-mr never shares its learning, and push can still revert a teammate's update #823.
  • Local adversarial review (the Codex prompt, run in a read-only Claude subagent) and /code-review found no P1. Fixed: the version range (the issue said 0.22), <date>-.md names, a removed never-tracked file failing the maintenance add, docs wording. Not changed: item 17 settles the old install before the clone, so a failed clone followed by init A again leaves A's queue in pending-learnings.git-<A> (named in the warning, nothing lost); settling after the clone would bring back item 17's busy-lock case. A remnant with the same source_mr as a published learning is deleted even if its content differs, as decided for [bug] Follow-ups to the worktree fixes: import --from-mr never shares its learning, and push can still revert a teammate's update #823.
  • Maintenance retry (review 3): maintenance writes before it takes the branch lock, and a rerun finds nothing to change, so each run records its paths first, in the learnings checkout's git dir (teamai-maintenance/, one file per run). No gitignore change is needed there, and the record goes with the checkout and its changes. Every later publishLearningsMaintenance, and every publishQueuedLearnings under the sync lock (pull, contribute, import --from-mr), publishes all the records, including when the queue is empty. A record is dropped only on published or already-present. A removed file that an earlier attempt already committed is left out of the git add, so the commit it made is still pushed. Remaining edge: if a refresh rebases an unpushed maintenance commit that conflicts with a teammate's change on origin, the refresh drops that commit, as it always has, so the teammate's version wins, and the record then clears as already-present. Review 4: records are written through writeFileAtomic, and only *.json is read, so an interrupted write leaves a .tmp nobody reads; a record that does not parse is removed with ⚠ Removed unreadable maintenance record <file> (...), naming the checkout to check with git status, and the other records still publish.
  • Promote has no e2e test here (it calls a model); its mark goes through the same exact-list path, which the maintenance-promote unit test checks.

…eckout (Tencent#823)

Item 7. import --from-mr in 0.25.0 to 0.26.0-beta.3 wrote
learnings/<date>-<title>.md, with source_mr in its frontmatter, into the
learnings checkout and never committed it. Nothing published it. In single-repo mode it also kept
`git worktree remove` from removing the checkout an older teamai left in
.teamai/, so every pull and contribute stopped on CheckoutRefusedError.
publishQueuedLearnings now takes the sync lock first, and under it, before
listing the queue, queues every untracked file of exactly that shape
(directly under learnings/, date name, source_mr), in the active namespace
and with contribute's name, then deletes the original. It finds the one
checkout this repo registers for the branch (git worktree list), so the
shared checkout and the old .teamai/learnings-wt are both covered and
another repository's never is. A file the branch or the queue already has,
by source_mr or by content, is deleted instead, and the warning names what
has it. A dry run touches nothing.

Item 21. The branch side of that duplicate check was the checkout's own
tracked files. In single-repo mode the checkout is often the old
.teamai/learnings-wt, which nothing syncs any more, so a teammate's later
import of the same MR was missed and the remnant went out as a duplicate.
When there are remnants, the check now also fetches origin/teamai-learnings
(best effort) and reads what origin has that the checkout's commit lacks.

Item 20. pull --dry-run published the queue: publishQueuedLearnings
honoured dryRun only for the remnants. It now stops after listing the queue,
and pull prints "[dry-run] Would publish N queued learning(s)" instead of
publishing or warning.

Maintenance sweep. publishLearningsMaintenance staged all of learnings/,
so a confidence write-back or a prune swept any uncommitted file into its
commit. confidence write-back, prune and promote now return the files they
wrote or removed, and only those are staged (a removed file git never
tracked is left out, since naming it would fail the add). That exposed a
second bug:
simple-git lists a staged rename under `renamed`, not `staged`, so a
`prune --archive` with nothing else to stage counted as nothing to commit
and was never published. commitAndPushAt now counts renames.

Tencent#814 follow-ups. drainCheckoutQueue is gone: the preAction migration moves a
checkout's queue before contribute and import --from-mr. Retire-only now
says "Retired <legacy> to <backup>: this project's data already lives in
<partition>"; a linked worktree lands there too, so "Finished an
interrupted migration" was wrong for it. config.yaml.*.tmp, the temp an
interrupted config save leaves (Tencent#831), is ignored in the single-repo and
project-scope .gitignore, and the single-repo self-heal adds it.

Item 15. After a failed refresh, readableReportsWorktree called ensure
without the reports lock, so it could create the checkout while a writer
that had just taken the lock created it too. It now refreshes once more
under the lock and throws the cause if that fails as well.

Item 17. init replaced the team clone before saving the new config, so an
init that stopped in between (an unknown --role, a busy queue lock) left
the old team's config.yaml beside the new team's clone. Just before it
clones another owner's repo, init now settles the old install as the final
save would (queue set aside, indexes dropped) and moves its config.yaml to
config.yaml.previous. A failed init then leaves no config, and commands ask
for teamai init.
@SaulMoro
SaulMoro force-pushed the fix-823-data-layout-followups branch from 14bd21b to b018180 Compare September 25, 2026 19:28
@jeff-r2026 jeff-r2026 self-assigned this Sep 26, 2026
@github-actions

Copy link
Copy Markdown
  • [P2 non-blocking] Preserve config settings across a failed replacement clone — src/init.ts:1502. The old config is renamed before the fallible clone. If cloning fails and the user reruns init, loadLocalConfigForScope finds no config, so settings such as enabledAgents, disabledAgents, toolRoots, and inheritUserScope are no longer carried forward despite remaining in config.yaml.previous. Load the backup on retry or defer final retirement until the replacement clone succeeds.
  • [P2 non-blocking] Treat maintenance filenames as literal Git paths — src/utils/learnings-publish.ts:309. The newly introduced per-file list is passed directly to git add, so valid learning filenames containing pathspec metacharacters such as [ or * can fail to stage themselves or match unrelated files. The preceding ls-files lookup has the same issue at line 303. Prefix paths with :(literal) or otherwise disable pathspec expansion.

The PR description includes sufficient unit and representative real-CLI testing evidence.

…ter a failed clone

Maintenance now stages exactly the files it names: commitAndPushAt and the
removed-file ls-files lookup pass --literal-pathspecs, so a learning named
with [ or * no longer stages the stray files it matches as a pattern.

init reads the config it set aside when the rerun finds none, so an init
whose replacement clone failed no longer drops enabledAgents,
disabledAgents, toolRoots and inheritUserScope on the next run.
@github-actions

Copy link
Copy Markdown
  • [P2 non-blocking] Preserve agent settings when retrying without --agent — src/init.ts:1815. enabledAgents and disabledAgents are copied from carriedConfig only when the retry supplies at least one agent. After a replacement clone fails, rerunning the same init command without --agent silently drops both settings, so previously disabled tools may become active. The earlier config-preservation finding is therefore only partially resolved.
  • [P2 non-blocking] Keep failed maintenance changes retryable — src/utils/learnings-publish.ts:309. Maintenance mutates files before attempting to acquire the branch lock. If commitAndPush returns busy, those changes remain unstaged; later maintenance no longer discovers a deleted or already-updated file, and other publishers now stage only their explicit paths. The advertised retry leaves that maintenance change permanently local.
  • [P2 non-blocking] Avoid running Git preflight for HTTP maintenance — src/utils/learnings-publish.ts:303. For an HTTP install, the learnings write directory is the non-Git HTTP cache. A prune supplies a removed path, causing this new git ls-files call to throw outside commitAndPush’s failure handling after the local deletion has already occurred. Previously this path returned a non-fatal failed publish result.

The earlier literal-pathspec finding is resolved. The PR description includes sufficient representative real-CLI testing evidence.

…lain rerun

An HTTP install's learnings dir is no git checkout, so the removed-file
ls-files lookup threw after a prune had already deleted the file. It now
returns the same non-fatal failed publish commitAndPush gives.

init without --agent keeps the carried enabledAgents and disabledAgents,
so a rerun after a failed replacement clone no longer reactivates tools
uninstall --agent excluded.
@github-actions

Copy link
Copy Markdown
  • [P2 non-blocking] Keep failed maintenance changes retryable — src/index.ts:1428. Maintenance mutates files before publishLearningsMaintenance acquires the branch lock. If that call returns busy or commits locally but fails to push, rerunning the command finds no candidates/changed confidence and never calls the publisher again; an empty learning queue also will not push the pending commit. The changes can remain permanently local or later be discarded by refresh. Perform the mutation under the branch lock or persist/retry the changed paths.
  • [P2 non-blocking] Do not finalize remnant adoption after a failed origin refresh — src/utils/learnings-publish.ts:263. If the fetch fails while the local tracking ref is stale and a teammate has already published the same source_mr, the remnant is queued and deleted based on stale data. A later online publish syncs the branch but never repeats duplicate detection, so it publishes a duplicate. Keep the remnant until a current comparison succeeds, or re-check queued remnants after synchronization.

The previous config-preservation, literal-pathspec, and HTTP-maintenance findings are resolved. The PR description contains sufficient representative real-CLI testing evidence.

…led push kept local, keep remnants while origin is unreachable
@github-actions

Copy link
Copy Markdown
  • [P2 non-blocking] Handle a genuinely absent remote learnings branch — src/utils/learnings-publish.ts:228. publishedSinceCheckout() returns null whenever the explicit fetch fails, including when origin/teamai-learnings does not exist. An older offline import can create the local worktree and leave a remnant after the initial branch push fails; later runs then always abandon adoption, and in self mode the remnant continues blocking replacement of the old worktree. Treat a confirmed missing remote branch as an empty remote rather than an unavailable one.
  • [P3 nit] Do not let one truncated maintenance record block every retry — src/utils/learnings-publish.ts:353. Records are written non-atomically, and JSON.parse() is outside a per-record catch. If a process exits during writeFile, that partial JSON causes every later pull or contribution to return failed before processing any valid records, permanently preventing automatic maintenance retries. Write records atomically and/or skip malformed records individually.

The earlier config-preservation, literal-pathspec, HTTP-maintenance, maintenance-retry, and stale-origin findings are otherwise resolved. The PR description includes sufficient representative real-CLI testing evidence.

…arnings branch, never let one bad maintenance record or remnant block the rest
@github-actions

Copy link
Copy Markdown

Findings

  • [P2 non-blocking] Deduplicate against the fetched branch rather than stale tracked files — src/utils/learnings-publish.ts:225. known retains the checkout’s tracked version even when origin has since deleted or replaced that file; the later diff only adds A/M entries and never removes the stale version. A remnant sharing its source_mr can therefore be deleted, and a later refresh can remove the stale tracked copy, leaving no copy of the learning.
  • [P3 nit] Keep maintenance records on transient read failures — src/utils/learnings-publish.ts:381. Any readFile failure is treated like malformed JSON and the record is removed at line 391. A temporary permission or I/O error therefore disables future automatic retries for that maintenance change. Remove records only after successful reading proves them malformed.

The previously reported findings are otherwise resolved. The PR description includes sufficient representative real-CLI testing evidence.

@github-actions

Copy link
Copy Markdown

Findings

  • [P2 non-blocking] Revalidate queue ownership before deleting a remnant — src/utils/learnings-publish.ts:230. The queue is inspected before listPendingForInstall verifies that the loaded config still owns it. A concurrent init can switch the project to another repository, after which this stale publisher may treat the new install’s queued item as a duplicate and delete the old install’s remnant. Perform the ownership check while holding the queue lock before using queue contents or removing remnants.
  • [P2 non-blocking] Prevent pre-staged files from joining maintenance commits — src/utils/learnings-publish.ts:451. Passing an exact file list to git add does not constrain the subsequent plain git commit, which commits every path already staged. If the checkout contains an unrelated staged draft, any maintenance publish will send it too, contrary to “publishes only what it changed.” Commit only the supplied paths or otherwise preserve and exclude the existing index.

All previously reported findings are resolved in the current diff. The PR description includes sufficient representative real-CLI testing evidence.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants