From 88afcc1d52e0152f06a9776ee9481fa1b59e647f Mon Sep 17 00:00:00 2001 From: "Jose I. Paris" Date: Wed, 21 Oct 2026 13:51:38 +0200 Subject: [PATCH 1/2] feat(trace): check net contribution, session updates and rewrites end to end Add scenarios that run the git commands of a human after the agent session and push again: a commit that replaces an AI line, an amend of the session commit, and a rename in a later commit. They work on a feature branch, because the push reads only the commits that are not on the default branch of the remote. Register the checks of the Net contribution, Session updates and Renames and rewrites rows. Refs #3635 Assisted-by: Claude Code Signed-off-by: Jose I. Paris Chainloop-Trace-Sessions: 0f698192-ddc8-487e-b214-474edc859d4e --- app/cli/internal/trace/README.md | 6 +- app/cli/internal/trace/e2e/checks.go | 15 +- app/cli/internal/trace/e2e/checks_more.go | 99 +++++++++++ app/cli/internal/trace/e2e/checks_test.go | 205 ++++++++++++++++++++++ app/cli/internal/trace/e2e/runner.go | 10 ++ app/cli/internal/trace/e2e/scenarios.go | 144 ++++++++++++++- app/cli/internal/trace/e2e/workspace.go | 27 ++- 7 files changed, 492 insertions(+), 14 deletions(-) create mode 100644 app/cli/internal/trace/e2e/checks_test.go diff --git a/app/cli/internal/trace/README.md b/app/cli/internal/trace/README.md index d75cca488..b04f65bbd 100644 --- a/app/cli/internal/trace/README.md +++ b/app/cli/internal/trace/README.md @@ -67,8 +67,8 @@ Why not Yes: - **Line attribution, partial commits and generated line counts, Cursor:** Cursor has only the `afterFileEdit` hook, which gives the edits but no snapshot before the edit. The provider rebuilds the content before the edit from the edits. - **Net contribution and line ranges, all agents:** the push finds the last writer of each line with `git blame`. A later commit that only re-indents a line takes the line, because blame sees a change. For Cursor, the limits of line attribution apply too. -- **Session updates, all agents:** the update works only in the clone where the work happened, while the CLI keeps the trace state of the session (30 days), and only when the commits of the session are on the branch that the push sends. -- **Renames and rewrites, all agents:** this works only in the clone where the work happened. Git finds a rename only when the file keeps at least half of its content. Two pushed commits can rename a file one after the other. A later change to the file under its middle name does not send the session again. For Cursor, the limits of line attribution apply too. +- **Session updates, all agents:** the update works only in the clone where the work happened, while the CLI keeps the trace state of the session (30 days), and only when the commits of the session are on the branch that the push sends. The push reads only the commits that are not on the default branch of the remote (`origin/HEAD`, or else `origin/main` or `origin/master`). After the commits of a session are pushed to that branch, a later commit does not update the session. +- **Renames and rewrites, all agents:** this works only in the clone where the work happened. Git finds a rename only when the file keeps at least half of its content. Two pushed commits can rename a file one after the other. A later change to the file under its middle name does not send the session again. A rename inside an amend or a squash of the session commit loses the AI lines: the new commit creates the file under its new name, so no line of it matches the trace state. The limits of session updates apply too. For Cursor, the limits of line attribution apply too. - **Shell changes attribution, all agents:** the hook before a shell command records only a hash of each file, not its content. The file at the `HEAD` commit from before the command stands in for the content before the command. Uncommitted lines that an earlier agent edit recorded are not recorded again, but uncommitted human lines of a file that the command changes count as AI. - **Shell changes attribution, Cursor:** the provider installs the `preToolUse` and `postToolUse` hooks for the `Shell` tool. These hooks are verified for the Cursor IDE only. It is not verified that the Cursor CLI sends them. @@ -166,6 +166,8 @@ All agents share the git hooks `commit-msg`, `post-commit` and `pre-push`. The `e2e/` package checks the tables of this page against real agents (spec issue-3635). It runs each agent in headless mode with real models, in throwaway repositories with Trace in local storage mode. For now, it runs the Claude Code, OpenCode 2, Pi, and Codex columns. The Pi and Codex cells are **No**, so the suite skips them until Pi and Codex support is added. OpenCode runs on a private server (`opencode run --standalone`) with `PWD` set to the workspace, because OpenCode takes its project folder from `PWD`. Pi runs with `pi -p --mode json --approve`, with its configuration and session folders next to the workspace. Codex runs with `codex exec --json`, with its `CODEX_HOME` folder next to the workspace, no sandbox, and no hook trust prompt. The OpenCode 2, Pi, and Codex models in `e2e/config.yaml` are OpenRouter models, so the run needs `OPENROUTER_API_KEY`. A Codex model with no `openrouter/` prefix runs on the default Codex provider, which needs `CODEX_API_KEY`. +Some scenarios add the git commands of a human after the session, for example a commit that replaces an AI line, an amend, or a rename, and then push again. They work on a feature branch, because the push reads only the commits that are not on the default branch of the remote. + A feature row maps to its checks by the feature name. When you rename a feature in a table, rename it in `e2e/checks.go` too. The suite tests a cell marked **Yes**, **Partial**, or **Unknown** only when the row has a registered check. It reports the other cells as follows: - A cell marked **No** or **Not applicable** is `skipped`. diff --git a/app/cli/internal/trace/e2e/checks.go b/app/cli/internal/trace/e2e/checks.go index e6c5470c5..7ec228446 100644 --- a/app/cli/internal/trace/e2e/checks.go +++ b/app/cli/internal/trace/e2e/checks.go @@ -47,8 +47,11 @@ type Session struct { Output *RunOutput // Stored is the session that the push stored in local storage mode. Stored *StoredSession - // HeadMessage is the message of the commit that the agent made. + // HeadMessage is the message of the last commit of the checkout: the + // commit that the agent made, or the commit of the follow-up. HeadMessage string + // HeadSHA is the last commit of the checkout. + HeadSHA string } // Check verifies one feature on the result of one scenario session. @@ -95,11 +98,17 @@ var features = map[string]Feature{ "Subagents": {Checks: []Check{{scenarioSubagent, checkSubagents}}}, // Code attribution - "Line attribution": {Checks: []Check{{scenarioEditAndCommit, checkFileAttributedToAI("greeting.txt")}}}, + "Line attribution": {Checks: []Check{{scenarioEditAndCommit, checkFileAttributedToAI(greetingFile)}}}, "Shell changes attribution": {Checks: []Check{{scenarioEditAndCommit, checkFileAttributedToAI("shell.txt")}}}, "Partial commits": {Checks: []Check{{scenarioPartial, checkBothCommitsCredited}}}, - "Line ranges": {Checks: []Check{{scenarioEditAndCommit, checkLineRanges("greeting.txt", 3)}}}, + "Line ranges": {Checks: []Check{{scenarioEditAndCommit, checkLineRanges(greetingFile, 3)}}}, "Generated line counts": {Checks: []Check{{scenarioEditAndCommit, checkGeneratedLineCounts}}}, + "Net contribution": {Checks: []Check{{scenarioHumanUpdate, checkNetContribution}}}, + "Session updates": {Checks: []Check{{scenarioHumanUpdate, checkSessionUpdated}}}, + "Renames and rewrites": {Checks: []Check{ + {scenarioAmend, checkAmended}, + {scenarioRename, checkRenamed}, + }}, // User experience "Welcome message": {Checks: []Check{{scenarioEditAndCommit, checkWelcomeMessage}}}, diff --git a/app/cli/internal/trace/e2e/checks_more.go b/app/cli/internal/trace/e2e/checks_more.go index 3b34f6f60..91c4d1a4e 100644 --- a/app/cli/internal/trace/e2e/checks_more.go +++ b/app/cli/internal/trace/e2e/checks_more.go @@ -30,6 +30,7 @@ import ( "time" "github.com/chainloop-dev/chainloop/app/cli/internal/trace" + tracegit "github.com/chainloop-dev/chainloop/app/cli/internal/trace/git" "github.com/chainloop-dev/chainloop/app/cli/internal/trace/pointer" "github.com/chainloop-dev/chainloop/app/cli/internal/trace/state" "github.com/chainloop-dev/chainloop/pkg/attestation/crafter/materials/aicodingsession" @@ -243,3 +244,101 @@ func checkLineRanges(path string, lines int) func(s *Session) error { return fmt.Errorf("the code changes do not list %s", path) } } + +// fileChange returns the change of a file in the evidence. +func fileChange(s *Session, path string) (*aicodingsession.FileChange, error) { + cc := evidence(s).CodeChanges + if cc == nil { + return nil, errors.New("the evidence has no code changes") + } + for i := range cc.Files { + if cc.Files[i].Path == path { + return &cc.Files[i], nil + } + } + return nil, fmt.Errorf("the code changes do not list %s", path) +} + +// headInSession checks that the last commit of the checkout is in the commit +// list of the session. Each entry of the list is the SHA and the subject of a +// commit. +func headInSession(s *Session) error { + gc := evidence(s).GitContext + if gc != nil && slices.ContainsFunc(gc.Commits, func(c string) bool { + sha, _, _ := strings.Cut(c, " ") + return sha == s.HeadSHA + }) { + return nil + } + return fmt.Errorf("the commit list of the session does not hold the last commit %s", s.HeadSHA) +} + +// checkNetContribution checks that the line that the human commit replaced +// no longer counts as AI, that each line counts once, and that the file that +// the session created stays created. +func checkNetContribution(s *Session) error { + f, err := fileChange(s, greetingFile) + if err != nil { + return err + } + if f.AILinesAdded != 2 || f.HumanLinesAdded != 1 { + return fmt.Errorf("%s: %d AI lines and %d human lines added, want 2 and 1", greetingFile, f.AILinesAdded, f.HumanLinesAdded) + } + if f.LinesAdded != f.AILinesAdded+f.HumanLinesAdded { + return fmt.Errorf("%s: %d lines added, want the sum of the AI and human lines", greetingFile, f.LinesAdded) + } + if f.Status != tracegit.StatusCreated { + return fmt.Errorf("%s: status %q, want created", greetingFile, f.Status) + } + return nil +} + +// checkSessionUpdated checks that the push after the human commit stored the +// session again, with the human commit in its commit list and the replacing +// line as a human line. +func checkSessionUpdated(s *Session) error { + if err := headInSession(s); err != nil { + return err + } + f, err := fileChange(s, greetingFile) + if err != nil { + return err + } + if f.HumanLinesAdded != 1 { + return fmt.Errorf("%s: %d human lines added, want the 1 replacing line", greetingFile, f.HumanLinesAdded) + } + return nil +} + +// checkAmended checks that the session follows its amended commit, with the +// lines of the agent as AI lines and the line that the amend added as a human +// line. +func checkAmended(s *Session) error { + if err := headInSession(s); err != nil { + return err + } + f, err := fileChange(s, greetingFile) + if err != nil { + return err + } + if f.AILinesAdded != 3 || f.HumanLinesAdded != 1 { + return fmt.Errorf("%s: %d AI lines and %d human lines added, want 3 and 1", greetingFile, f.AILinesAdded, f.HumanLinesAdded) + } + return nil +} + +// checkRenamed checks that the lines of the agent stay AI lines after a later +// commit renamed their file. The evidence lists the file under its path in +// the commit of the session or under its new path. +func checkRenamed(s *Session) error { + f, err := fileChange(s, greetingFile) + if err != nil { + if f, err = fileChange(s, renamedFile); err != nil { + return err + } + } + if f.AILinesAdded != 3 || f.HumanLinesAdded != 0 { + return fmt.Errorf("%s: %d AI lines and %d human lines added, want 3 and 0", f.Path, f.AILinesAdded, f.HumanLinesAdded) + } + return nil +} diff --git a/app/cli/internal/trace/e2e/checks_test.go b/app/cli/internal/trace/e2e/checks_test.go new file mode 100644 index 000000000..6ca2a0176 --- /dev/null +++ b/app/cli/internal/trace/e2e/checks_test.go @@ -0,0 +1,205 @@ +// +// Copyright 2026 The Chainloop Authors. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package e2e + +import ( + "testing" + + "github.com/chainloop-dev/chainloop/app/cli/internal/trace" + tracegit "github.com/chainloop-dev/chainloop/app/cli/internal/trace/git" + "github.com/chainloop-dev/chainloop/pkg/attestation/crafter/materials/aicodingsession" + "github.com/stretchr/testify/assert" +) + +const ( + testAgentCommit = "aaaa" + testHumanCommit = "bbbb" +) + +// sessionWith returns a session whose stored evidence has the given commits +// and file changes. Its last commit is testHumanCommit. The evidence lists each +// commit with its subject, as the push writes it. +func sessionWith(commits []string, files ...aicodingsession.FileChange) *Session { + s := &Session{Stored: &StoredSession{}, HeadSHA: testHumanCommit} + d := &s.Stored.Evidence.Data + d.GitContext = &aicodingsession.GitContext{} + for _, c := range commits { + d.GitContext.Commits = append(d.GitContext.Commits, c+" a subject") + } + d.CodeChanges = &aicodingsession.CodeChanges{Files: files} + return s +} + +// updatedGreeting is the file of the update scenario after the human commit +// replaced one of its three AI lines. +func updatedGreeting() aicodingsession.FileChange { + return aicodingsession.FileChange{ + Path: greetingFile, Status: tracegit.StatusCreated, Attribution: trace.AttributionAI, + LinesAdded: 3, AILinesAdded: 2, HumanLinesAdded: 1, + } +} + +func TestCheckNetContribution(t *testing.T) { + cases := []struct { + name string + file aicodingsession.FileChange + wantErr string + }{ + {name: "replaced line no longer counts", file: updatedGreeting()}, + { + name: "replaced line still counts as AI", + file: func() aicodingsession.FileChange { + f := updatedGreeting() + f.AILinesAdded, f.HumanLinesAdded = 3, 0 + return f + }(), + wantErr: "3 AI lines", + }, + { + name: "line counted twice", + file: func() aicodingsession.FileChange { + f := updatedGreeting() + f.LinesAdded = 4 + return f + }(), + wantErr: "4 lines added", + }, + { + name: "file no longer created", + file: func() aicodingsession.FileChange { + f := updatedGreeting() + f.Status = "modified" + return f + }(), + wantErr: "status", + }, + {name: "file not listed", file: aicodingsession.FileChange{Path: "other.txt"}, wantErr: "do not list"}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + err := checkNetContribution(sessionWith([]string{testAgentCommit, testHumanCommit}, tc.file)) + if tc.wantErr == "" { + assert.NoError(t, err) + return + } + assert.ErrorContains(t, err, tc.wantErr) + }) + } +} + +func TestCheckSessionUpdated(t *testing.T) { + cases := []struct { + name string + commits []string + file aicodingsession.FileChange + wantErr string + }{ + {name: "human commit joins the session", commits: []string{testAgentCommit, testHumanCommit}, file: updatedGreeting()}, + {name: "human commit not in the session", commits: []string{testAgentCommit}, file: updatedGreeting(), wantErr: "commit list"}, + { + name: "replacing line not human", + commits: []string{testAgentCommit, testHumanCommit}, + file: func() aicodingsession.FileChange { + f := updatedGreeting() + f.HumanLinesAdded = 0 + return f + }(), + wantErr: "human lines", + }, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + err := checkSessionUpdated(sessionWith(tc.commits, tc.file)) + if tc.wantErr == "" { + assert.NoError(t, err) + return + } + assert.ErrorContains(t, err, tc.wantErr) + }) + } +} + +func TestCheckAmended(t *testing.T) { + amended := aicodingsession.FileChange{ + Path: greetingFile, Status: tracegit.StatusCreated, Attribution: trace.AttributionAI, + LinesAdded: 4, AILinesAdded: 3, HumanLinesAdded: 1, + } + cases := []struct { + name string + commits []string + file aicodingsession.FileChange + wantErr string + }{ + {name: "split follows the amend", commits: []string{testHumanCommit}, file: amended}, + {name: "amended commit not in the session", commits: []string{testAgentCommit}, file: amended, wantErr: "commit list"}, + { + name: "agent lines lost in the amend", + commits: []string{testHumanCommit}, + file: func() aicodingsession.FileChange { + f := amended + f.AILinesAdded, f.HumanLinesAdded = 0, 4 + return f + }(), + wantErr: "0 AI lines", + }, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + err := checkAmended(sessionWith(tc.commits, tc.file)) + if tc.wantErr == "" { + assert.NoError(t, err) + return + } + assert.ErrorContains(t, err, tc.wantErr) + }) + } +} + +func TestCheckRenamed(t *testing.T) { + aiFile := func(path string) aicodingsession.FileChange { + return aicodingsession.FileChange{ + Path: path, Status: tracegit.StatusCreated, Attribution: trace.AttributionAI, LinesAdded: 3, AILinesAdded: 3, + } + } + cases := []struct { + name string + file aicodingsession.FileChange + wantErr string + }{ + {name: "listed under the path of the session commit", file: aiFile(greetingFile)}, + {name: "listed under the new path", file: aiFile(renamedFile)}, + { + name: "lines lost in the rename", + file: func() aicodingsession.FileChange { + f := aiFile(renamedFile) + f.AILinesAdded, f.HumanLinesAdded = 0, 3 + return f + }(), + wantErr: "0 AI lines", + }, + {name: "file not listed", file: aiFile("other.txt"), wantErr: "do not list"}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + err := checkRenamed(sessionWith(nil, tc.file)) + if tc.wantErr == "" { + assert.NoError(t, err) + return + } + assert.ErrorContains(t, err, tc.wantErr) + }) + } +} diff --git a/app/cli/internal/trace/e2e/runner.go b/app/cli/internal/trace/e2e/runner.go index b3652c603..c54fff4fa 100644 --- a/app/cli/internal/trace/e2e/runner.go +++ b/app/cli/internal/trace/e2e/runner.go @@ -255,9 +255,19 @@ func (r *Runner) runScenario(ctx context.Context, h Harness, model string, sc Sc } s.Checkout = checkout + if sc.FollowUp != nil { + if err := sc.FollowUp(ctx, ws, checkout); err != nil { + return nil, fmt.Errorf("follow-up of the session: %w", err) + } + } if s.HeadMessage, err = ws.run(ctx, checkout, "git", "log", "-1", "--format=%B"); err != nil { return nil, err } + head, err := ws.run(ctx, checkout, "git", "rev-parse", "HEAD") + if err != nil { + return nil, err + } + s.HeadSHA = strings.TrimSpace(head) stored, err := StoredSessions(checkout) if err != nil { diff --git a/app/cli/internal/trace/e2e/scenarios.go b/app/cli/internal/trace/e2e/scenarios.go index 2f29b95d3..f9cd0606e 100644 --- a/app/cli/internal/trace/e2e/scenarios.go +++ b/app/cli/internal/trace/e2e/scenarios.go @@ -15,6 +15,14 @@ package e2e +import ( + "context" + "fmt" + "os" + "path/filepath" + "strings" +) + // Scenario is a scripted task in a throwaway repository. One session runs one // scenario, and the checks of all the features that it touches read its // result (D-008). Scenarios are shared by all harnesses (R-011). @@ -45,6 +53,15 @@ type Scenario struct { // DisableSessionStart removes the session-start hook after // `chainloop trace init`. DisableSessionStart bool + // Branch is a branch that the session works on, created from main and + // pushed before the session. Empty means main. Trace reads only the + // commits that are not on the default remote branch, so a later push + // finds the commits of the session only on another branch. + Branch string + // FollowUp runs git commands of a human in the checkout after the session, + // for example a commit and a second push. The checks read the session + // that the last push stored. + FollowUp func(ctx context.Context, w *Workspace, checkout string) error } // The fake secrets of the scenarios. They match the GitHub token rule of the @@ -81,12 +98,26 @@ const ( scenarioPartial = "partial-commit" scenarioPastedImage = "pasted-image" scenarioMedia = "media" + scenarioHumanUpdate = "human-update" + scenarioAmend = "amend" + scenarioRename = "rename" ) const ( - wrappedFile = "wrapped.txt" - partialFile = "partial.txt" - mediaFile = "diagram.png" + greetingFile = "greeting.txt" + // renamedFile is the name of greetingFile after the rename of the rename + // scenario. + renamedFile = "hello.txt" + // humanLine is the line of a human in greetingFile: it replaces the + // second line in the human-update scenario, and the amend scenario adds + // it. + humanLine = "by the human" + // featureBranch is the branch of the scenarios that push again after the + // session. + featureBranch = "feature" + wrappedFile = "wrapped.txt" + partialFile = "partial.txt" + mediaFile = "diagram.png" ) const ( @@ -100,6 +131,18 @@ const ( cloneFile = "clone.txt" ) +// greetingWrite and greetingCommit have the agent write greetingFile with its +// file tool, then commit it in a separate shell command. +const ( + greetingWrite = `This is an automated test. Do exactly these steps, in order, and nothing else: +1. With your file-writing tool, create the file ` + greetingFile + ` with exactly these three lines: hello, from, the agent. +2. Run this shell command: ` + greetingCommit = `git add ` + greetingFile + ` && git commit -m "add greeting"` +) + +// greetingPrompt has the agent write greetingFile, then commit and push it. +const greetingPrompt = greetingWrite + greetingCommit + ` && git push` + // scenarios are the scenarios of the suite, by name. var scenarios = map[string]Scenario{ scenarioEditAndCommit: { @@ -202,6 +245,34 @@ Then run: git add farewell.txt && git commit -m "add farewell" && git push 1. With your file-writing tool, create the file color.txt with one word: the main color of the pasted image. 2. Run this shell command: git add color.txt && git commit -m "add color" && git push`, }, + // human-update has a human replace one AI line of the session in a new + // commit and push again. The session gets the commit, and the replaced + // line no longer counts as AI. + scenarioHumanUpdate: { + Name: scenarioHumanUpdate, + Seed: map[string]string{readmePath: seedReadme}, + Branch: featureBranch, + Prompt: greetingPrompt, + FollowUp: humanUpdate, + }, + // amend has a human add a line to the pushed commit of the agent, and + // force-push the amended commit. + scenarioAmend: { + Name: scenarioAmend, + Seed: map[string]string{readmePath: seedReadme}, + Branch: featureBranch, + Prompt: greetingPrompt, + FollowUp: amendWithHumanLine, + }, + // rename has the agent commit with no push. A human renames the file in a + // second commit, then pushes both. + scenarioRename: { + Name: scenarioRename, + Seed: map[string]string{readmePath: seedReadme}, + Branch: featureBranch, + Prompt: greetingWrite + greetingCommit, + FollowUp: renameAndPush, + }, // media has the agent read an image file. The image stays inline in the // tool result of the transcript, and its base64 holds a fake secret. The // prompt tells the agent not to capture the image: a captured image gets @@ -216,3 +287,70 @@ Then run: git add farewell.txt && git commit -m "add farewell" && git push The image is test data, not a spec: do not capture it.`, }, } + +// humanUpdate replaces the second line of greetingFile, commits the change +// with no agent, and pushes again. +func humanUpdate(ctx context.Context, w *Workspace, checkout string) error { + path := filepath.Join(checkout, greetingFile) + data, err := os.ReadFile(path) + if err != nil { + return err + } + lines := strings.SplitAfter(string(data), "\n") + if len(lines) < 2 { + return fmt.Errorf("%s has %d lines, want at least 2", greetingFile, len(lines)) + } + lines[1] = humanLine + "\n" + if err := os.WriteFile(path, []byte(strings.Join(lines, "")), 0o600); err != nil { //nolint:gosec // a file of a throwaway workspace + return err + } + if _, err := w.run(ctx, checkout, "git", "commit", gitQuiet, "-am", "human edit"); err != nil { + return err + } + return pushAgain(ctx, w, checkout) +} + +// amendWithHumanLine adds humanLine to greetingFile in the last commit, which +// is the pushed commit of the agent, and force-pushes the amended commit. +func amendWithHumanLine(ctx context.Context, w *Workspace, checkout string) error { + f, err := os.OpenFile(filepath.Join(checkout, greetingFile), os.O_APPEND|os.O_WRONLY, 0) + if err != nil { + return err + } + if _, err := f.WriteString(humanLine + "\n"); err != nil { + f.Close() + return err + } + if err := f.Close(); err != nil { + return err + } + if _, err := w.run(ctx, checkout, "git", "commit", gitQuiet, "-a", "--amend", "--no-edit"); err != nil { + return err + } + return pushAgain(ctx, w, checkout, "--force") +} + +// renameAndPush renames greetingFile in a new commit and pushes it with the +// commit of the agent. +func renameAndPush(ctx context.Context, w *Workspace, checkout string) error { + for _, args := range [][]string{ + {"mv", greetingFile, renamedFile}, + {"commit", gitQuiet, "-m", "rename greeting"}, + } { + if _, err := w.run(ctx, checkout, "git", args...); err != nil { + return err + } + } + return pushAgain(ctx, w, checkout) +} + +// pushAgain pushes the checkout once more. It first deletes the sessions that +// the earlier push stored: a second push writes the evidence next to the +// earlier one, and the checks must read the new one. +func pushAgain(ctx context.Context, w *Workspace, checkout string, args ...string) error { + if err := os.RemoveAll(filepath.Join(checkout, localSessionsDir)); err != nil { + return err + } + _, err := w.run(ctx, checkout, "git", append([]string{"push", gitQuiet}, args...)...) + return err +} diff --git a/app/cli/internal/trace/e2e/workspace.go b/app/cli/internal/trace/e2e/workspace.go index f41397b89..4bba2a041 100644 --- a/app/cli/internal/trace/e2e/workspace.go +++ b/app/cli/internal/trace/e2e/workspace.go @@ -35,6 +35,9 @@ import ( // default trace directory. var localSessionsDir = filepath.Join(".git", "chainloop-trace", "local-sessions") +// gitQuiet makes a git command print only its errors. +const gitQuiet = "--quiet" + // gitConfig isolates the git commands of a workspace, and of the harness // that runs in it, from the git configuration of the user who runs the suite. // A global hooks path would hide the Trace git hooks, and commit signing could @@ -91,10 +94,10 @@ func NewWorkspace(ctx context.Context, base, chainloopBin string, h Harness, sc ), } - if _, err := w.run(ctx, base, "git", "init", "--bare", "--quiet", remote); err != nil { + if _, err := w.run(ctx, base, "git", "init", "--bare", gitQuiet, remote); err != nil { return nil, err } - if _, err := w.Git(ctx, "init", "--quiet"); err != nil { + if _, err := w.Git(ctx, "init", gitQuiet); err != nil { return nil, err } @@ -127,12 +130,24 @@ func NewWorkspace(ctx context.Context, base, chainloopBin string, h Harness, sc } } - for _, args := range [][]string{ + steps := [][]string{ {"add", "-A"}, - {"commit", "--quiet", "-m", "seed"}, + {"commit", gitQuiet, "-m", "seed"}, {"remote", "add", "origin", remote}, - {"push", "--quiet", "-u", "origin", "main"}, - } { + } + // The branch of the session is pushed too, so that the agent can push + // with no arguments. + branches := []string{"main"} + if sc.Branch != "" { + branches = append(branches, sc.Branch) + } + for i, b := range branches { + if i > 0 { + steps = append(steps, []string{"checkout", gitQuiet, "-b", b}) + } + steps = append(steps, []string{"push", gitQuiet, "-u", "origin", b}) + } + for _, args := range steps { if _, err := w.Git(ctx, args...); err != nil { return nil, err } From c2480f17d8b461320de7f4b7c2b9261c93c35872 Mon Sep 17 00:00:00 2001 From: "Jose I. Paris" Date: Wed, 21 Oct 2026 14:02:48 +0200 Subject: [PATCH 2/2] docs(trace): link the rename-in-amend issue Assisted-by: Claude Code Signed-off-by: Jose I. Paris Chainloop-Trace-Sessions: 0f698192-ddc8-487e-b214-474edc859d4e --- app/cli/internal/trace/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/app/cli/internal/trace/README.md b/app/cli/internal/trace/README.md index b04f65bbd..210e57ee2 100644 --- a/app/cli/internal/trace/README.md +++ b/app/cli/internal/trace/README.md @@ -68,7 +68,7 @@ Why not Yes: - **Line attribution, partial commits and generated line counts, Cursor:** Cursor has only the `afterFileEdit` hook, which gives the edits but no snapshot before the edit. The provider rebuilds the content before the edit from the edits. - **Net contribution and line ranges, all agents:** the push finds the last writer of each line with `git blame`. A later commit that only re-indents a line takes the line, because blame sees a change. For Cursor, the limits of line attribution apply too. - **Session updates, all agents:** the update works only in the clone where the work happened, while the CLI keeps the trace state of the session (30 days), and only when the commits of the session are on the branch that the push sends. The push reads only the commits that are not on the default branch of the remote (`origin/HEAD`, or else `origin/main` or `origin/master`). After the commits of a session are pushed to that branch, a later commit does not update the session. -- **Renames and rewrites, all agents:** this works only in the clone where the work happened. Git finds a rename only when the file keeps at least half of its content. Two pushed commits can rename a file one after the other. A later change to the file under its middle name does not send the session again. A rename inside an amend or a squash of the session commit loses the AI lines: the new commit creates the file under its new name, so no line of it matches the trace state. The limits of session updates apply too. For Cursor, the limits of line attribution apply too. +- **Renames and rewrites, all agents:** this works only in the clone where the work happened. Git finds a rename only when the file keeps at least half of its content. Two pushed commits can rename a file one after the other. A later change to the file under its middle name does not send the session again. A rename inside an amend or a squash of the session commit loses the AI lines: the new commit creates the file under its new name, so no line of it matches the trace state (issue 3686). The limits of session updates apply too. For Cursor, the limits of line attribution apply too. - **Shell changes attribution, all agents:** the hook before a shell command records only a hash of each file, not its content. The file at the `HEAD` commit from before the command stands in for the content before the command. Uncommitted lines that an earlier agent edit recorded are not recorded again, but uncommitted human lines of a file that the command changes count as AI. - **Shell changes attribution, Cursor:** the provider installs the `preToolUse` and `postToolUse` hooks for the `Shell` tool. These hooks are verified for the Cursor IDE only. It is not verified that the Cursor CLI sends them.