Skip to content

[agent-issue]: debug: candidate smoke integrity varies with checkout remote #1633

Description

@jamesadevine

Submission requirements

  • I generated this issue with an agent that used the repository's ado-aw guidance.
  • I reviewed the generated issue and confirm it is being filed directly in githubnext/ado-aw.

Diagnostic summary

The first live candidate-compiler smoke run generated lock YAML in a detached
worktree whose Git remote was GitHub, then executed that YAML from the Azure
Repos ado-aw-mirror checkout. The runtime integrity check recompiles the same
source under the mirror remote and produces different compiler metadata:

  • staged lock: org="", repo=""
  • runtime recompile: org="msazuresphere", repo="ado-aw-mirror"

The semantic pipeline is otherwise unchanged, but ado-aw check compares the
generated YAML byte-for-byte and fails.

Build details

  • Organization/project: msazuresphere/AgentPlayground
  • Parent orchestrator: build
    625952,
    definition 2559
  • Failing child: build
    625954,
    definition 2558 (Candidate compiler smoke - failure reporter)
  • Source branch: refs/heads/ado-aw-smoke-candidate/625952
  • Source version: 1701f04a98151722dab09ff1868360a45aecbaa9
  • Failed job/step: Agent / Verify pipeline integrity
  • Task record: e16f045e-3223-5d9a-19b7-d1e5a64bc357
  • Result: parent and failing child failed; the harness canceled children
    625953, 625955, 625956, and 625957.

Error evidence

- # ado-aw-metadata: {"org":"","repo":"","schema":1,"source":"tests/safe-outputs/smoke-failure-reporter.md","target":"standalone","version":"0.45.1"}
- echo 'ado-aw metadata: source=tests/safe-outputs/smoke-failure-reporter.md org= repo= version=0.45.1 target=standalone'
+ # ado-aw-metadata: {"org":"msazuresphere","repo":"ado-aw-mirror","schema":1,"source":"tests/safe-outputs/smoke-failure-reporter.md","target":"standalone","version":"0.45.1"}
+ echo 'ado-aw metadata: source=tests/safe-outputs/smoke-failure-reporter.md org=msazuresphere repo=ado-aw-mirror version=0.45.1 target=standalone'

The same context difference appears in aw_info.json.

Error: Integrity check failed: generated pipeline for
'ado-aw smoke failure reporter' does not match
tests/safe-outputs/smoke-failure-reporter.lock.yml.

Artifact acquisition and provenance validation succeeded before the failure:

  • Download candidate artifact for agentic pipeline compiler: succeeded
  • Stage candidate agentic pipeline compiler: succeeded
  • checkout of the exact candidate ref/SHA: succeeded

Root cause

CompileContext::new calls infer_ado_context from the compile directory's Git
remote (src/compile/extensions/mod.rs). AdoAwMarkerExtension then embeds the
inferred org/repo into the marker and aw_info.json
(src/compile/extensions/ado_aw_marker.rs).

The compiler-smoke harness compiles in a detached worktree created from the
GitHub checkout, so ADO context inference is empty. It subsequently pushes that
commit to ado-aw-mirror. The child pipeline checks out the mirror and runs
the integrity compilation there, where ADO remote parsing succeeds. Therefore
identical source and compiler bytes produce different lock YAML solely because
the checkout remote changed.

Impact

The candidate smoke lane cannot pass for any generated pipeline containing the
ADO marker while compilation and execution occur under different repository
remote identities. This blocks PR/main candidate-compiler validation but does
not affect release-smoke execution.

Proposed next step

Define and test a deterministic context contract for candidate compilation and
runtime integrity checking. The compile that stages the candidate lock and the
compile performed by ado-aw check must receive the same ADO org/repository
identity (or exclude environment-derived metadata from integrity-sensitive
output). Add an integration test that compiles under a GitHub checkout and
checks under an Azure Repos mirror so this cross-remote drift cannot recur.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions