Skip to content

Write a bundle manifest describing the bite artifacts - #131

Merged
pepoviola merged 7 commits into
mainfrom
feat/bundle-manifest
Aug 31, 2026
Merged

Write a bundle manifest describing the bite artifacts#131
pepoviola merged 7 commits into
mainfrom
feat/bundle-manifest

Conversation

@mordamax

@mordamax mordamax commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

A bite writes manifest.json next to ready.json: per chain the bite block, source RPC, spec/snapshot file names with sizes, any carried upgrade and its hash, and the doppelganger --version output that produced the snapshots. spawn warns when the local binaries differ, since a snapshot from a newer node fails to restore in ways that look like corruption.

Snapshot sizes are measured when the snapshot is written, because ZOMBIE_BITE_CI_PATH moves the files afterwards. The version and bundle fields say which bundle the manifest describes and let an older shape be reported instead of silently dropped.

Part of #126 — the rest of that issue (specs, snapshots and overrides as one re-restorable artifact) is not in this PR.

@mordamax
mordamax requested a review from pepoviola August 26, 2026 20:03
@mordamax
mordamax marked this pull request as ready for review August 26, 2026 20:03
Base automatically changed from feat/metadata-overrides to main August 28, 2026 20:43
Comment thread src/manifest.rs Outdated
Comment on lines +23 to +25
/// Binaries that produce the snapshots, and whose versions therefore have to
/// match on restore.
const SNAPSHOT_BINARIES: [&str; 2] = ["doppelganger", "doppelganger-parachain"];

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not needed anymore, the binary we use for bite is not needed for spawning later.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed the version tracking entirely, spawn runs the plain binaries anyway so there was nothing to compare. Turns out doppelganger has no --version either

Comment thread src/manifest.rs Outdated
Comment on lines +54 to +56
/// `--version` of the binaries that produced the snapshots.
#[serde(default)]
pub binaries: Vec<(String, String)>,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can remove this part :)

Suggested change
/// `--version` of the binaries that produced the snapshots.
#[serde(default)]
pub binaries: Vec<(String, String)>,

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed

Comment thread src/manifest.rs Outdated
Comment on lines +59 to +74
async fn binary_version(cmd: &str) -> Option<String> {
let out = Command::new(cmd).arg("--version").output().await.ok()?;
let version = String::from_utf8_lossy(&out.stdout).trim().to_string();
(!version.is_empty()).then_some(version)
}

pub async fn binary_versions() -> Vec<(String, String)> {
let mut versions = vec![];
for cmd in SNAPSHOT_BINARIES {
if let Some(version) = binary_version(cmd).await {
versions.push((cmd.to_string(), version));
}
}
versions
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same, don't need it

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed

Comment thread src/manifest.rs
Comment on lines +79 to +84
pub fn now_unix() -> u64 {
SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|d| d.as_secs())
.unwrap_or_default()
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is weird, is a general helper function that is used from outside this module (and only onces). I think we can un in place or if we still need the helper make a support /helper module.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

inlined at the single call site

Comment thread src/manifest.rs Outdated
Comment on lines +109 to +141
/// Compare the binaries that produced the bundle with the ones on this machine.
/// A mismatch is a warning, not an error: it usually still restores, and when it
/// does not the failure otherwise looks like a corrupt snapshot.
///
/// Both sides are the *doppelganger* binaries: the bite writes what produced the
/// snapshots, and a restore needs the same ones to import that state.
pub async fn warn_on_binary_mismatch(base_path: &Path) {
let Some(manifest) = Manifest::read(base_path).await else {
info!("no bundle manifest found, skipping the binary version check");
return;
};
if manifest.version != VERSION {
warn!(
"bundle manifest is version {} but this build writes {VERSION}; some fields may be missing",
manifest.version
);
}
if manifest.binaries.is_empty() {
info!("bundle manifest records no binary versions, skipping the check");
return;
}

let local = binary_versions().await;
for (cmd, bundled) in &manifest.binaries {
match local.iter().find(|(name, _)| name == cmd) {
Some((_, current)) if current == bundled => {}
Some((_, current)) => warn!(
"{cmd}: bundle was produced with '{bundled}' but this machine has '{current}'; a snapshot from a newer node can fail to restore in ways that look like corruption"
),
None => warn!("{cmd}: not found locally, can't compare with the bundle's '{bundled}'"),
}
}
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, we can remove this part.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed

@pepoviola pepoviola left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, some small comments 🚀

mordamax and others added 3 commits August 29, 2026 16:04
* Publish the fork's own nodes as bootNodes, optionally under a public host

* Pack a step's artifacts into one restorable bundle

---------

Co-authored-by: Javier Viola <363911+pepoviola@users.noreply.github.com>
@mordamax
mordamax requested a review from pepoviola August 30, 2026 17:16
@pepoviola
pepoviola merged commit 342f928 into main Aug 31, 2026
4 checks passed
@pepoviola
pepoviola deleted the feat/bundle-manifest branch August 31, 2026 10:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants