Write a bundle manifest describing the bite artifacts - #131
Conversation
| /// Binaries that produce the snapshots, and whose versions therefore have to | ||
| /// match on restore. | ||
| const SNAPSHOT_BINARIES: [&str; 2] = ["doppelganger", "doppelganger-parachain"]; |
There was a problem hiding this comment.
This is not needed anymore, the binary we use for bite is not needed for spawning later.
There was a problem hiding this comment.
removed the version tracking entirely, spawn runs the plain binaries anyway so there was nothing to compare. Turns out doppelganger has no --version either
| /// `--version` of the binaries that produced the snapshots. | ||
| #[serde(default)] | ||
| pub binaries: Vec<(String, String)>, |
There was a problem hiding this comment.
We can remove this part :)
| /// `--version` of the binaries that produced the snapshots. | |
| #[serde(default)] | |
| pub binaries: Vec<(String, String)>, |
| async fn binary_version(cmd: &str) -> Option<String> { | ||
| let out = Command::new(cmd).arg("--version").output().await.ok()?; | ||
| let version = String::from_utf8_lossy(&out.stdout).trim().to_string(); | ||
| (!version.is_empty()).then_some(version) | ||
| } | ||
|
|
||
| pub async fn binary_versions() -> Vec<(String, String)> { | ||
| let mut versions = vec![]; | ||
| for cmd in SNAPSHOT_BINARIES { | ||
| if let Some(version) = binary_version(cmd).await { | ||
| versions.push((cmd.to_string(), version)); | ||
| } | ||
| } | ||
| versions | ||
| } | ||
|
|
| pub fn now_unix() -> u64 { | ||
| SystemTime::now() | ||
| .duration_since(UNIX_EPOCH) | ||
| .map(|d| d.as_secs()) | ||
| .unwrap_or_default() | ||
| } |
There was a problem hiding this comment.
This is weird, is a general helper function that is used from outside this module (and only onces). I think we can un in place or if we still need the helper make a support /helper module.
There was a problem hiding this comment.
inlined at the single call site
| /// Compare the binaries that produced the bundle with the ones on this machine. | ||
| /// A mismatch is a warning, not an error: it usually still restores, and when it | ||
| /// does not the failure otherwise looks like a corrupt snapshot. | ||
| /// | ||
| /// Both sides are the *doppelganger* binaries: the bite writes what produced the | ||
| /// snapshots, and a restore needs the same ones to import that state. | ||
| pub async fn warn_on_binary_mismatch(base_path: &Path) { | ||
| let Some(manifest) = Manifest::read(base_path).await else { | ||
| info!("no bundle manifest found, skipping the binary version check"); | ||
| return; | ||
| }; | ||
| if manifest.version != VERSION { | ||
| warn!( | ||
| "bundle manifest is version {} but this build writes {VERSION}; some fields may be missing", | ||
| manifest.version | ||
| ); | ||
| } | ||
| if manifest.binaries.is_empty() { | ||
| info!("bundle manifest records no binary versions, skipping the check"); | ||
| return; | ||
| } | ||
|
|
||
| let local = binary_versions().await; | ||
| for (cmd, bundled) in &manifest.binaries { | ||
| match local.iter().find(|(name, _)| name == cmd) { | ||
| Some((_, current)) if current == bundled => {} | ||
| Some((_, current)) => warn!( | ||
| "{cmd}: bundle was produced with '{bundled}' but this machine has '{current}'; a snapshot from a newer node can fail to restore in ways that look like corruption" | ||
| ), | ||
| None => warn!("{cmd}: not found locally, can't compare with the bundle's '{bundled}'"), | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
Again, we can remove this part.
pepoviola
left a comment
There was a problem hiding this comment.
Looks good, some small comments 🚀
* Publish the fork's own nodes as bootNodes, optionally under a public host * Pack a step's artifacts into one restorable bundle --------- Co-authored-by: Javier Viola <363911+pepoviola@users.noreply.github.com>
A bite writes
manifest.jsonnext toready.json: per chain the bite block, source RPC, spec/snapshot file names with sizes, any carried upgrade and its hash, and thedoppelganger --versionoutput that produced the snapshots.spawnwarns when the local binaries differ, since a snapshot from a newer node fails to restore in ways that look like corruption.Snapshot sizes are measured when the snapshot is written, because
ZOMBIE_BITE_CI_PATHmoves the files afterwards. Theversionandbundlefields say which bundle the manifest describes and let an older shape be reported instead of silently dropped.Part of #126 — the rest of that issue (specs, snapshots and overrides as one re-restorable artifact) is not in this PR.