feat: rework manager api for cross-major calls - #8
Open
kp2pml30 wants to merge 20 commits into
Open
Conversation
The host protocol is the same interface for every executor line, but each line carried its own copy of host-fns.json and its own generated host_fns.rs. The definition now lives once in the umbrella modules-interfaces crate and generates once; common re-exports it, so genvm_common::host_fns call sites are unchanged.
Case collection globs **/*.jsonnet and skips names starting with an underscore, a prefix that marks a parameterized template imported by siblings. These three are plain test definitions, not templates, so the prefix meant they had never run once.
ExecutionData carries host_hello_data indexed by host connection index, the same index space method_hosts points into. On connecting host i the executor writes entry i verbatim -- raw bytes, no length prefix, before the first method byte -- so the host, which supplied them, knows their length and can frame them however it likes. A missing, short or empty entry writes nothing, making an empty vector byte-identical to the previous behaviour. notify_finished goes away: the manager now reports completion, exit code and result to the host, so a dedicated end-of-work method has no job left and the loop ends on EOF. Its ACK round-trip was, incidentally, what forced the buffered host writers to flush before exit. Every host connection is therefore now flushed explicitly before exiting, on both the normal and the early setup-failure path; without that the result would die in the buffer.
Mutating the environment of a live process races every getenv in it, including the ones inside libc and third-party code. The umbrella has rejected this for a while; this tree, which owns the surviving startup snapshot, did not.
Wasm compilation is unusable when cranelift is built unoptimized: a profile of `genvm precompile` attributes ~80% of its samples to the crates raised to opt-level 2 here, ~46% of it in `cranelift-codegen` and `regalloc2` alone — neither of which the previous `wasmtime` / `wasmparser` overrides covered. genvm's own code stays at opt-level 0. Generic code (hashbrown, smallvec, core) is monomorphized into whichever crate instantiates it, so optimizing these covers it too. Debug assertions stay on: switching them off in the cranelift crates buys ~4% of `precompile`, which is not worth silencing third-party checks for.
On every `CallContract` the executor now asks host 0 who should run the callee, before the local major check. A null answer keeps today's path unchanged, `invalid_contract major_mismatch` included; a payload means another line runs it, so the executor builds a nested-run envelope, sends it to the manager, folds the returned execution hash into `det_subvm_hashes` and hands the result to the contract. Cross-major *loading* stays forbidden — only the call path changes. Version detection belongs to the host: the root-slot major byte is contract controlled on older roots, so it travels as advisory data only. * feat(host): add `resolve_callcontract_executor` and `run_nested` 🌐 * refactor(sdk): share the `SingleVMData` derivation between both call paths ♻️ * feat(exe): accept a nested run envelope as the entrypoint ✨ * feat(rt): honour the envelope's memory limit, fuel budget and depth 🔒 * test(sdk): cover delegation, effect-freedom and the unreachable re-check ✅ The caller requires the reply to be effect-free with a 32-byte execution hash before folding that hash in — the cross-process stand-in for the in-process accumulator-empty assertion. `granted_custom` deliberately does not cross the boundary, and every boundary failure maps to the internal error for now.
`VM_RECURSION` is a per-line constant, so an absolute depth compared against it makes the effective bound whichever line happens to be executing. The VM now carries the recursion left to it, minted by the chain root and decremented per spawn, so a chain that crosses executor lines keeps the bound its root set and no line has to agree with any other on the number. This also matches `remaining_det_fuel`, which was already a remainder on the same envelope. Recorded actions and logs keep reporting an absolute depth, derived from the budget, so their shape does not change.
A crash mid-write left a truncated artifact at the final path, and the loader feeds whatever sits there straight to deserialize_file without validating it.
verify_runner accepts any single-colon id, so a custom:<hash> runner an attacker registers derived a precompile path and reached the unsafe deserialize_file. Only ids listed in all.json can have artifacts written for them, so nothing else may look one up.
The second guard re-tested det_mod, so a missing non-det artifact fell through to deserialize_file and failed there instead of falling back to a compile.
Every executor line compiles the one `crates/modules-interfaces/src`, so the field could never disagree with itself: a change to the envelope is a compile error at each construction site in the same build.
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Move the `float_math` probe out of the agent scratch tree into
`nasty-determinism/floats`, where its neighbours live, and report the
results as `struct.pack('<d', x)` bytes instead of a `repr()` string.
`repr()` rounds, so it hides exactly the last-bit divergence the probe
exists to catch; it also cannot distinguish `-0.0` from `0.0` or carry a
NaN payload, both of which the case now covers.
- delete `run-manager.sh` (it passed a `--reroute-to` flag the manager does not have), `run_test.py` and `AGENT_PROMPT.md`; the procedure now lives in the manager's `agentic-fuzzing` skill and the runner is `genvm-tool test run --filter-name` - probes become throwaway: they are written to a gitignored `_scratch/`, and the one worth keeping is promoted by hand instead of accumulating as a test of nothing in particular - refresh the intelligence notes, whose harness section described a toolchain that no longer exists
The topmost runner id hardcoded the accepted chain state, so a run reading finalized storage resolved its code slot through finalized data but loaded the archive at the latest non-final state. The id is also the archive cache key, so both halves now follow the imported storage mode. Passes a literal false for is_init: the deploy state has no host storage type and only the inline-code branch seeds a deploy cell, so deriving it here would turn deploy-from-storage into a malformed runner error.
The fee expression language has no boolean literals, so the `else false` arm of the budget-floor check referenced an unbound identifier. Every non-zero budget takes the other arm, which is why nothing caught it; a balance-funded message declaring a zero per-round budget aborted the whole transaction with an internal error instead of being metered. Spelled as a comparison that never holds until the language grows real boolean literals.
The language produced booleans from comparisons and `hasKey` but had no way to write one, so a constant false had to be spelled as a comparison that never holds. Writing the obvious `false` instead resolved as an identifier and failed at evaluation time. `true` and `false` are reserved like the other keywords, so they cannot be rebound or used as bare attribute names; the quoted form still works for keys.
Replaces the comparison-that-never-holds workaround now that the expression language has literals.
A guest picks the `name:hash` pair passed to spawn_sandbox and map_file. A well formed pair that is simply absent from the registry fell through to a plain anyhow error, so the transaction aborted with an internal error where the neighbouring format and gvm32 failures already produced `invalid_contract malformed_runner`. An absent `custom:` runner reported it too, so the pair shape was the odd one out. Adds runtime coverage for all three shapes, plus the zero-budget message fee case that motivated the fee expression fix.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Auto-opened executor mirror of genlayerlabs/genvm-manager#9.
Carries the executor-side work for that manager PR. Auto-closed as merged when the manager PR lands (its
pr/v0.3/feat/rework-manager-apibranch is moved ontov0.3-dev).