Skip to content

Migrate filesystem clients to broker - #1334

Open
Weidong Cui (wdcui) wants to merge 32 commits into
uliteboxfrom
wdcui/ulitebox/broker-file-clients
Open

Migrate filesystem clients to broker#1334
Weidong Cui (wdcui) wants to merge 32 commits into
uliteboxfrom
wdcui/ulitebox/broker-file-clients

Conversation

@wdcui

Copy link
Copy Markdown
Member

This PR moves LiteBox filesystem access behind the broker-owned file service by wiring file RPCs into the guest resolver, constructing authoritative Linux and Windows broker filesystems, requiring broker connections in production shims and runners, and migrating registry, loader, mmap, and test paths to the production broker transport while preserving existing filesystem behavior.

Weidong Cui (wdcui) and others added 9 commits September 8, 2026 14:40
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 239ec5f7-870a-4259-bcae-4ca85fb913a0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 239ec5f7-870a-4259-bcae-4ca85fb913a0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 239ec5f7-870a-4259-bcae-4ca85fb913a0
Filesystem resolution, backends, and the 9P client now live in
litebox_broker_core, so their tests move with them and exercise the
broker-core Resolver and backends directly. The ported suites no longer
build a BrokerCore, a host/local transport pair, shared memory, or guest
descriptors, which lets `litebox` drop its test-only broker harness and
the compatibility module paths that existed only for these tests.

- litebox_broker_core: add `fs::tests` (in-memory, tar, overlay, devices,
  and composed backends), `fs::nine_p::tests`, and the `fs::test_support`
  facade that pairs a resolver with session-shaped device I/O; share one
  `test_platform` between the fs and session tests.
- litebox: replace `fs::tests` with focused guest-facade tests that script
  broker responses over a local channel, covering context path resolution,
  value conversion, descriptor lifetime, and broker error mapping.
- Delete `litebox::test_broker` (and its `OnceLock<BrokerCore>`), the
  test-only `fs` re-export modules, and the now-unused dev-dependencies.
- Run litebox_broker_core on the AArch64 CI job so the 9P coverage that
  moved out of `litebox` still runs there.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 239ec5f7-870a-4259-bcae-4ca85fb913a0
Move runner loader coverage onto the production broker launcher and replace shim-wide BrokerCore fixtures with protocol-level guest tests. Windows registry defaults are now seeded lazily so constructing unrelated shim tests performs no file requests, while explicit file integration tests own their broker setup without process-global state.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 239ec5f7-870a-4259-bcae-4ca85fb913a0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 239ec5f7-870a-4259-bcae-4ca85fb913a0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 239ec5f7-870a-4259-bcae-4ca85fb913a0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 239ec5f7-870a-4259-bcae-4ca85fb913a0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 239ec5f7-870a-4259-bcae-4ca85fb913a0
@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown

🤖 SemverChecks 🤖 ⚠️ Potential breaking API changes detected ⚠️

Click for details
--- failure enum_missing: pub enum removed or renamed ---

Description:
A publicly-visible enum cannot be imported by its prior path. A `pub use` may have been removed, or the enum itself may have been renamed or removed entirely.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.50.0/src/lints/enum_missing.ron

Failed in:
  enum litebox::fs::FileType, previously in file /home/runner/work/litebox/litebox/target/semver-checks/git-ulitebox/a813a7f2b26a346df12a5496b298b75bc3edb3c9/litebox/src/fs/mod.rs:158

--- failure module_missing: pub module removed or renamed ---

Description:
A publicly-visible module cannot be imported by its prior path. A `pub use` may have been removed, or the module may have been renamed, removed, or made non-public.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.50.0/src/lints/module_missing.ron

Failed in:
  mod litebox::fs::resolver, previously in file /home/runner/work/litebox/litebox/target/semver-checks/git-ulitebox/a813a7f2b26a346df12a5496b298b75bc3edb3c9/litebox/src/fs/resolver.rs:4

--- failure struct_missing: pub struct removed or renamed ---

Description:
A publicly-visible struct cannot be imported by its prior path. A `pub use` may have been removed, or the struct itself may have been renamed or removed entirely.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.50.0/src/lints/struct_missing.ron

Failed in:
  struct litebox::fs::resolver::Resolver, previously in file /home/runner/work/litebox/litebox/target/semver-checks/git-ulitebox/a813a7f2b26a346df12a5496b298b75bc3edb3c9/litebox/src/fs/resolver.rs:25
  struct litebox::fs::resolver::Context, previously in file /home/runner/work/litebox/litebox/target/semver-checks/git-ulitebox/a813a7f2b26a346df12a5496b298b75bc3edb3c9/litebox/src/fs/resolver.rs:73
  struct litebox::fs::resolver::ResolvedPath, previously in file /home/runner/work/litebox/litebox/target/semver-checks/git-ulitebox/a813a7f2b26a346df12a5496b298b75bc3edb3c9/litebox/src/fs/resolver.rs:151

--- failure enum_missing: pub enum removed or renamed ---

Description:
A publicly-visible enum cannot be imported by its prior path. A `pub use` may have been removed, or the enum itself may have been renamed or removed entirely.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.50.0/src/lints/enum_missing.ron

Failed in:
  enum litebox_broker_core::fs::FileType, previously in file /home/runner/work/litebox/litebox/target/semver-checks/git-ulitebox/a813a7f2b26a346df12a5496b298b75bc3edb3c9/litebox_broker_core/src/fs/mod.rs:79

--- failure auto_trait_impl_removed: auto trait no longer implemented ---

Description:
A public type has stopped implementing one or more auto traits. This can break downstream code that depends on the traits being implemented.
        ref: https://doc.rust-lang.org/reference/special-types-and-traits.html#auto-traits
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.50.0/src/lints/auto_trait_impl_removed.ron

Failed in:
  type BrokerCoreBuilder is no longer UnwindSafe, in /home/runner/work/litebox/litebox/litebox_broker_userland/src/builder.rs:77
  type BrokerCoreBuilder is no longer RefUnwindSafe, in /home/runner/work/litebox/litebox/litebox_broker_userland/src/builder.rs:77

--- failure constructible_struct_adds_field: struct exhaustively constructible through public API adds field ---

Description:
A pub struct that could be exhaustively constructed with a literal using only public API has a new pub field, breaking existing exhaustive literals.
        ref: https://doc.rust-lang.org/reference/expressions/struct-expr.html
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.50.0/src/lints/constructible_struct_adds_field.ron

Failed in:
  field CliArgs.broker_control_channel in /home/runner/work/litebox/litebox/litebox_runner_windows_on_linux_userland/src/lib.rs:45

--- failure inherent_method_missing: pub method removed or renamed ---

Description:
A publicly-visible method or associated fn is no longer available under its prior name. It may have been renamed or removed entirely.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.50.0/src/lints/inherent_method_missing.ron

Failed in:
  LinuxShimBuilder::default_fs, previously in file /home/runner/work/litebox/litebox/target/semver-checks/git-ulitebox/a813a7f2b26a346df12a5496b298b75bc3edb3c9/litebox_shim_linux/src/lib.rs:228

--- failure method_parameter_count_changed: pub method parameter count changed ---

Description:
A publicly-visible method now takes a different number of parameters, not counting the receiver (self) parameter.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#fn-change-arity
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.50.0/src/lints/method_parameter_count_changed.ron

Failed in:
  litebox_shim_linux::LinuxShim::load_program takes 5 parameters in /home/runner/work/litebox/litebox/target/semver-checks/git-ulitebox/a813a7f2b26a346df12a5496b298b75bc3edb3c9/litebox_shim_linux/src/lib.rs:265, but now takes 4 parameters in /home/runner/work/litebox/litebox/litebox_shim_linux/src/lib.rs:250

--- failure inherent_method_missing: pub method removed or renamed ---

Description:
A publicly-visible method or associated fn is no longer available under its prior name. It may have been renamed or removed entirely.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.50.0/src/lints/inherent_method_missing.ron

Failed in:
  WindowsShimBuilder::new, previously in file /home/runner/work/litebox/litebox/target/semver-checks/git-ulitebox/a813a7f2b26a346df12a5496b298b75bc3edb3c9/litebox_shim_windows/src/lib.rs:406
  WindowsShimBuilder::default_fs, previously in file /home/runner/work/litebox/litebox/target/semver-checks/git-ulitebox/a813a7f2b26a346df12a5496b298b75bc3edb3c9/litebox_shim_windows/src/lib.rs:423

--- failure method_parameter_count_changed: pub method parameter count changed ---

Description:
A publicly-visible method now takes a different number of parameters, not counting the receiver (self) parameter.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#fn-change-arity
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.50.0/src/lints/method_parameter_count_changed.ron

Failed in:
  litebox_shim_windows::WindowsShim::load_program takes 4 parameters in /home/runner/work/litebox/litebox/target/semver-checks/git-ulitebox/a813a7f2b26a346df12a5496b298b75bc3edb3c9/litebox_shim_windows/src/lib.rs:507, but now takes 3 parameters in /home/runner/work/litebox/litebox/litebox_shim_windows/src/lib.rs:488

Weidong Cui (wdcui) and others added 20 commits September 10, 2026 09:18
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 239ec5f7-870a-4259-bcae-4ca85fb913a0
Use protocol file status, directory entries, and node identities directly across LiteBox, broker core, and shims. Preserve nonzero device identities and narrow metadata only at ABI and memory boundaries.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 239ec5f7-870a-4259-bcae-4ca85fb913a0
Remove the separate File marker and make FileFd non-generic. Update Linux and Windows consumers while preserving Arc ownership and descriptor lifetime behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 239ec5f7-870a-4259-bcae-4ca85fb913a0
Keep raw Linux open flags at the ABI boundary and use protocol access modes and flags directly throughout LiteBox and broker backends. Consolidate lexical path normalization in the protocol and remove duplicate definitions and intermediate conversions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 239ec5f7-870a-4259-bcae-4ca85fb913a0
Remove the obsolete guest 9P transport and broker-core dependency, drop dead file backing APIs, and simplify Windows file and registry integration.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 239ec5f7-870a-4259-bcae-4ca85fb913a0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 239ec5f7-870a-4259-bcae-4ca85fb913a0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 239ec5f7-870a-4259-bcae-4ca85fb913a0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 239ec5f7-870a-4259-bcae-4ca85fb913a0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 239ec5f7-870a-4259-bcae-4ca85fb913a0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 239ec5f7-870a-4259-bcae-4ca85fb913a0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 239ec5f7-870a-4259-bcae-4ca85fb913a0
Re-export canonical protocol filesystem types under the existing broker-core names to minimize migration-only churn.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 239ec5f7-870a-4259-bcae-4ca85fb913a0
Keep canonical filesystem metadata aligned with the existing broker-core field names to avoid migration-only churn.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 239ec5f7-870a-4259-bcae-4ca85fb913a0
Keep the existing qualified type references where imports would add migration-only churn.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 239ec5f7-870a-4259-bcae-4ca85fb913a0
Convert canonical file size once and retain the existing usize-based resolver offset calculation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 239ec5f7-870a-4259-bcae-4ca85fb913a0
Remove the test-only filesystem facade and have inherited filesystem and 9P tests call broker-core Resolver APIs directly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 239ec5f7-870a-4259-bcae-4ca85fb913a0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 239ec5f7-870a-4259-bcae-4ca85fb913a0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 239ec5f7-870a-4259-bcae-4ca85fb913a0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 239ec5f7-870a-4259-bcae-4ca85fb913a0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 239ec5f7-870a-4259-bcae-4ca85fb913a0
Weidong Cui (wdcui) and others added 3 commits September 11, 2026 16:04
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 239ec5f7-870a-4259-bcae-4ca85fb913a0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 239ec5f7-870a-4259-bcae-4ca85fb913a0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 239ec5f7-870a-4259-bcae-4ca85fb913a0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant