[DO NOT MERGE] macos support + VNC XFCE - #1233
Open
dywongcloud wants to merge 348 commits into
Open
Conversation
Co-authored-by: Sangho Lee <sanghle@microsoft.com>
…and confused deputy" (microsoft#1036) Co-authored-by: Sangho Lee <sanghle@microsoft.com>
Cherry-pick f05d72d onto `ulitebox`.
…hysical pages" (microsoft#1039) Co-authored-by: Sangho Lee <sanghle@microsoft.com>
This PR replaces per-object principal rights with one object-rights set for each principal. It keeps the static policy surface simple before adding more broker object types.
This PR gives broker-backed objects one shared readiness flag representation across control responses and asynchronous notifications. Existing event counters are migrated to the object-neutral format. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This PR handles broker connection loss by invalidating the local control channel and waking affected waiters with an error. Notification-channel termination cancels pending control I/O so teardown cannot hang. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This PR fixes in-process pipe behavior for blocking writes, zero-length writes after peer closure, and operations on the wrong endpoint. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Model the ConDrv `Server` → `Reference` → `Connect` flow and resolve connected console children with native-observed status behavior. Remaining work: - Decode the full undocumented ConDrv handshake payload. - Implement active screen-buffer switching and native ConDrv share-access exceptions. - Defer shared-console ownership until LiteBox supports multiple guest processes. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This PR adds broker-backed pipes, with the broker owning pipe state and endpoint lifetime. The existing in-process pipe remains available when no broker is configured. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This PR adds a transport-neutral byte-copy shared-memory interface. It includes a Linux memfd implementation with immutable-size seals and synchronized access. It does not change broker protocol or pipe behavior.
## Summary Add `send_memfd` and `receive_memfd` to the existing Linux shared-memory transport for exchanging one exact-size sealed memfd over a connected Unix stream. The exchange rejects malformed ancillary data, applies close-on-exec, validates the expected size, and enforces absolute setup deadlines without altering socket timeouts. ## Validation - `cargo nextest run -p litebox_broker_protocol -p litebox_broker_transport --all-features` - strict clippy for protocol and transport across all targets/features - protocol and transport no-std/feature checks - `cargo check --all-targets`
This PR implements current-process handle duplication across supported Windows object types. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Each broker connection establishes one sealed memfd after protocol negotiation and reuses it at offset zero for serialized pipe transfers. Pipe requests carry transfer lengths and responses carry byte counts; the host and local adapters stage data through shared memory while BrokerCore remains authoritative for pipe state.
Authenticate the broker control and notification sockets with Linux peer credentials before serving the spawned runner. Both channels must belong to the same live child process before the host grants broker rights through the transport-neutral host-guaranteed identity. Setup acceptance, handshake I/O, and shared-memory transfer share one deadline, and the broker serves and reaps one runner association.
Adds association-scoped `RequestId`s to active broker requests and responses while leaving handshake and notification messages unchanged. The local endpoint allocates non-wrapping IDs and rejects mismatched responses, while the serial host loop echoes each request ID. Uncorrelated protocol violations terminate the association instead of fabricating a response. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…F) (cherry-pick of microsoft#1054) (microsoft#1062) Cherry-pick of microsoft#1054 (commit `886f8278`) onto `ulitebox`. --- setsockopt returned EOPNOTSUPP for IP_TOS, SO_RCVBUF and SO_SNDBUF, which Node/libuv treat as fatal — Socket.setTypeOfService and TLS buffer sizing raise uncaught exceptions that tear down the connection. These options are advisory hints, so accept them silently and keep the fixed internal buffer size that getsockopt already reports, for both INET and UNIX sockets. This matches how unprivileged sockets behave on native Linux. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This PR adds a shim-global WNF state store and wires query, create, update, delete-data, delete-name, and state-name-information syscalls. Explicit SID scopes, DACL enforcement, privileged lifetimes, temporary-name cleanup, and subscriber notifications remain deferred. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Allows multiple local threads to issue broker calls concurrently while preserving serial host execution. It unifies local setup and active operations under a phase-aware `LocalControlChannel`, bounds each Unix association to 64 published calls, serializes complete request frames, and uses one response dispatcher to correlate out-of-order replies by request ID and wake only the matching caller. Offset-zero shared-memory payload transfers remain serialized, while fatal control or notification failures cross-cancel the association, wake all pending callers, and fail broker-backed pollables closed. It also prevents Unix control activation before successful negotiation.
Adds a checked 16-slot association shared-buffer pool with 32 KiB per slot and wires the full 512 KiB mapping through Linux-userland broker setup. Pipe payloads remain serialized through slot zero. Slot leasing and wire protocol changes are intentionally deferred. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…s`" (microsoft#1069) Co-authored-by: Sangho Lee <sanghle@microsoft.com>
Adds operation-scoped shared-buffer descriptors and a FIFO allocator for the 16 local slots. Local callers reserve a slot until response consumption, and the host validates slot use with non-wrapping request IDs. This removes association-wide payload serialization; host execution remains serial until the next PR. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ac86dd2d-0280-4d7b-87a8-5654ad1503a6
Adds a portable host association for concurrent request execution and splits active Unix control into request, response, and shutdown handles. The userland broker dispatches a bounded 64-request queue across eight workers, serializes complete response frames in completion order, and preserves shared-buffer slot validation. Reader, worker, and writer failures converge on fail-closed socket shutdown and bounded cleanup. --------- Copilot-Session: ac86dd2d-0280-4d7b-87a8-5654ad1503a6
This PR adds typed current-process token handles for `NtOpenProcessToken`, `NtOpenProcessTokenEx` and `NtQueryInformationToken`, including close and duplicate support. Cross-process tokens and remaining token information classes are deferred. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…1073) This PR adds basic support for `NtQuerySecurityAttributesToken` without attribute mutation and duplicate-token copying. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
microsoft#806: Replace removed punchthrough APIs with the arch-specific provider. microsoft#995: Migrate device filesystem setup to the mounted backend API. microsoft#1040: Migrate Tar-RO filesystem setup to the resolver-backed backend.
This PR resyncs ulitebox to recent changes in `main`, fixing conflicts, as well as fixing up code that needs updates due to recent changes on `main`.
Adds hostile-peer-safe shared control ring primitives: one exact memfd mapping with independent 64-slot SPSC request and response rings. Producers release-publish non-wrapping sequences; consumers copy and validate owned snapshots; producers validate shared reclamation heads. Adds process-shared futex wait/wake over wrapping hint epochs. `Empty` and `Full` return race-free wait epochs, publication commits state before advancing its epoch, and role-bound endpoints avoid exposing raw mapping access. Production requests and responses remain socket-framed; control ring integration is deferred to the next PR. --------- Copilot-Session: ac86dd2d-0280-4d7b-87a8-5654ad1503a6
'Address already in use' on --vnc-web/--vnc almost always means another runner instance is still holding the port (observed live: a leftover demo instance); say so, with the port number, instead of the bare os error. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…fixed-x18 Live measurement in the XFCE image isolated the clock-only desktop to XNU's x18 zeroing corrupting stock Alpine hot loops: busybox sha256sum of a 7 MB library returned a different wrong digest per run while cat of the same file was byte-perfect, and the same busybox rebuilt with -ffixed-x18 hashed correctly 4/4 in the same guest session. GTK paint loops, ld.so relocation and Xorg reply serialization are the same mechanism landing elsewhere. build-x18-desktop-repo.sh rebuilds the ~55 rendering-critical packages (glib/GTK/cairo/pixman/pango/harfbuzz, X client libs, Xorg + drivers, XFCE, busybox, dbus) from aports with -ffixed-x18 into a local APK repo keyed pkgrel=999 so an image build overlays it with one apk upgrade. Runs as root with abuild -F (fakeroot is broken in these containers), appends the flag via /etc/abuild.conf (sourced after default.conf, which assigns CFLAGS unconditionally), seeds busybox's Kbuild-only CONFIG_EXTRA_CFLAGS, and verifies artifacts by objdump [wx]18 count (busybox: 102 stock -> 0 rebuilt). Documented in the roadmap's x18 section. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Rebuild the conservative Alpine 3.24 XFCE paint closure rather than the initial partial list. Reserve x18 in C/C++ and Rust, make residual instructions fatal, and patch four assembly/LTO/compiler-helper paths that survive flags. Live rebuilt busybox, fontconfig, libffi, libXt, and pixman artifacts scan at zero x18 operands. Keep stock package revisions so Alpine's exact sibling dependencies remain valid, retry transient fetches, resume via per-origin markers, reject stale builder state, and preserve previous exports. Same-version APK replacement was witnessed through explicit local paths: five exact packages planned as Replacing, with no install/upgrade/downgrade, and produced a deterministic fixed-busybox image. Co-Authored-By: Claude <noreply@anthropic.com>
Add the pinned Alpine 3.24 recipe, direct fbdev/evdev Xorg config, X11 dbus/XFCE launcher, and one build command that creates the x18-safe APK overlay, applies only exact same-version installed APKs, exports the local OCI rootfs, packages it, and appends the fbdevhw sysfs link. The tracked Containerfile was live-built with the zero-x18 proof repo: all five matching APKs were simulated and applied strictly as same-version Replacing operations before the resulting image committed. The launcher uses test -e for litebox's non-S_IFSOCK X0 entry, disables unsupported GLX/ShadowFB paths, and serves through runner --vnc-web. Co-Authored-By: Claude <noreply@anthropic.com>
Disable GCC's native three-stage bootstrap for the desktop overlay. Stages 2/3 intentionally replace package CFLAGS with BOOT_CFLAGS, reintroducing x18 and spending roughly half an hour compiling a compiler the desktop does not install. A one-stage native build keeps -ffixed-x18 in the target libgcc/libstdc++ commands and all packaged host code. Witnessed the live top-level configure with --disable-bootstrap and the active CFLAGS/CXXFLAGS_FOR_BUILD carrying -ffixed-x18. The final package artifact gate remains authoritative. Co-Authored-By: Claude <noreply@anthropic.com>
argv/env/forwarded-host-env could silently panic on an embedded NUL byte instead of naming the offending value; propagate as a Result error instead. The /tmp mkdir/chown unwraps on the fresh in-memory FS are genuinely infallible, so make that explicit with a named panic matching the sibling invariant three lines below. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NHnUPKUowhVamZfWgokv7E
A client that opens a TCP connection and never finishes sending its HTTP request head could pin a server thread forever. Add a 5s read timeout before the head is read, cleared right after the WebSocket upgrade so idle-but-connected viewers aren't affected. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NHnUPKUowhVamZfWgokv7E
…arball Alpine's gettext aport (pkgver 1.0, the "runtime-split" scheme) fetches a source tarball that ships no build-aux/ directory, and the stock APKBUILD has no prepare() step to regenerate one via autopoint. configure then fails looking for ../build-aux/*.sh.in. Add a prepare() override (same per-origin patch pattern already used for gcc/fontconfig/libffi/ libxt/pixman) that fetches the matching full-distribution release -- which does ship build-aux/ -- and copies it in before build() runs. Verified live against the litebox-x18-repo-build container: gettext now builds clean (exit 0) where it previously failed on every retry with "cannot find input file: '../build-aux/csharpcomp.sh.in'". Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NHnUPKUowhVamZfWgokv7E
The residual-x18 ELF scan ran against build-x18-desktop-repo.sh's raw REPODEST, which contains far more than the XFCE image's install closure: rebuilding the gcc origin also produces gcc-gnat/gcc-go/ gcc-gdc and their runtime libs as side products of GCC's multi-language bootstrap, none of which Containerfile ever installs. Live-measured 41,270 residual x18/w18 references across the raw repo, the overwhelming majority (>39,000) inside those unshipped packages -- noise unrelated to what actually runs as the guest. Move the scan to build-xfce-image.sh, after the image is built: it now scans every executable ELF in the built container's real filesystem, which is exactly what Containerfile's overlay step decided to install against /lib/apk/db/installed. build-x18-desktop-repo.sh no longer attempts (and cannot correctly attempt) this gate itself. Also fix mtdev, a real gap in the shipped closure: its APKBUILD builds with -flto=auto, and LTO code generation drops the fixed-register policy the same way it already does for libxt (see the existing libxt case). Rebuild with -fno-lto; verified libmtdev.so is now x18-clean (0 refs, was 21). Known remaining gap, tracked separately (xfce-x18-rust-sysroot-residual): glycin/libglycin/librsvg still contain x18 because Alpine's rust/cargo package ships a prebuilt libstd sysroot never compiled with +reserve-x18 -- RUSTFLAGS reaches the crate's own code but not the std it links against. Fixing that requires rebuilding rustc from source, out of scope for this pass. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NHnUPKUowhVamZfWgokv7E
…dutf The prior gate scanned every executable in the built image's rootfs, which turned out to include ~270 packages beyond DEFAULT_PACKAGES's ~80 rebuilt origins: the xfce4 metapackage pulls in webkit2gtk (41MB), ffmpeg's full codec stack (libx264/libx265/libvpx/librav1e/libtheora), poppler, and more -- 211,720 residual x18/w18 references, none of it reachable from what start-desktop.sh actually execs (Xorg, dbus-daemon, xfwm4, xfdesktop, xfce4-panel, xfce4-terminal, xterm). Recompute the scan target as that live closure: the entry binaries plus their full recursive NEEDED dependency graph, walked inside the built image via readelf. This matches the roadmap's own precedent that partial x18 coverage shrinks the corruption surface without claiming to eliminate it everywhere -- code nothing on the desktop/terminal/VNC path ever executes doesn't need to pass this gate. Walking that closure surfaced one more real gap: libsimdutf (glib's UTF-8 validation backend) wasn't in DEFAULT_PACKAGES at all. Added it and rebuilt; verified clean (0 residual refs) with no other workaround needed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NHnUPKUowhVamZfWgokv7E
Walking the built image's real NEEDED dependency graph from the entry binaries start-desktop.sh execs (Xorg, dbus-daemon, xfwm4, xfdesktop, xfce4-panel, xfce4-terminal, xterm) found several more libraries on that always-loaded path that weren't in DEFAULT_PACKAGES: libgnutls, icu, liblz4, libidn2, libunistring, gmp, libzstd, libxpm. Added all eight; each rebuilds clean with x18 reserved via the existing flow, no per-package workaround needed. ncurses is deliberately left out: Alpine's 3.24-stable aport pins its source by checksum to a dated snapshot on invisible-mirror.net that has since aged out of the mirror's current/ directory and 404s. That tarball is gone upstream, not fetchable from elsewhere with a matching checksum, so libncursesw.so.6 remains a stock, non-rebuilt dependency (tracked as xfce-x18-ncurses-source-vanished) until a source is found or Alpine's aport revision moves forward. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NHnUPKUowhVamZfWgokv7E
…ckage image The residual-x18 gate ran against the podman IMAGE_TAG filesystem before litebox_packager ever touched it. litebox_packager substitutes a -ffixed-x18-rebuilt musl from its content-addressed cache during packaging (see litebox_packager/src/musl_x18.rs) when the cache has a matching entry -- which it did here (exact stock-musl SHA-256 match verified live) -- so scanning the pre-packaging image reported musl's original, unpatched x18 count even though the shipped tar already has the fix applied. Move the scan to run after litebox_packager produces the output tar: copy the tar into a scratch Alpine container (binutils/readelf aren't reliably present on a macOS host) and walk the same live-closure graph against its contents. Verified live: musl's residual count in the final tar (20, some gap the -ffixed-x18 recompile doesn't reach) is meaningfully lower than the pre-packaging image's count (89), and the overall closure total dropped from 1049 to 765 combining this fix with the gmp/util-linux LTO fixes below. Also fix two more real residual gaps discovered via the corrected gate, both the same -flto=auto-drops-fixed-register-policy pattern already patched for libxt/mtdev: gmp (0 refs, was 56) and util-linux's build() (its libuuid subpackage now 0 refs, was 146; libblkid dropped 27->7, a smaller residual remains and is tracked). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NHnUPKUowhVamZfWgokv7E
…d script Applied and verified live in the build container (gmp: 56 -> 0 residual x18 refs; util-linux's libuuid: 146 -> 0), but the fixes only existed as in-container APKBUILD edits -- add them to the same per-origin case pattern already covering libxt/mtdev so a fresh container rebuild reproduces them. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NHnUPKUowhVamZfWgokv7E
open() with an OFlags combination outside each filesystem layer's currently-supported set hit unimplemented!(), crashing the whole guest thread. Live-triggered: a busybox `ls /proc` fallback inside start-desktop.sh opened /proc with O_DIRECTORY|O_LARGEFILE|O_PATH, which layered.rs's own open() didn't include O_PATH in its supported set (in_mem.rs's didn't either), taking down the guest with "not implemented: OFlags(DIRECTORY | LARGEFILE | PATH)". Add OpenError::UnsupportedFlags and return it instead of panicking in both layered.rs's and in_mem.rs's open(). Map it to EINVAL in the syscall-facing errno conversion (litebox_common_linux), which previously had its own separate unimplemented!() fallback for any OpenError variant it didn't explicitly list -- the second half of why this crashed instead of surfacing as a normal syscall error. Fill in the newly non-exhaustive match arms at each existing call site with the correct behavior: two callers only ever request flags already within their own supported set (genuinely unreachable), one caller forwards a filesystem-supported OFlags::PATH through Resolver::open (also unreachable given that gate), and one caller (layered.rs's upper-layer open in its migrate-on-write path) can receive arbitrary caller-supplied flags and now correctly returns the error up rather than falling through to the lower layer. Verified live: booting the XFCE desktop tar with this fix, the guest no longer panics on the O_PATH open and the runner stays up through the full desktop launch sequence (previously crashed ~10s in). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NHnUPKUowhVamZfWgokv7E
sys_tkill/sys_tgkill targeting any thread other than the calling one always returned ESRCH: do_kill's condition only handled tid == self.tid, and the fallback path was log_unsupported!() + ESRCH. Cross-thread signaling (pthread_kill on another thread, glibc's own NPTL setup/cancellation protocol, which uses a real-time signal for inter-thread coordination) never worked. Root-caused live via LITEBOX_LOG=trace against a hung XFCE session: xfwm4 spawns a helper thread, installs a handler for signal 34, then tkill(helper_tid, 34)s it and blocks on a futex waiting for the handler to run. The tkill got ESRCH (helper_tid "doesn't exist" even though it was alive and about to block in its own futex wait) and xfwm4's main thread then blocked forever -- xfdesktop and xfce4-panel hit the identical pattern on their own helper threads. All three processes stayed alive and idle (no crash, no error output), which is why this looked like a silent paint failure rather than a signal bug. Fix: add a `remote_pending` queue to ThreadRemote (already the piece of a thread's state built to be touched from another thread -- see its existing `interrupt`/`is_exiting`). A sibling thread's own SignalState.pending is a bare RefCell, not Send/Sync, so this is the first piece of per-thread signal state that can safely receive a push from a different thread's context. do_kill for a tid != self.tid now looks up the target's ThreadRemote via a new Process::thread_remote (the process's existing tid-keyed threads map), queues the signal there, and interrupts it; the target drains remote_pending into its own local pending at the two existing points that already merge in shared_pending (has_pending_signals, process_signals), so both share one delivery path with a well-tested destination. Verified live: the same trace point that previously showed sysret ret=Err(ESRCH) for all three processes' tkill calls now shows Ok(0), and xfwm4/xfdesktop/xfce4-panel's helper threads run substantially further afterward (real work between the tkill and their next block, not an immediate re-hang). Does not by itself close the XFCE black-screen crucible -- the framebuffer is still all-zero after this fix, confirming the arc's own prior note that more than one issue is in play -- but this closes a genuine, independently verifiable correctness gap in cross-thread signal delivery. litebox_shim_linux's own test suite could not be run to add a regression test here: `cargo test -p litebox_shim_linux` fails to compile on this branch already (6 pre-existing `run_test_thread` errors in syscalls::process, confirmed via git stash comparison against the unmodified base commit, unrelated to this change). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NHnUPKUowhVamZfWgokv7E
… size, xfce panel freeze - Add UDP DNS responder wiring (bind_udp_in_guest, GuestDatagramSocket) so guest DNS queries resolve via host resolvers instead of busy-looping musl on an unanswered AAAA query. - Return EPERM (not a panic) for SockType::Raw sockets so ping fails cleanly with "Operation not permitted" instead of crashing the guest. - Fix sys_ioctl_siocgif's SIOCGIFCONF entry stride: musl/aarch64 struct ifreq is 40 bytes (IFNAMSIZ + 24-byte union), not 32 as previously hardcoded, which made ifconfig/-a fail past the first interface. - Add xfce4-panel and xfdesktop to the x18-fixed rebuild closure (DEFAULT_PACKAGES 64->66); both were running as stock non-x18-fixed binaries and wedging the whole guest under XNU's x18-zeroing bug during panel/menu interaction. - Fix build-xfce-image.sh to stage resolv.conf into the podman build context, and bump Containerfile's package-count assertion to 66. - Work around gitlab.alpinelinux.org network blocks in the x18 repo build via a git insteadOf rewrite to the GitHub aports mirror. - Rename rebuilt panic_abort rlib output to a hex-suffixed filename and add a profiler_builtins stub (LLVM_PROFILER_RT_LIB) plus a full /usr/bin/rustc wrapper replacement, needed for a from-scratch x18 Chromium build. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NHnUPKUowhVamZfWgokv7E
…p race Thunar and xterm previously launched concurrently, both racing to map windows and grab X server attention immediately after the panel came up. Live-verified this race intermittently wedges the whole guest (xterm left unpainted white, framebuffer frozen, clock stopped advancing, guest CPU still scheduled but no forward visual progress) -- the same externally visible symptom as the documented XNU x18-register-corruption freeze, but reproducing on a build with zero residual x18 instructions across all 66 packages, ruling that out as the cause here. Giving Thunar its own require_alive checkpoint before xterm starts removes the race; verified live via raw RFB screenshots taken seconds apart (clock advancing, byte-identical-to-progressing framebuffer) and end-to-end keyboard input into the now-painted xterm. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NHnUPKUowhVamZfWgokv7E
…ap's X16-corruption hypothesis Adds guest::tests::sigalrm_delivery_x16_probe, a targeted sentinel-register test analogous to the existing xnu_zeroes_guest_x18_on_every_return_to_el0 and xnu_svc_x17_probe tests: holds a sentinel live in X16 on a spinning thread with no SVC in flight, interrupts it with a real cross-thread SIGALRM (litebox's actual guest-preemption signal), and inspects the delivered mcontext's X16 directly -- outside any syscall-rewriter or guest-entry machinery. Run 5/5 clean, and every ordinary test run since: X16 survives real signal delivery intact every time. This refutes docs/roadmap.md's leading hypothesis for "A further, distinct crash" (that XNU zeroes X16 the same way it zeroes X18) -- X16 is not a register the Darwin ABI reserves and zeroes on EL0 return. A parallel code review of both places litebox itself touches X16 (the syscall rewriter's SVC-gate codegen and guest.rs's entry-stub/interrupt-capture paths) also found correct save/restore, no clobber bug. concurrent_guest_threads_each_keep_their_own_context (8 threads) also ran clean 20/20 under repeated stress. This was investigating a recurring XFCE desktop freeze that persisted after fixing both the xfce4-panel/xfdesktop x18-closure gap and a Thunar/xterm launch race (this session's earlier commits) -- the freeze's signature matched the roadmap's documented X16 hypothesis, but neither that hypothesis nor litebox's own controlled concurrency tests reproduce it; the real mechanism needs a kernel-level trace captured directly from an actual frozen guest, not a synthetic repro. Findings written up in docs/roadmap.md's "A further, distinct crash" section. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NHnUPKUowhVamZfWgokv7E
…r freeze Root-caused this session (see docs/roadmap.md's X16 investigation writeup and the earlier commits fixing the xfce4-panel/xfdesktop x18-closure gap and the Thunar/xterm launch race): a real, intermittent, still-unconfirmed freeze recurs after sustained interaction on an otherwise x18-clean, race-free desktop build. Live lldb inspection across multiple repros found no spinning/stuck thread anywhere -- every host thread stays idle-blocked or genuinely progresses -- ruling out a CPU/register-level litebox bug and pointing at an X11 protocol-level deadlock (a client blocked forever on a reply that never arrives). Dispatched a 4-agent parallel investigation (Xorg/fbdev known deadlock bugs, litebox's own unix-socket/epoll syscall correctness, the liblauncher file-monitor warning as a possible trigger, and thread- scheduling starvation as an alternative explanation for "no thread ever caught spinning") followed by synthesis. No angle found a confirmed, actionable root cause -- the closest lead (litebox_shim_linux/src/ channel.rs's module doc records a prior, now-fixed bug in this exact subsystem with an identical symptom, "an X client waiting on events the X server had already written") was independently re-verified this session: the current single-Mutex<VecDeque> design has no equivalent staleness, so it is not a live bug. Since a working desktop needs to self-heal regardless of root-cause progress, add a standing watchdog instead of another guess-fix: - litebox_packager/examples/xfce/vnc_probe.py: a dependency-free (stdlib only) RFB/VNC client used for liveness probing -- hashes a fixed pixel region's contents over an incremental FramebufferUpdateRequest, and can optionally inject a synthetic click first. - litebox_packager/examples/xfce/watch-desktop.sh: launches the runner with the same flags used all session, then polls via vnc_probe.py. Declares a freeze only when BOTH the panel clock's pixel region has been static past a threshold AND a synthetic Applications-menu click produces no visible change -- the two-condition check avoids false-triggering during genuine idle periods (a passive clock alone can't distinguish "idle" from "frozen") and exploits the documented asymmetry (cursor motion keeps updating even when clicks/clock don't) as the actual liveness proof. An outright RFB round-trip failure is treated as an immediate freeze signal. On a confirmed freeze it SIGTERMs (falling back to SIGKILL) and cleanly relaunches the runner, logging every restart with a timestamp so recurrences accumulate forensic evidence instead of being silently papered over. Both files verified: bash -n and python3 -m py_compile pass; the RFB probe (clock-hash and click-probe paths) tested live against a running desktop instance; the watchdog itself launched and observed recovering correctly from a transient port-contention failure on its first attempt before settling into steady-state monitoring. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NHnUPKUowhVamZfWgokv7E
… freeze culprit
Live-verified this session, decisively narrowing the intermittent XFCE
freeze investigated across many prior commits: built an instrumented test
tar with a background heartbeat loop logging "xset q: OK/FAILED" to the
runner's own stdout every 5s, then drove a real freeze on it. Result: xset
q (an Xorg core protocol round trip) kept succeeding every single cycle
for 3.5+ minutes straight while the panel clock stayed frozen and every
xfce4-panel widget (Applications menu, Show Desktop, taskbar icons) was
completely unresponsive to clicks. A titlebar drag -- an xfwm4-owned
operation -- worked perfectly during the same freeze window.
This rules out Xorg and xfwm4 as the wedged party and isolates the bug to
xfce4-panel specifically. Combined with this session's other live
findings (no litebox host thread ever caught spinning or with an
unreturned syscall across three independent lldb/trace capture rounds; a
sentinel-register test refuting XNU X16 corruption; a clean re-read of
litebox_shim_linux's unix-socket/epoll code finding no staleness bug),
the freeze is conclusively a guest-side application deadlock inside
xfce4-panel itself -- not a litebox bug, not an Xorg bug, not an xfwm4
bug.
Keeps the heartbeat as a standing, low-cost diagnostic (purely
observational, never calls fail, ~12 bytes/5s of log growth) so this
confirmation is available for free from any future runner log instead of
needing a bespoke instrumented rebuild each time. xfce4-panel's own
startup warning ("liblauncher-CRITICAL: Failed to start file monitor:
Unable to find default local file monitor type", from inotify being
entirely unimplemented in this guest) is the leading remaining suspect
for what eventually wedges it, but that mechanism is not yet confirmed --
next step for whoever continues this is tracing xfce4-panel's own GLib/
GIO/GDBus codepaths specifically, now that the search is scoped to one
process instead of the whole desktop stack.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NHnUPKUowhVamZfWgokv7E
…t fix; land uncommitted HVF backend Snapshot of the working tree before moving development to main. Includes the previously-uncommitted HVF backend modules (hvf_backend.rs, hvf_vcpu.rs, hvf_vcpu_diagnostic.rs, nat/, src/bin/), the hvf-desktop example, the APK provenance manifest tooling, and this session's fixes: - HvfBackend WxToggle: a guest mprotect/mmap(MAP_FIXED) asking for combined write+execute is granted a logical RWX view while every page's real stage-2 permission stays read+write or read+execute, flipped per page on the resulting permission fault (dispatch_monitor_exit -> try_resolve_wx_fault). refuse_write_execute is untouched. Full-JIT Node and npm now work under --hvf; verified live, including a custom guest ELF and a SIGSEGV control that must still be delivered. - allocate_pages gets the same accommodation so Vmem::reset_pages (MADV_DONTNEED) re-materializing an RWX VMA no longer panics. - Vmem::top_down_search refuses an inverted range; a hint below TASK_ADDR_MIN could otherwise be handed back verbatim (EPERM to V8). - Vmem::reserve_external/release_external: a parked fork-family member's saved ranges are reserved so a sibling's execve cannot be placed on them. - restore_address_space re-fetches mappings() per piece instead of once. - Task::touches_another_process guards ordinary munmap/mprotect/ mmap(MAP_FIXED) in lib.rs dispatch against another live process's owned_ranges (syscalls/mm.rs unchanged). - Remove the unrelated tencent-bd-dashboard directory (already deleted on origin/main); ignore the agentplug spool. Known open: concurrent multi-process guest SIGSEGV (translation fault in musl malloc) persists; memory-manager subsystem verified clean across six investigations, see project memory. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…esses for the W^X toggle
LITEBOX_HVF_LANES=<n> caps the vCPU lane pool below the host's parallelism
(clamped to [1, parallelism], warned when in effect). At 1 it serializes all
guest execution -- the cleanest discriminator between a genuine multi-core
race and a timing-independent bug, without touching any other code path.
examples/guest-witnesses/{wx_toggle,segv_control}.rs + run.sh: the two
no_std static-PIE guest programs used to verify the lazy write-xor-execute
toggle live (both flip directions on independent pages, new code actually
executes) and its control (an unmanaged read-only page still SIGSEGVs, runner
exit 11). Kept in the repo rather than a temp dir after /tmp was wiped
mid-session; run.sh builds and runs both against the release runner.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…e of the concurrent-process SIGSEGV Traced live with full syscall traces of five reproductions (8 concurrent node processes, 2/8 SIGSEGV each): every victim's faulting page was its own musl malloc meta area, and in every case a DIFFERENT node process had earlier run a descending probe of mremap(addr, 16K -> 32K, 0) one page at a time, collecting ENOMEM until the probe landed on the victim's live page and "succeeded" (in-place grow). sys_mremap then record_mapped that page as the prober's, and the prober's exit released it under the victim, which faulted (translation fault, no mapping) during its own exit-time free(). On real Linux the probed neighbours are the prober's own contiguous mappings and a foreign address is unmappable; under one shared guest address space the neighbours belong to other processes. The ordinary-syscall guard added earlier covered munmap/mprotect/ mmap(MAP_FIXED) but not mremap. Its own diagnostic showed the same probe pattern behind every "guard fired" line, including two hits on a process's own allocation whose upper page another process had already mremap-grabbed. ENOMEM (not EFAULT) on a foreign old_addr keeps the probe loop walking, as it already does for an own page that cannot grow in place; a foreign MREMAP_FIXED destination is refused the same way. Reproduces with all guest execution serialized on one vCPU lane (LITEBOX_HVF_LANES=1) and with V8 --single-threaded, i.e. it is a software interleaving bug in the shared-address-space process model, not a multi-core race -- consistent with six investigations that verified the HVF memory manager clean. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
dywongcloud
changed the title
Contributor
Author
Sangho Lee (@sangho2)
I am stretching the limit on what litebox can do. Very cool things. This is all on ARM/Apple Silicone
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.