From 5891a9b7e5cd05f8d436e0027b833cf3013a6f04 Mon Sep 17 00:00:00 2001 From: Shane Michael Mathews Date: Tue, 7 Jul 2026 16:16:29 -0400 Subject: [PATCH 01/15] feat: N=1 cooperative fiber scheduler replacing per-thread host threads Replace the per-guest-thread std::thread model (g_hostThreads) with an N=1 cooperative fiber scheduler: exactly one host executor thread runs all guest EE threads as fibers, highest priority first, FIFO within a priority, with cooperative yield points sampled every 128 back-edges. This matches the real EE's single-core execution model that games' kernel-object usage assumes (semaphores as ownership tokens, priority scheduling, RotateThreadReadyQueue rotation) and removes the cross-thread races and lock-order deadlocks of the previous model. Blocking syscalls park through a publish -> arm_park -> block_current protocol whose wake_pending handshake cannot lose wakeups; all wait sites re-check their predicates in Mesa-style loops. Host workers (IRQ, vsync, alarm, RPC) either wake fibers through gated entry points (with {generation,tid} tokens rejecting stale wakeups after tid reuse) or borrow the guest token via AsyncGuestScope. Host threads carrying a guest tid get full ThreadInfo wait bookkeeping (wait lists, ReleaseWaitThread targeting) but never park on the fiber scheduler; they poll with bounded backoff. Four fiber backends: ucontext (POSIX default), Win32 Fibers (_WIN32), SceFiber (PLATFORM_VITA; compiles against a real VitaSDK, untested on hardware, no guard page - documented), and pthread (PS2X_FIBER_PTHREAD=ON, exists so ThreadSanitizer can instrument the scheduler). PS2X_SANITIZE wires TSan/ASan through every target from the top-level CMakeLists. Preserves upstream's guest-visible semantics (#136 sid-on-success semaphore returns throughout, including older tests). Bugs found and fixed while hardening: lost ReleaseWaitThread during Mesa retries; alarm-worker use-after-free vs CancelAlarm; non-fiber WaitEventFlag and WaitForNextVSyncTick returning without waiting; INTC/DMAC enable-mask races; a terminate or resume wake dropped in the publish->park window; a tid-recycling ABA in join_fiber; main-thread guest identity (tid 1) restored to upstream semantics. Dogfooding against a real commercial title found more: executor/worker token-handoff starvation, fixed across three layers (the executor's defer-to-parked-worker predicate, yield_point's fiber-side handoff, and a dispatchLoop back-edge hook so cross-function spins reach both); sceSifRpcLoop's stub returning instead of parking, which monopolized the executor; async callback stacks carved from top-of-RAM inside the guest's own main stack, relocated to kernel-reserved memory (0x80000-0x100000); requestStop joining workers from guest context (deadlock - now signal-only); DMAC dispatch borrowing the guest token from guest context (fatal abort - now dispatches inline); per-OS-thread thread_local state shared across fibers under N=1 - the dispatch-history ring, the syscall-override reentrancy guard, and the RPC-invoke/inline- DMAC/MPEG-callback scratch stacks - all now fiber-owned or per-invocation; pthread-backend fiber threads not marked as guest context, self-deadlocking the is_guest_thread gates; and join_fiber flooring the joiner one priority level below the target (t->priority + 1) instead of at the target's own level, so an equal-priority sibling of the target kept the joiner off the run-queue head and TerminateThread never returned (DQ8: main thread terminating a same-priority worker while a sibling stayed runnable). EE priority is "lower number = higher"; flooring at the target's own level lets the joiner tie the siblings and cycle back via FIFO rotation to observe the target finish and return. Tests: passing across ucontext and pthread backends (deterministic across repeated runs), TSan-clean (zero warnings) with the three documented intentional hardware-modeled vsync/GS-CSR reports suppressed via ps2xTest/tsan.supp, ASan/LSan clean. Scheduler suites cover the wakeup-window protocol, suspend/resume gating, tid reuse, terminate windows, priority rotation and join floors (including a SchedulerJoinStarvation regression: a higher-priority joiner terminating a target with an equal-priority runnable sibling, asserting TerminateThread returns within a bounded number of rounds), shutdown ordering, and borrowed workers, with regression tests that fail against the unfixed code. The workload dogfood regression suite (ps2_scheduler_workload_regression_tests.cpp) adds targeted coverage for the bugs above: token handoff, RPC-loop park, recovery isolation, guest-context stop, kernel stack pool, DMAC guest dispatch, stack isolation, and override isolation. --- .gitignore | 2 +- CMakeLists.txt | 15 + README.md | 1 + ps2xRuntime/CMakeLists.txt | 44 +- ps2xRuntime/Readme.md | 12 + ps2xRuntime/include/ps2_dispatch_history.h | 15 + ps2xRuntime/include/ps2_runtime.h | 116 +- ps2xRuntime/include/ps2_scheduler.h | 192 + .../include/ps2_syscall_override_state.h | 22 + ps2xRuntime/include/ps2_syscalls.h | 2 - ps2xRuntime/src/lib/Kernel/Stubs/GS.cpp | 9 +- ps2xRuntime/src/lib/Kernel/Stubs/IPU.cpp | 1 - ps2xRuntime/src/lib/Kernel/Stubs/MPEG.cpp | 30 +- ps2xRuntime/src/lib/Kernel/Stubs/SIF.cpp | 15 + ps2xRuntime/src/lib/Kernel/Syscalls/Common.h | 9 +- .../src/lib/Kernel/Syscalls/Helpers/Runtime.h | 289 +- .../src/lib/Kernel/Syscalls/Helpers/State.h | 141 +- .../lib/Kernel/Syscalls/Helpers/ThreadExit.h | 11 + .../src/lib/Kernel/Syscalls/Interrupt.cpp | 484 +- .../src/lib/Kernel/Syscalls/Interrupt.h | 30 +- .../src/lib/Kernel/Syscalls/Lifecycle.cpp | 81 +- .../src/lib/Kernel/Syscalls/Lifecycle.h | 2 - ps2xRuntime/src/lib/Kernel/Syscalls/RPC.cpp | 14 +- ps2xRuntime/src/lib/Kernel/Syscalls/Sync.cpp | 793 +- ps2xRuntime/src/lib/Kernel/Syscalls/Sync.h | 14 + .../src/lib/Kernel/Syscalls/System.cpp | 13 +- .../src/lib/Kernel/Syscalls/Thread.cpp | 676 +- ps2xRuntime/src/lib/ps2_debug_panel.cpp | 1 - ps2xRuntime/src/lib/ps2_fiber.cpp | 675 ++ ps2xRuntime/src/lib/ps2_fiber.h | 49 + ps2xRuntime/src/lib/ps2_runtime.cpp | 354 +- ps2xRuntime/src/lib/ps2_scheduler.cpp | 1169 +++ ps2xRuntime/src/lib/ps2_scheduler_internal.h | 216 + ps2xTest/CMakeLists.txt | 2 + ps2xTest/include/SchedTestSupport.h | 75 + ps2xTest/src/main.cpp | 60 + ps2xTest/src/ps2_gs_tests.cpp | 58 +- ps2xTest/src/ps2_runtime_expansion_tests.cpp | 6704 ++++++++++++++++- ps2xTest/src/ps2_runtime_kernel_tests.cpp | 73 +- ...s2_scheduler_workload_regression_tests.cpp | 1635 ++++ ps2xTest/tsan.supp | 13 + 41 files changed, 12469 insertions(+), 1648 deletions(-) create mode 100644 ps2xRuntime/include/ps2_dispatch_history.h create mode 100644 ps2xRuntime/include/ps2_scheduler.h create mode 100644 ps2xRuntime/include/ps2_syscall_override_state.h create mode 100644 ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/ThreadExit.h create mode 100644 ps2xRuntime/src/lib/ps2_fiber.cpp create mode 100644 ps2xRuntime/src/lib/ps2_fiber.h create mode 100644 ps2xRuntime/src/lib/ps2_scheduler.cpp create mode 100644 ps2xRuntime/src/lib/ps2_scheduler_internal.h create mode 100644 ps2xTest/include/SchedTestSupport.h create mode 100644 ps2xTest/src/ps2_scheduler_workload_regression_tests.cpp create mode 100644 ps2xTest/tsan.supp diff --git a/.gitignore b/.gitignore index 17aa25b4a..5931959d3 100644 --- a/.gitignore +++ b/.gitignore @@ -17,4 +17,4 @@ ps2xRuntime/include/ps2_recompiled_functions.h ps2xRuntime/include/ps2_recompiled_stubs.h ps2xRuntime/src/runner/ps2_recompiled_functions.cpp ps2xRuntime/src/runner/register_functions.cpp -ps2xRuntime/output \ No newline at end of file +ps2xRuntime/outputbuild-*/ diff --git a/CMakeLists.txt b/CMakeLists.txt index abe5bf5f1..0932b6397 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -19,6 +19,21 @@ option(PS2X_BUILD_ANALYZER "Build ps2xAnalyzer" ON) option(PS2X_BUILD_TEST "Build ps2xTest" ON) option(PS2X_BUILD_STUDIO "Build ps2xStudio" ON) +# Sanitizer flags must apply to EVERY target that links ps2_runtime (including +# ps2x_tests in a sibling directory), so they are set here at the root rather +# than inside ps2xRuntime/. PS2X_SANITIZE=thread additionally requires +# PS2X_FIBER_PTHREAD=ON, validated in ps2xRuntime/CMakeLists.txt. +set(PS2X_SANITIZE "" CACHE STRING "Sanitizer to enable: thread, address, or empty") +set_property(CACHE PS2X_SANITIZE PROPERTY STRINGS "" thread address) +if(PS2X_SANITIZE) + if(MSVC) + add_compile_options(/fsanitize=${PS2X_SANITIZE}) + else() + add_compile_options(-fsanitize=${PS2X_SANITIZE} -fno-omit-frame-pointer -g) + add_link_options(-fsanitize=${PS2X_SANITIZE}) + endif() +endif() + set(PS2X_IS_ARM_TARGET OFF) set(PS2X_IS_AARCH64_TARGET OFF) if(CMAKE_SYSTEM_PROCESSOR MATCHES "arm64|aarch64|ARM64") diff --git a/README.md b/README.md index b9d3ac840..04793ce47 100644 --- a/README.md +++ b/README.md @@ -131,6 +131,7 @@ To execute the recompiled code. * Guest memory model and function dispatch table. * Some syscall dispatcher with common kernel IDs. * Basic GS/VU/file/system stubs. +* Multi-platform cooperative fiber scheduler. * Foundation to expand and port your game. ### Game Override Hooks diff --git a/ps2xRuntime/CMakeLists.txt b/ps2xRuntime/CMakeLists.txt index 43dc4167b..f2e87978c 100644 --- a/ps2xRuntime/CMakeLists.txt +++ b/ps2xRuntime/CMakeLists.txt @@ -1,6 +1,6 @@ cmake_minimum_required(VERSION 3.20) -project(PS2Runtime VERSION 0.1.0 LANGUAGES CXX) +project(PS2Runtime VERSION 0.1.0 LANGUAGES C CXX) set(CMAKE_CXX_STANDARD 20) set(CMAKE_CXX_STANDARD_REQUIRED ON) @@ -8,6 +8,10 @@ set(CMAKE_CXX_STANDARD_REQUIRED ON) option(PS2X_ENABLE_RUNNER_UNITY_BUILD "Build ps2EntryRunner with CMake unity build" ON) set(PS2X_RUNNER_UNITY_BUILD_BATCH_SIZE 8 CACHE STRING "Unity build batch size for ps2EntryRunner") option(PS2X_ENABLE_SCCACHE "Use sccache as compiler launcher when available" ON) +option(PS2X_FIBER_PTHREAD "Build the pthread fiber backend on a non-Vita host (for TSan/ASan testing)" OFF) +# PS2X_SANITIZE is declared and applied in the top-level CMakeLists.txt so the +# flags reach every target that links ps2_runtime (ps2x_tests lives in a +# sibling directory). Only the backend-compatibility validation lives here. option(PS2X_ENABLE_RUNTIME_LOGS "Enable PS2 runtime logs" OFF) option(PS2X_ENABLE_AGRESSIVE_LOGS "Enable very verbose/agressive PS2 runtime logs" OFF) @@ -114,6 +118,10 @@ else() target_link_libraries(rlImGui PUBLIC raylib imgui) endif() +if(PS2X_SANITIZE STREQUAL "thread" AND NOT PS2X_FIBER_PTHREAD) + message(FATAL_ERROR "PS2X_SANITIZE=thread requires PS2X_FIBER_PTHREAD=ON; the ucontext backend is not TSan-compatible because TSan does not instrument swapcontext.") +endif() + add_library(ps2_host_backend INTERFACE) set(PS2X_DEFAULT_BOOT_ELF_DEFINE "") @@ -357,6 +365,7 @@ add_library(ps2_runtime STATIC src/lib/ps2_memory.cpp src/lib/ps2_pad.cpp src/lib/ps2_runtime.cpp + src/lib/ps2_scheduler.cpp src/lib/ps2_vif1_interpreter.cpp src/lib/ps2_vu1.cpp src/lib/games_database.cpp @@ -388,6 +397,12 @@ target_sources(ps2_runtime PRIVATE ${KERNEL_SRC_FILES} ) +# Fiber backend (ucontext on POSIX; SceFiber on Vita; Win32 Fibers on Windows; +# pthread under PS2X_FIBER_PTHREAD for Linux TSan testing) +target_sources(ps2_runtime PRIVATE + src/lib/ps2_fiber.cpp +) + file(GLOB RUNNER_SRC_FILES CONFIGURE_DEPENDS "${CMAKE_CURRENT_SOURCE_DIR}/src/runner/*.cpp" ) @@ -427,7 +442,34 @@ target_include_directories(ps2_runtime PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/src/lib/Kernel ) +# Private include dir so Kernel/Syscalls/*.cpp can reach ps2_scheduler_internal.h +# and ps2_fiber.h without relative path hacks. +target_include_directories(ps2_runtime PRIVATE + ${CMAKE_CURRENT_SOURCE_DIR}/src/lib +) + target_link_libraries(ps2_runtime PUBLIC ps2_host_backend ffmpeg) + +# ps2_scheduler.cpp and ps2_fiber.cpp use std::thread/std::mutex/std::atomic +# which require pthread on Linux. Link explicitly so the dependency is not +# implicit via raylib (ps2_host_backend). The Vita toolchain provides its own +# pthread; the ucontext default build on Linux also needs it. +if(NOT PS2X_IS_VITA) + find_package(Threads REQUIRED) + target_link_libraries(ps2_runtime PUBLIC Threads::Threads) +endif() + +if(PS2X_FIBER_PTHREAD) + if(PS2X_IS_VITA) + message(FATAL_ERROR "PS2X_FIBER_PTHREAD is a non-Vita test flag; do not combine with the Vita toolchain") + endif() + if(WIN32) + message(FATAL_ERROR "PS2X_FIBER_PTHREAD requires POSIX /; Windows uses the native Win32 Fibers backend instead") + endif() + target_compile_definitions(ps2_runtime PUBLIC PS2X_FIBER_PTHREAD) + # Threads::Threads already linked above for all non-Vita builds. +endif() + target_link_libraries(ps2EntryRunner PRIVATE ps2_runtime diff --git a/ps2xRuntime/Readme.md b/ps2xRuntime/Readme.md index 4ae035fef..dbaa909bd 100644 --- a/ps2xRuntime/Readme.md +++ b/ps2xRuntime/Readme.md @@ -50,6 +50,18 @@ PS2-specific 128-bit MMI instructions and VU0 macro mode instructions are suppor ## Instruction Patching You can patch specific instructions in the recompiled code to fix game issues or implement custom behavior. +## Threading model + +Guest threads run as cooperative fibers on a single host executor thread +(matching the real EE's single-core scheduling), not on concurrent host +threads. See `ps2xRuntime/include/ps2_scheduler.h` for the API and design notes. + +The fiber backend is selected by platform: ucontext (Linux/macOS, default), +Win32 Fibers (`_WIN32`), or SceFiber (`PLATFORM_VITA`). Build with +`-DPS2X_FIBER_PTHREAD=ON` for the pthread backend required by ThreadSanitizer, +and `-DPS2X_SANITIZE=thread|address` to enable sanitizers on targets that link +`ps2_runtime` (`thread` requires the pthread backend). + ## Limitations * Graphics and sound output require external implementations diff --git a/ps2xRuntime/include/ps2_dispatch_history.h b/ps2xRuntime/include/ps2_dispatch_history.h new file mode 100644 index 000000000..25b8038b9 --- /dev/null +++ b/ps2xRuntime/include/ps2_dispatch_history.h @@ -0,0 +1,15 @@ +#ifndef PS2_DISPATCH_HISTORY_H +#define PS2_DISPATCH_HISTORY_H +#include +#include + +// Fixed-size ring of recently dispatched guest PCs, used only for diagnostic +// "trace=..." output. Owned per-fiber by FiberContext (fresh per fiber, gone at +// teardown); host workers / non-fiber callers use a per-OS-thread fallback. +struct DispatchHistory +{ + std::array pcs{}; + uint32_t next = 0u; + bool wrapped = false; +}; +#endif // PS2_DISPATCH_HISTORY_H diff --git a/ps2xRuntime/include/ps2_runtime.h b/ps2xRuntime/include/ps2_runtime.h index b58ddb841..7d5c9ef3e 100644 --- a/ps2xRuntime/include/ps2_runtime.h +++ b/ps2xRuntime/include/ps2_runtime.h @@ -23,6 +23,7 @@ #include #include "ps2_log.h" +#include "ps2_scheduler.h" #include "runtime/ps2_address.h" #include "runtime/ps2_gif_arbiter.h" #include "runtime/ps2_memory.h" @@ -328,6 +329,19 @@ struct PS2DtxCompatLayout } }; +// Async callback stack pool [floor, top): kernel-reserved guest memory. See +// the pool layout comment in ps2_runtime.cpp for why this range is disjoint +// from every other guest allocation. +constexpr uint32_t kAsyncCallbackStackFloor = 0x00080000u; +constexpr uint32_t kAsyncCallbackStackTop = 0x00100000u; + +// $sp for a host-dispatched guest callback that could not get a stack out of +// the async-callback pool (pool exhausted or runtime unavailable). Top of the +// kernel-reserved pool, NOT PS2_RAM_SIZE-0x10 — that address is inside the +// guest's own main stack (games place it at top-of-RAM via SetupThread), and +// running a handler there would corrupt live guest frames. +constexpr uint32_t kAsyncCallbackFallbackSp = kAsyncCallbackStackTop - 0x10u; + class PS2Runtime { public: @@ -381,31 +395,30 @@ class PS2Runtime SkipCallDebug = 3, }; + // No-op RAII guards. Only one fiber ever executes guest code at a time + // under the N=1 cooperative scheduler, and exclusion between the fiber + // executor and borrowed host worker threads is provided by + // ps2sched::async_guest_begin/async_guest_end (AsyncGuestScope). Kept as + // no-ops only so code that still references them (MPEG/IPU decoder stubs) + // compiles unchanged. class GuestExecutionScope { public: - explicit GuestExecutionScope(PS2Runtime *runtime) noexcept; - ~GuestExecutionScope(); + explicit GuestExecutionScope(PS2Runtime *) noexcept {} + ~GuestExecutionScope() = default; GuestExecutionScope(const GuestExecutionScope &) = delete; GuestExecutionScope &operator=(const GuestExecutionScope &) = delete; - - private: - PS2Runtime *m_runtime = nullptr; }; class GuestExecutionReleaseScope { public: - explicit GuestExecutionReleaseScope(PS2Runtime *runtime) noexcept; - ~GuestExecutionReleaseScope(); + explicit GuestExecutionReleaseScope(PS2Runtime *) noexcept {} + ~GuestExecutionReleaseScope() = default; GuestExecutionReleaseScope(const GuestExecutionReleaseScope &) = delete; GuestExecutionReleaseScope &operator=(const GuestExecutionReleaseScope &) = delete; - - private: - PS2Runtime *m_runtime = nullptr; - uint32_t m_depth = 0u; }; bool replaceFunction(uint32_t address, RecompiledFunction func); @@ -430,6 +443,12 @@ class PS2Runtime MissingFunctionPolicy missingFunctionPolicy() const; void resetMissingFunctionReportOnce(); + // Test/diagnostic seam: formats the dispatch-trace ring owned by whichever + // fiber (or per-OS-thread fallback) is executing on the calling thread. + // MUST only be called from inside a running fiber's step function — from + // any other thread it returns the non-fiber fallback, not a fiber's trace. + std::string debugCurrentDispatchTrace() const; + static const IoPaths &getIoPaths(); static void setIoPaths(const IoPaths &paths); static void configureIoPathsFromElf(const std::string &elfPath); @@ -462,13 +481,9 @@ class PS2Runtime void dispatchLoop(uint8_t *rdram, R5900Context *ctx); void drainCompletedDmacHandlers(uint8_t *rdram); bool shouldPreemptGuestExecution(); - void yieldGuestExecutionAfterWake(); void requestStop(); + void requestStopFlagOnly(); bool isStopRequested() const; - uint32_t guestExecutionWaiterCountForTesting() const - { - return m_guestExecutionWaiters.load(std::memory_order_acquire); - } uint8_t Load8(uint8_t *rdram, R5900Context *ctx, uint32_t vaddr); uint16_t Load16(uint8_t *rdram, R5900Context *ctx, uint32_t vaddr); @@ -535,17 +550,9 @@ class PS2Runtime uint32_t allocateGuestBlockLocked(uint32_t size, uint32_t alignment); void freeGuestBlockLocked(uint32_t guestAddr); void coalesceGuestHeapLocked(); - void enterGuestExecution(); - void leaveGuestExecution(); - uint32_t releaseGuestExecution(); - void reacquireGuestExecution(uint32_t depth); - void markGuestExecutionAcquired(); void HandleIntegerOverflow(R5900Context *ctx); - friend class GuestExecutionScope; - friend class GuestExecutionReleaseScope; - private: PS2Memory m_memory; GifArbiter m_gifArbiter; @@ -556,11 +563,6 @@ class PS2Runtime VU1Interpreter m_vu0; VU1Interpreter m_vu1; R5900Context m_cpuContext; - mutable std::recursive_mutex m_guestExecutionMutex; - mutable std::atomic m_guestExecutionWaiters{0u}; - mutable std::mutex m_guestExecutionHandoffMutex; - mutable std::condition_variable m_guestExecutionHandoffCv; - std::atomic m_guestExecutionHandoffEpoch{0u}; mutable std::mutex m_guestHeapMutex; mutable std::mutex m_asyncCallbackStackMutex; std::vector m_guestHeapBlocks; @@ -569,8 +571,12 @@ class PS2Runtime uint32_t m_guestHeapLimit = PS2_RAM_SIZE; uint32_t m_guestHeapSuggestedBase = 0x00100000u; bool m_guestHeapConfigured = false; - uint32_t m_asyncCallbackStackFloor = 0x01F00000u; - uint32_t m_asyncCallbackStackTop = PS2_RAM_SIZE; + // Async callback stack pool [floor, top): kernel-reserved guest memory, + // carved downward. See the pool layout comment in ps2_runtime.cpp (near + // kAsyncCallbackStackFloor's namespace-level block) for the full + // disjointness rationale. loadELF() re-arms these on each load. + uint32_t m_asyncCallbackStackFloor = kAsyncCallbackStackFloor; + uint32_t m_asyncCallbackStackTop = kAsyncCallbackStackTop; std::atomic m_missingFunctionPolicy{static_cast(MissingFunctionPolicy::ContinueToTarget)}; std::atomic m_missingFunctionReported{false}; @@ -582,6 +588,9 @@ class PS2Runtime bool m_debugUiInitialized = false; public: + // Live snapshot of the currently-dispatching guest thread's PC/RA/SP/GP, + // updated from PS2Runtime::dispatchLoop() on every step. Read by the + // debug UI (ps2_debug_panel.cpp); independent of the threading backend. std::atomic m_debugPc{0}; std::atomic m_debugRa{0}; std::atomic m_debugSp{0}; @@ -601,6 +610,51 @@ class PS2Runtime uint8_t *m_boundGSVram = nullptr; }; +// One-shot per-invocation guest scratch stack for running recompiled guest code +// inline on the calling fiber (RPC/override invoke, inline DMAC handlers, MPEG +// stream callbacks). Reserves a fresh guest-heap region on construction and +// releases it on destruction, so two fibers interleaving through the same path +// — or one fiber re-entering it — always run on DISJOINT stacks. RAII release +// also fires on the ThreadExitException unwind used for cooperative fiber +// teardown, so a killed fiber cannot leak. +class GuestScratchStack +{ +public: + GuestScratchStack(PS2Runtime *runtime, uint32_t size) + : m_runtime(runtime) + { + if (m_runtime != nullptr && size != 0u) + { + m_base = m_runtime->guestMalloc(size, 16u); + if (m_base != 0u) + { + m_top = (m_base + size) & ~0xFu; + } + } + } + ~GuestScratchStack() + { + if (m_runtime != nullptr && m_base != 0u) + { + m_runtime->guestFree(m_base); + } + } + GuestScratchStack(const GuestScratchStack &) = delete; + GuestScratchStack &operator=(const GuestScratchStack &) = delete; + + // Aligned guest $sp for the reserved stack; 0 if the reservation failed. + uint32_t top() const { return m_top; } + bool valid() const { return m_top != 0u; } + // Reservation base; 0 if the reservation failed. Callers that need a flat + // scratch BUFFER (not a stack) use this instead of top(). + uint32_t base() const { return m_base; } + +private: + PS2Runtime *m_runtime = nullptr; + uint32_t m_base = 0u; + uint32_t m_top = 0u; +}; + // Generated by ps2xRecomp in ps2xRuntime/src/runner/register_functions.cpp. extern const uint32_t g_ps2RecompiledFunctionTableBase; extern const uint32_t g_ps2RecompiledFunctionTableEnd; diff --git a/ps2xRuntime/include/ps2_scheduler.h b/ps2xRuntime/include/ps2_scheduler.h new file mode 100644 index 000000000..33e4f2b49 --- /dev/null +++ b/ps2xRuntime/include/ps2_scheduler.h @@ -0,0 +1,192 @@ +#ifndef PS2_SCHEDULER_H +#define PS2_SCHEDULER_H + +// --------------------------------------------------------------------------- +// ps2_scheduler.h — public API for the N=1 fiber cooperative scheduler. +// +// Exactly one dedicated host OS thread (g_guest_thread, the "guest executor") +// runs all guest fibers. This eliminates cross-thread swapcontext UB structurally. +// +// No , , , or other heavy headers are included here; +// this keeps the header safe for Vita (VitaSDK lacks C++20 ) and +// keeps compile times low. +// --------------------------------------------------------------------------- + +#include + +// Guest thread identity. -1 means this host thread is not currently running a fiber. +// Defined in ps2_scheduler.cpp; extern-declared here and in State.h. +extern thread_local int g_currentThreadId; + +class PS2Runtime; +struct DispatchHistory; // defined in ps2_dispatch_history.h +struct SyscallOverrideStack; // defined in ps2_syscall_override_state.h + +namespace ps2sched +{ + // --- Lifecycle --- + + // Initialise global scheduler state and create the single guest executor + // thread. Call once before create_fiber(). + void scheduler_init(); + + // Signal all fibers to terminate, join the guest executor thread, and free + // all fibers. Safe to call from the main OS thread after the render loop exits. + void scheduler_shutdown(); + + // Register a callback that scheduler_shutdown() invokes once, just before + // joining the guest executor thread. The runtime sets this to request a + // stop so dispatchLoop's while(!isStopRequested()) exits between dispatched + // functions. Pass nullptr to clear. May be called before scheduler_init(). + void scheduler_set_stop_callback(void (*fn)()); + + // --- Thread lifecycle (called from Thread.cpp / Lifecycle.cpp) --- + + // Create a fiber for guest thread `tid` with the given CPU registers and + // enqueue it as Ready. May THROW std::bad_alloc / std::runtime_error on + // stack or fiber allocation failure. StartThread catches the throw and reports allocation failure. + void create_fiber(int tid, int priority, uint32_t entry, + uint32_t sp, uint32_t gp, uint32_t arg, + PS2Runtime* rt, uint8_t* rdram); + + // Set terminateRequested on tid's fiber and wake it if blocked. + void request_terminate(int tid); + + // Cooperatively wait (yielding) until tid's fiber has finished. + // Must be called from a fiber (not from a host thread). + void join_fiber(int tid); + + // Reorder the fiber in the run queue with its new priority. + void update_priority(int tid, int newPriority); + + // --- Blocking / readiness (Sync.cpp, Thread.cpp, Interrupt.cpp) --- + + // Arm the park BEFORE publishing to an object wait-list. Sets the running + // fiber's scheduler state to Blocked and clears wake_pending under + // g_sched_mutex, so a waker that fires after the wait-list publish but before + // block_current() sees state==Blocked and routes through wake_locked (setting + // wake_pending) instead of dropping the wakeup. Acts on the fiber currently + // running on the executor thread (tls_current_fiber); callers must only invoke + // it from a fiber. No-op if called with no current fiber. + void arm_park(); + + // Result of block_current(). See block_current() below. + enum class BlockResult + { + Parked, // a fiber actually parked and was later resumed by a waker. + WokenInWindow, // a fiber: a wakeup arrived in the arm/publish window; do + // not park. Caller re-checks its wait condition. + NonFiberOwner, // a borrowed host worker that HOLDS the guest token: caller + // must async_guest_end()/yield/async_guest_begin(), then + // re-check. + NonFiberNoTok // a borrowed host worker that does NOT hold the token: + // caller just yields the host thread and re-checks; it + // must NOT touch the guest token. + }; + + // Park the currently-running fiber. Caller MUST have called arm_park() first + // and then published itself to the object wait-list. The fiber must release any + // object mutexes BEFORE calling this. See BlockResult for the four outcomes. + BlockResult block_current(); + + // Wake a blocked fiber from within another fiber (from a guest thread). + // Gated on suspendCount == 0. + void make_ready(int tid); + + // Wake a blocked fiber from a non-fiber host thread, but ONLY if the fiber + // currently mapped to `tid` is still the exact fiber identified by `token` + // (from current_fiber_token()). The token encodes the fiber's generation, so + // a recycled tid whose new fiber has a different generation will not match + // and the stale wakeup is dropped. Validation happens under g_sched_mutex. + void enqueue_external_wakeup_validated(int tid, uint64_t token); + + // Opaque identity token for the fiber currently running on this thread, or 0 + // if not on a fiber. Encodes generation + tid. Only ever compared for + // equality / passed back to enqueue_external_wakeup_validated. + uint64_t current_fiber_token(); + + // Yield if a higher-priority fiber is ready. Enqueues self Ready BEFORE + // yielding, so the fiber is never 'Running but off-queue'. + void maybe_yield(); + + // Suspend the current fiber (SuspendThread on self). Increments suspendCount. + void suspend_self(); + + // Suspend another fiber (SuspendThread on other). Increments suspendCount. + void suspend_other(int tid); + + // Force the fiber's suspendCount to 0 and wake it if Blocked. Called by + // ResumeThread when the PS2-visible ThreadInfo::suspendCount reaches 0, so + // nested SuspendThread calls resolve in a single resume. + void clear_suspend(int tid); + + // Rotate the equal-priority group in the run queue (RotateThreadReadyQueue). + void rotate_ready_queue(int priority); + + // --- Async guest-code borrow --- + // (called by IRQ worker / alarm worker / RPC worker host threads) + // Use the AsyncGuestScope RAII guard below instead of calling these + // directly. MUST NOT be called from a fiber. + + // Acquire the "guest token": blocks until no fiber is executing guest code. + void async_guest_begin(); + + // Release the guest token. + void async_guest_end(); + + // --- Back-edge hook — called by PS2Runtime::shouldPreemptGuestExecution() --- + + // Sampled every 128 back-edges. Checks terminateRequested, suspendCount, + // priority, and parked host workers; may ps2fiber_yield internally. + // Returns true iff it yielded to a parked host worker (step 4), false + // otherwise. + bool yield_point(); + + // Number of host workers currently blocked in async_guest_begin() waiting + // for the guest token. The executor's resume predicate is gated on this + // (it must genuinely sleep — releasing the scheduler mutex — while a + // worker is parked, or the worker can never win the token). + int host_token_waiters(); + + // True iff the calling OS thread is a guest execution context: the single + // guest executor thread (which runs all fibers on the ucontext backend), + // or a fiber's own OS thread on the pthread backend (where each fiber owns + // the execution slot whenever it runs). Lets shared helpers that are + // reachable BOTH from host workers (which must borrow the guest token via + // AsyncGuestScope) AND synchronously from already-running guest code + // (which owns the execution slot implicitly and must NOT call + // async_guest_begin — it aborts by design there) pick the correct + // behavior. Also used by requestStop paths: joining a host worker from + // guest context can deadlock (the worker may be parked in + // async_guest_begin waiting for the very fiber that is joining). + bool is_guest_thread(); + + // Diagnostic dispatch-trace ring owned by the fiber currently running on + // this thread. Never null: returns the fiber's own ring when a fiber runs + // here, otherwise a persistent per-OS-thread instance (borrowed host + // worker, executor between fibers, or a direct non-fiber caller). + // Discriminates on tls_current_fiber, the same identity used by current_fiber_token(). + ::DispatchHistory& current_dispatch_history() noexcept; + + // Stack of syscall numbers whose overrides are currently running on the + // fiber executing on this thread. Never null: returns the fiber's own + // stack when a fiber runs here, otherwise a persistent per-OS-thread + // stack (borrowed host worker, executor between fibers, or a direct + // non-fiber caller), which is what the syscall layer uses to catch + // single-context self-recursion. + // Discriminates on tls_current_fiber, same identity as current_dispatch_history(). + ::SyscallOverrideStack& current_active_syscall_overrides() noexcept; + +} // namespace ps2sched + +// RAII guard that ensures async_guest_begin/async_guest_end are always paired, +// even on exception paths. Use this in IRQ/DMAC/alarm workers instead of bare calls. +struct AsyncGuestScope +{ + AsyncGuestScope() { ps2sched::async_guest_begin(); } + ~AsyncGuestScope() { ps2sched::async_guest_end(); } + AsyncGuestScope(const AsyncGuestScope&) = delete; + AsyncGuestScope& operator=(const AsyncGuestScope&) = delete; +}; + +#endif // PS2_SCHEDULER_H diff --git a/ps2xRuntime/include/ps2_syscall_override_state.h b/ps2xRuntime/include/ps2_syscall_override_state.h new file mode 100644 index 000000000..2e4c9acf9 --- /dev/null +++ b/ps2xRuntime/include/ps2_syscall_override_state.h @@ -0,0 +1,22 @@ +#ifndef PS2_SYSCALL_OVERRIDE_STATE_H +#define PS2_SYSCALL_OVERRIDE_STATE_H +#include +#include + +// Stack of syscall numbers whose overrides are currently executing on ONE guest +// context. dispatchSyscallOverride pushes a number before invoking its guest +// handler and pops it after; if the number is already present the handler is NOT +// re-entered (it falls through to the builtin) — this is what lets an override +// handler safely re-issue the very syscall it overrides without recursing. +// +// Owned per-fiber by FiberContext under the N=1 executor: a fiber parked mid- +// override keeps its own active set, so it never marks the syscall active for a +// different fiber, and every push/pop stays LIFO-balanced within one fiber's +// call chain. Host workers and direct non-fiber callers use a per-OS-thread +// fallback (each runs its override chain to completion without yielding). +// Mirrors the DispatchHistory ownership model (ps2_dispatch_history.h). +struct SyscallOverrideStack +{ + std::vector active; +}; +#endif // PS2_SYSCALL_OVERRIDE_STATE_H diff --git a/ps2xRuntime/include/ps2_syscalls.h b/ps2xRuntime/include/ps2_syscalls.h index 0411a2aff..809f95f97 100644 --- a/ps2xRuntime/include/ps2_syscalls.h +++ b/ps2xRuntime/include/ps2_syscalls.h @@ -32,8 +32,6 @@ namespace ps2_syscalls void initializeGuestKernelState(uint8_t *rdram); void TODO(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime, uint32_t encodedSyscallId); void notifyRuntimeStop(); - void joinAllGuestHostThreads(); - void detachAllGuestHostThreads(); void resetSoundDriverRpcState(); void setSoundDriverCompatLayout(const PS2SoundDriverCompatLayout &layout); void clearSoundDriverCompatLayout(); diff --git a/ps2xRuntime/src/lib/Kernel/Stubs/GS.cpp b/ps2xRuntime/src/lib/Kernel/Stubs/GS.cpp index 7e46972fc..db3b0074c 100644 --- a/ps2xRuntime/src/lib/Kernel/Stubs/GS.cpp +++ b/ps2xRuntime/src/lib/Kernel/Stubs/GS.cpp @@ -682,9 +682,16 @@ namespace ps2_stubs try { + // Acquire the guest token before running recompiled PS2 code. This + // dispatch runs on the interrupt worker (a host thread); without the + // token the callback executes concurrently with whatever fiber the + // guest executor is running, violating the N=1 invariant. It also + // makes the worker a visible g_host_token_waiters waiter, which the + // executor's resume predicate is gated on. + AsyncGuestScope guestScope; R5900Context callbackCtx{}; SET_GPR_U32(&callbackCtx, 28, gp); - SET_GPR_U32(&callbackCtx, 29, (callbackStackTop != 0u) ? callbackStackTop : (PS2_RAM_SIZE - 0x10u)); + SET_GPR_U32(&callbackCtx, 29, (callbackStackTop != 0u) ? callbackStackTop : kAsyncCallbackFallbackSp); SET_GPR_U32(&callbackCtx, 31, 0u); SET_GPR_U32(&callbackCtx, 4, static_cast(callbackTick)); callbackCtx.pc = callback; diff --git a/ps2xRuntime/src/lib/Kernel/Stubs/IPU.cpp b/ps2xRuntime/src/lib/Kernel/Stubs/IPU.cpp index a589216e6..d7059ce7b 100644 --- a/ps2xRuntime/src/lib/Kernel/Stubs/IPU.cpp +++ b/ps2xRuntime/src/lib/Kernel/Stubs/IPU.cpp @@ -37,7 +37,6 @@ namespace ps2_stubs auto setD4 = runtime->lookupFunction(SETD4_CHCR_ENTRY); ctx->r[4] = _mm_set_epi64x(0, 1); { - PS2Runtime::GuestExecutionScope guestExecution(runtime); setD4(rdram, ctx, runtime); } } diff --git a/ps2xRuntime/src/lib/Kernel/Stubs/MPEG.cpp b/ps2xRuntime/src/lib/Kernel/Stubs/MPEG.cpp index 8bd093f99..d3046fe25 100644 --- a/ps2xRuntime/src/lib/Kernel/Stubs/MPEG.cpp +++ b/ps2xRuntime/src/lib/Kernel/Stubs/MPEG.cpp @@ -1282,23 +1282,25 @@ namespace ps2_stubs return; } - thread_local PS2Runtime *s_callbackStackRuntime = nullptr; - thread_local uint32_t s_callbackStackTop = 0u; - if (s_callbackStackRuntime != runtime || s_callbackStackTop == 0u) - { - constexpr uint32_t kCallbackStackSize = 0x4000u; - s_callbackStackRuntime = runtime; - s_callbackStackTop = runtime->reserveAsyncCallbackStack(kCallbackStackSize, 16u); - } - - const uint32_t cbDataAddr = runtime->guestMalloc(kMpegCallbackDataSize, 16u); + // Per-invocation scratch stack (fresh guest-heap region, RAII-freed + // on every return below): MPEG stream callbacks run inline on the + // calling fiber and the body can yield at a back-edge, so a shared + // stack would be clobbered by another fiber dispatching a callback + // or by this fiber re-entering via a nested MPEG syscall. Mirrors + // the cbData reservation this function already frees on return. + constexpr uint32_t kCallbackStackSize = 0x4000u; + GuestScratchStack callbackStack(runtime, kCallbackStackSize); + + // RAII like the scratch stack above: a ThreadExitException from a + // callback step must not leak the callback-data reservation. + GuestScratchStack cbData(runtime, kMpegCallbackDataSize); + const uint32_t cbDataAddr = cbData.base(); if (cbDataAddr == 0u) { return; } if (!writeMpegCallbackData(rdram, cbDataAddr, event)) { - runtime->guestFree(cbDataAddr); return; } @@ -1307,7 +1309,9 @@ namespace ps2_stubs SET_GPR_U32(&callbackCtx, 5, cbDataAddr); SET_GPR_U32(&callbackCtx, 6, callback.data); SET_GPR_U32(&callbackCtx, 7, 0u); - SET_GPR_U32(&callbackCtx, 29, (s_callbackStackTop != 0u) ? s_callbackStackTop : (PS2_RAM_SIZE - 0x10u)); + // Failure fallback: kernel-pool top, not inside the guest's own + // main stack (see kAsyncCallbackFallbackSp). + SET_GPR_U32(&callbackCtx, 29, callbackStack.valid() ? callbackStack.top() : kAsyncCallbackFallbackSp); SET_GPR_U32(&callbackCtx, 31, 0u); callbackCtx.pc = callback.func; @@ -1351,8 +1355,6 @@ namespace ps2_stubs ++stepLimitLogCount; } } - - runtime->guestFree(cbDataAddr); } void dispatchStreamCallbacks(uint8_t *rdram, diff --git a/ps2xRuntime/src/lib/Kernel/Stubs/SIF.cpp b/ps2xRuntime/src/lib/Kernel/Stubs/SIF.cpp index feb6e8b74..d41c7348a 100644 --- a/ps2xRuntime/src/lib/Kernel/Stubs/SIF.cpp +++ b/ps2xRuntime/src/lib/Kernel/Stubs/SIF.cpp @@ -1,6 +1,7 @@ #include "Common.h" #include "SIF.h" #include "../Syscalls/RPC.h" +#include "../Syscalls/Thread.h" #include "runtime/ps2_address.h" #include @@ -587,6 +588,20 @@ namespace ps2_stubs void sceSifRpcLoop(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) { + // The real sceSifRpcLoop is `while (1) { SleepThread(); serve; }` - + // the RPC server thread spends its life parked in SleepThread, woken + // only by SIF RPC deliveries. All SIF RPC traffic here is HLE'd + // host-side, so no WakeupThread ever targets this thread: park it via + // the real SleepThread syscall. If this ever returns without blocking, + // ctx->pc stays at the loop entry, so the dispatch loop re-enters this + // stub forever and the RPC server fiber monopolizes the N=1 guest + // executor, starving every other guest thread. Any + // title that starts a SIF RPC server thread (pad, memcard, audio, + // filesystem) and then drives its main thread would hang forever. + // ctx->pc is intentionally left at the loop entry: a stray + // wakeup simply re-enters and sleeps again, exactly like the real + // loop. + ps2_syscalls::SleepThread(rdram, ctx, runtime); setReturnS32(ctx, 0); } diff --git a/ps2xRuntime/src/lib/Kernel/Syscalls/Common.h b/ps2xRuntime/src/lib/Kernel/Syscalls/Common.h index 8ad6cdca7..be02a6402 100644 --- a/ps2xRuntime/src/lib/Kernel/Syscalls/Common.h +++ b/ps2xRuntime/src/lib/Kernel/Syscalls/Common.h @@ -1,6 +1,7 @@ #include "ps2_syscalls.h" #include "ps2_log.h" #include "ps2_runtime.h" +#include "ps2_scheduler.h" #include "runtime/ps2_iop_audio.h" #include "ps2_runtime_macros.h" #include "ps2_stubs.h" @@ -35,11 +36,3 @@ std::string translatePs2Path(const char *ps2Path); #include "Helpers/State.h" #include "Helpers/Loader.h" #include "Helpers/Runtime.h" - -namespace ps2_syscalls -{ - inline void yieldGuestExecutionAfterWake(PS2Runtime *runtime) - { - runtime->yieldGuestExecutionAfterWake(); - } -} diff --git a/ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/Runtime.h b/ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/Runtime.h index 47b560a16..50c92e7f3 100644 --- a/ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/Runtime.h +++ b/ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/Runtime.h @@ -1,89 +1,115 @@ -struct ThreadExitException final : public std::exception -{ - const char *what() const noexcept override - { - return "PS2 Thread Exit"; +#include "ThreadExit.h" +#include "ps2_scheduler.h" +#include +#include +#include +#include +#include + +// Swaps a wait-list out from under its mutex and delivers a validated external +// wakeup to every waiter that was on it. This is ONLY the pure swap-and-drain +// shape: callers whose critical section also mutates other state alongside +// the swap (e.g. marking the object deleted) must keep doing that inline +// instead of calling this, so the lock is still held across both writes. +static inline void wakeWaiters(std::mutex &m, std::vector> &list) +{ + std::vector> waiters; + { + std::lock_guard lk(m); + waiters.swap(list); } -}; - -static void throwIfTerminated(const std::shared_ptr &info) -{ - if (info && info->terminated.load()) + for (const auto &[tid, token] : waiters) { - throw ThreadExitException(); + ps2sched::enqueue_external_wakeup_validated(tid, token); } } -// Condition-variable waits in the EE runtime must release the global guest -// execution mutex, but must not reacquire it while still holding the local -// wait-object mutex. Reacquiring guest execution while holding a semaphore, -// thread, event-flag, or vsync mutex can create an ABBA deadlock: -// -// awakened thread: local wait mutex -> waiting for GuestExecutionScope -// running thread: GuestExecutionScope -> waiting for local wait mutex -// -// This helper keeps guest execution released for the whole host wait and for -// any post-wake bookkeeping that needs the local mutex, then unlocks the local -// mutex before the GuestExecutionReleaseScope destructor reacquires guest code. -template -static void waitWithGuestExecutionReleasedUntilUnlocked(PS2Runtime *runtime, - Lock &lock, - WaitFn waitFn, - FinishFn finishFn) +// Bounded backoff for a borrowed host worker that hit a blocking syscall. +// Translates a non-fiber BlockResult into the correct token handling, then +// sleeps with exponential backoff (1us -> 1ms cap). After kMaxSpins iterations +// it logs ONCE and keeps sleeping at the cap so a self-deadlocked interrupt +// handler cannot busy-spin the CPU or starve the guest executor. State is held +// in a per-call counter object so each blocking site ramps independently. +struct NonFiberBackoff { - auto releaseGuestExecution = std::make_unique(runtime); - - waitFn(); - finishFn(); + int spins = 0; + std::chrono::microseconds delay{1}; - if (lock.owns_lock()) + // Sleeps this worker once with exponential backoff. The syscall's Mesa loop + // decides whether to re-check its wait condition and loop again. + void step(ps2sched::BlockResult br) { - lock.unlock(); - } + // Drop/reacquire the guest token only if we actually own it. + if (br == ps2sched::BlockResult::NonFiberOwner) + { + ps2sched::async_guest_end(); + std::this_thread::sleep_for(delay); + ps2sched::async_guest_begin(); + } + else // NonFiberNoTok (or, defensively, any non-Parked non-fiber result) + { + std::this_thread::sleep_for(delay); + } - releaseGuestExecution.reset(); -} + constexpr int kMaxSpins = 50; + if (spins == kMaxSpins) + { + std::fprintf(stderr, + "[ps2sched] WARNING: borrowed host worker has blocked on a guest " + "condition for %d retries; capping backoff at 1ms (possible " + "interrupt-context deadlock)\n", kMaxSpins); + } + if (spins < kMaxSpins) + { + ++spins; + delay *= 2; + if (delay > std::chrono::microseconds(1000)) + delay = std::chrono::microseconds(1000); + } + } +}; -template -static void waitWithGuestExecutionReleasedUntilUnlocked(PS2Runtime *runtime, Lock &lock, WaitFn waitFn) +// One-shot: drop/reacquire token (if owned) and yield once. SleepThread for +// a borrowed worker has no wait-list to re-check, so it does not loop. +inline void nonFiberBlockBackoff(ps2sched::BlockResult br) { - waitWithGuestExecutionReleasedUntilUnlocked(runtime, lock, waitFn, []() {}); + if (br == ps2sched::BlockResult::NonFiberOwner) + { + ps2sched::async_guest_end(); + std::this_thread::yield(); + ps2sched::async_guest_begin(); + } + else + { + std::this_thread::yield(); + } } -static void waitWhileSuspended(const std::shared_ptr &info, PS2Runtime *runtime = nullptr) +static void throwIfTerminated(const std::shared_ptr &info) { - if (!info) - return; - - std::unique_lock lock(info->m); - if (info->suspendCount > 0) + if (info && info->terminated.load()) { - info->status = THS_SUSPEND; - info->waitType = TSW_NONE; - info->waitId = 0; + throw ThreadExitException(); + } +} - bool terminated = false; - waitWithGuestExecutionReleasedUntilUnlocked( - runtime, - lock, - [&]() - { - info->cv.wait(lock, [&]() - { return info->suspendCount == 0 || info->terminated.load(); }); - }, - [&]() - { - terminated = info->terminated.load(); - if (!terminated) - { - info->status = THS_RUN; - } - }); - - if (terminated) - { - throw ThreadExitException(); - } +// Waiting is done via arm_park()/block_current(), which already +// cooperatively yields to other fibers/borrowed host workers, so no explicit +// guest-execution release/reacquire step is needed. +static void waitWhileSuspended(const std::shared_ptr &info) +{ + if (!info) return; + while (info->suspendCount > 0 && !info->terminated.load()) { + // arm_park() before publishing to any wait-list so a concurrent + // clear_suspend that fires between publish and block_current sees + // wake_pending rather than missing the wakeup. + ps2sched::arm_park(); + ps2sched::block_current(); + } + if (info->terminated.load()) { throw ThreadExitException(); } + if (info->suspendCount == 0) { + std::lock_guard lock(info->m); + info->status = THS_RUN; } } @@ -162,96 +188,6 @@ static std::chrono::microseconds alarmTicksToDuration(uint16_t ticks) return std::chrono::microseconds(clampedTicks * kAlarmTickUsec); } -static void ensureAlarmWorkerRunning() -{ - std::call_once(g_alarm_worker_once, []() - { std::thread([]() - { - for (;;) - { - std::shared_ptr readyAlarm; - { - std::unique_lock lock(g_alarm_mutex); - while (!readyAlarm) - { - if (g_alarms.empty()) - { - g_alarm_cv.wait(lock); - continue; - } - - auto nextIt = std::min_element(g_alarms.begin(), g_alarms.end(), - [](const auto &a, const auto &b) - { - return a.second->dueAt < b.second->dueAt; - }); - if (nextIt == g_alarms.end()) - { - g_alarm_cv.wait(lock); - continue; - } - - const auto now = std::chrono::steady_clock::now(); - if (nextIt->second->dueAt > now) - { - g_alarm_cv.wait_until(lock, nextIt->second->dueAt); - continue; - } - - readyAlarm = nextIt->second; - g_alarms.erase(nextIt); - } - } - - if (!readyAlarm || !readyAlarm->runtime || !readyAlarm->rdram || !readyAlarm->handler) - { - continue; - } - if (!readyAlarm->runtime->hasFunction(readyAlarm->handler)) - { - continue; - } - - try - { - constexpr uint32_t kAlarmCallbackStackSize = 0x4000u; - thread_local PS2Runtime *s_alarmStackRuntime = nullptr; - thread_local uint32_t s_alarmStackTop = 0u; - if (s_alarmStackRuntime != readyAlarm->runtime || s_alarmStackTop == 0u) - { - s_alarmStackRuntime = readyAlarm->runtime; - s_alarmStackTop = readyAlarm->runtime->reserveAsyncCallbackStack(kAlarmCallbackStackSize, 16u); - } - - R5900Context callbackCtx{}; - setRegU32(&callbackCtx, 28, readyAlarm->gp); - setRegU32(&callbackCtx, 29, - (s_alarmStackTop != 0u) ? s_alarmStackTop : (PS2_RAM_SIZE - 0x10u)); - setRegU32(&callbackCtx, 31, 0); - setRegU32(&callbackCtx, 4, static_cast(readyAlarm->id)); - setRegU32(&callbackCtx, 5, static_cast(readyAlarm->ticks)); - setRegU32(&callbackCtx, 6, readyAlarm->commonArg); - setRegU32(&callbackCtx, 7, 0); - callbackCtx.pc = readyAlarm->handler; - - PS2Runtime::RecompiledFunction func = readyAlarm->runtime->lookupFunction(readyAlarm->handler); - func(readyAlarm->rdram, &callbackCtx, readyAlarm->runtime); - } - catch (const ThreadExitException &) - { - } - catch (const std::exception &e) - { - static int alarmExceptionLogs = 0; - if (alarmExceptionLogs < 8) - { - std::cerr << "[SetAlarm] callback exception: " << e.what() << std::endl; - ++alarmExceptionLogs; - } - } - } }) - .detach(); }); -} static void rpcCopyToRdram(uint8_t *rdram, uint32_t dst, uint32_t src, size_t size) { @@ -355,6 +291,11 @@ static const char *rpcInvokeExitReasonName(RpcInvokeExitReason reason) } } +// rpcInvokeFunction runs on the CALLING FIBER (the guest thread that issued +// the RPC syscall), not on a worker thread. Do NOT wrap calls to this in +// AsyncGuestScope — the calling fiber already holds the guest execution slot. +// If, in the future, RPC server dispatch is moved to a dedicated worker thread, +// that worker must wrap its guest invocation in AsyncGuestScope. static bool rpcInvokeFunction(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime, uint32_t funcAddr, uint32_t a0, uint32_t a1, uint32_t a2, uint32_t a3, uint32_t *outV0) { @@ -371,22 +312,17 @@ static bool rpcInvokeFunction(uint8_t *rdram, R5900Context *ctx, PS2Runtime *run setRegU32(&tmp, 6, a2); setRegU32(&tmp, 7, a3); - thread_local uint32_t s_rpcInvokeStackBase = 0u; - thread_local uint32_t s_rpcInvokeStackTop = 0u; - if (s_rpcInvokeStackTop == 0u) + // Per-invocation scratch stack: a fresh guest-heap region reserved for the + // duration of THIS invoke and released by RAII on every return path below + // (and on a ThreadExitException thrown out of the invoke loop). Isolates + // interleaving fibers — the N=1 executor runs them all on one OS thread — + // AND same-fiber re-entry: a nested override or an exit-handler invoke gets + // its own stack and never clobbers the outer frame. + GuestScratchStack invokeStack(runtime, kRpcInvokeStackSize); + if (invokeStack.valid()) { - const uint32_t stackBase = runtime->guestMalloc(kRpcInvokeStackSize, 16u); - if (stackBase != 0u) - { - s_rpcInvokeStackBase = stackBase; - s_rpcInvokeStackTop = (stackBase + kRpcInvokeStackSize) & ~0xFu; - } - } - if (s_rpcInvokeStackTop != 0u) - { - setRegU32(&tmp, 29, s_rpcInvokeStackTop); + setRegU32(&tmp, 29, invokeStack.top()); } - (void)s_rpcInvokeStackBase; setRegU32(&tmp, 31, kRpcInvokeReturnSentinel); tmp.pc = funcAddr; @@ -417,10 +353,7 @@ static bool rpcInvokeFunction(uint8_t *rdram, R5900Context *ctx, PS2Runtime *run } PS2Runtime::RecompiledFunction func = runtime->lookupFunction(pc); - { - PS2Runtime::GuestExecutionScope guestExecution(runtime); - func(rdram, &tmp, runtime); - } + func(rdram, &tmp, runtime); ++steps; } diff --git a/ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/State.h b/ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/State.h index c9551764b..fb7a310d0 100644 --- a/ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/State.h +++ b/ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/State.h @@ -30,9 +30,16 @@ struct ThreadInfo std::atomic currentPc{0}; std::mutex m; - std::condition_variable cv; std::atomic forceRelease{false}; std::atomic terminated{false}; + // Set true when StartThread mints this thread's g_activeThreads token + // (fetch_add). Cleared by exactly ONE consumer via exchange(true->false): + // on_fiber_exit for a normal exit, ExitDeleteThread when it removes its own + // g_threads entry, notifyRuntimeStop when it reaps residual guest threads, + // or StartThread's own create_fiber failure path. Whoever wins the exchange + // performs the single matching fetch_sub: one token per started thread, + // exactly one decrement. + std::atomic activeCounted{false}; }; // Thread status @@ -52,6 +59,7 @@ struct ThreadInfo // Common kernel-like error codes used by thread/event/alarm syscalls. constexpr int KE_OK = 0; constexpr int KE_ERROR = -1; +constexpr int KE_NO_MEMORY = -400; constexpr int KE_ILLEGAL_PRIORITY = -403; constexpr int KE_ILLEGAL_MODE = -405; constexpr int KE_ILLEGAL_THID = -406; @@ -157,7 +165,10 @@ struct SemaInfo int waiters = 0; bool deleted = false; std::mutex m; - std::condition_variable cv; + // Wait list of blocked guest threads. Each entry is {tid, generation token} + // where the token was captured via ps2sched::current_fiber_token() at push + // time. Protected by m; never hold m across a scheduling yield. + std::vector> waitList; }; struct EventFlagInfo @@ -169,7 +180,8 @@ struct EventFlagInfo int waiters = 0; bool deleted = false; std::mutex m; - std::condition_variable cv; + // See SemaInfo::waitList. + std::vector> waitList; }; struct AlarmInfo @@ -205,10 +217,8 @@ static constexpr uint32_t kFioSoIXOth = 0x0001; inline std::unordered_map> g_threads; inline int g_nextThreadId = 2; // Reserve 1 for the main thread -inline thread_local int g_currentThreadId = 1; +extern thread_local int g_currentThreadId; inline std::mutex g_thread_map_mutex; -inline std::unordered_map g_hostThreads; -inline std::mutex g_host_thread_mutex; inline std::unordered_map> g_semas; inline int g_nextSemaId = 1; @@ -220,119 +230,18 @@ inline std::unordered_map> g_alarms; inline int g_nextAlarmId = 1; inline std::mutex g_alarm_mutex; inline std::condition_variable g_alarm_cv; -inline std::once_flag g_alarm_worker_once; +// Stop mechanism. g_alarm_thread is joinable so stopAlarmWorker() can join it +// before rdram/runtime are destroyed. +inline std::thread g_alarm_thread; +inline std::atomic g_alarm_stop_flag{false}; +// Plain resettable flag (not once-only) so stopAlarmWorker() can clear it and +// allow ensureAlarmWorkerRunning() to restart the thread in a subsequent +// scheduler cycle (e.g. repeated init/shutdown in the test suite). +// Protected by g_alarm_mutex. +inline bool g_alarm_worker_running{false}; inline std::atomic g_activeThreads{0}; inline std::mutex g_fd_mutex; -static void registerHostThread(int tid, std::thread worker) -{ - std::thread stale; - { - std::lock_guard lock(g_host_thread_mutex); - auto it = g_hostThreads.find(tid); - if (it != g_hostThreads.end()) - { - stale = std::move(it->second); - g_hostThreads.erase(it); - } - g_hostThreads.emplace(tid, std::move(worker)); - } - - if (stale.joinable()) - { - if (stale.get_id() == std::this_thread::get_id()) - { - stale.detach(); - } - else - { - stale.join(); - } - } -} - -static void joinHostThreadById(int tid) -{ - std::thread worker; - { - std::lock_guard lock(g_host_thread_mutex); - auto it = g_hostThreads.find(tid); - if (it != g_hostThreads.end()) - { - worker = std::move(it->second); - g_hostThreads.erase(it); - } - } - - if (!worker.joinable()) - { - return; - } - - if (worker.get_id() == std::this_thread::get_id()) - { - worker.detach(); - } - else - { - worker.join(); - } -} - -static void joinAllHostThreads() -{ - std::vector workers; - { - std::lock_guard lock(g_host_thread_mutex); - workers.reserve(g_hostThreads.size()); - const std::thread::id selfId = std::this_thread::get_id(); - for (auto it = g_hostThreads.begin(); it != g_hostThreads.end();) - { - std::thread &worker = it->second; - if (worker.joinable() && worker.get_id() == selfId) - { - ++it; - continue; - } - - workers.push_back(std::move(worker)); - it = g_hostThreads.erase(it); - } - } - - for (auto &worker : workers) - { - if (!worker.joinable()) - { - continue; - } - worker.join(); - } -} - -static void detachAllHostThreads() -{ - std::vector workers; - { - std::lock_guard lock(g_host_thread_mutex); - workers.reserve(g_hostThreads.size()); - for (auto &entry : g_hostThreads) - { - workers.push_back(std::move(entry.second)); - } - g_hostThreads.clear(); - } - - for (auto &worker : workers) - { - if (!worker.joinable()) - { - continue; - } - worker.detach(); - } -} - struct RpcServerState { uint32_t sid = 0; diff --git a/ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/ThreadExit.h b/ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/ThreadExit.h new file mode 100644 index 000000000..6fef2cb46 --- /dev/null +++ b/ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/ThreadExit.h @@ -0,0 +1,11 @@ +#pragma once +#include + +// Must not be in an anonymous namespace — ps2_scheduler.cpp must catch the same type. +struct ThreadExitException final : public std::exception +{ + const char* what() const noexcept override + { + return "PS2 Thread Exit"; + } +}; diff --git a/ps2xRuntime/src/lib/Kernel/Syscalls/Interrupt.cpp b/ps2xRuntime/src/lib/Kernel/Syscalls/Interrupt.cpp index 220eadf18..5b6126fd5 100644 --- a/ps2xRuntime/src/lib/Kernel/Syscalls/Interrupt.cpp +++ b/ps2xRuntime/src/lib/Kernel/Syscalls/Interrupt.cpp @@ -2,6 +2,7 @@ #include "Interrupt.h" #include "ps2_log.h" #include "Stubs/GS.h" +#include "ps2_fiber.h" namespace ps2_syscalls { @@ -16,11 +17,17 @@ namespace ps2_syscalls std::mutex g_irq_worker_mutex; std::condition_variable g_irq_worker_cv; std::mutex g_vsync_flag_mutex; - std::condition_variable g_vsync_cv; + std::vector> g_vsync_waitList; std::atomic g_irq_worker_stop{false}; std::atomic g_irq_worker_running{false}; - uint32_t g_enabled_intc_mask = 0xFFFFFFFFu; - uint32_t g_enabled_dmac_mask = 0xFFFFFFFFu; + std::thread g_irq_worker_thread; // joinable worker handle so stopInterruptWorker() can join it + // See Interrupt.h: read from the IRQ worker thread without holding + // g_irq_handler_mutex (the dispatch*HandlersForCause call sites read this + // while evaluating a function argument), so it must be atomic rather than + // mutex-protected like the rest of the handler bookkeeping. + std::atomic g_enabled_intc_mask{0xFFFFFFFFu}; + std::atomic g_enabled_dmac_mask{0xFFFFFFFFu}; + constexpr uint32_t kAsyncHandlerStackSize = 0x4000u; // 16 KB, one pool slot uint64_t g_vsync_tick_counter = 0u; VSyncFlagRegistration g_vsync_registration{}; } @@ -77,13 +84,14 @@ namespace ps2_syscalls static uint32_t getAsyncHandlerStackTop(PS2Runtime *runtime) { - constexpr uint32_t kAsyncHandlerStackSize = 0x4000u; + // Only reachable if the callback stack pool is exhausted or runtime is + // null; see kAsyncCallbackFallbackSp for the fallback. thread_local PS2Runtime *s_cachedRuntime = nullptr; thread_local uint32_t s_cachedStackTop = 0u; if (runtime == nullptr) { - return PS2_RAM_SIZE - 0x10u; + return kAsyncCallbackFallbackSp; } if (s_cachedRuntime != runtime || s_cachedStackTop == 0u) @@ -92,10 +100,28 @@ namespace ps2_syscalls s_cachedStackTop = runtime->reserveAsyncCallbackStack(kAsyncHandlerStackSize, 16u); } - return (s_cachedStackTop != 0u) ? s_cachedStackTop : (PS2_RAM_SIZE - 0x10u); - } - - static void dispatchIntcHandlersForCause(uint8_t *rdram, PS2Runtime *runtime, uint32_t cause) + return (s_cachedStackTop != 0u) ? s_cachedStackTop : kAsyncCallbackFallbackSp; + } + + // Unified INTC/DMAC dispatch: collect handlers from `handlerMap` that are + // enabled for `cause`, then run them under AsyncGuestScope. `enabledMask` + // is the per-cause enable bitmask (g_enabled_intc_mask or g_enabled_dmac_mask). + // `tag` is used for exception logging ("INTC" or "DMAC"). + // + // NOTE on guest-memory concurrency: the handler bodies invoked below run on + // the IRQ worker thread (a real, separate host thread — see AsyncGuestScope), + // genuinely in parallel with the main guest fiber executing on the scheduler's + // single guest-executor thread. If a handler and the main guest code both + // touch the same rdram address without the guest itself arranging a lock/ + // semaphore, that is an intentional characteristic of this "IRQ handlers are + // real concurrent workers" design (it mirrors how an interrupt handler + // touching a shared variable requires the GUEST to synchronize, exactly as + // on real hardware) rather than a synchronization bug in the runtime. Do not + // add locking around guest rdram accesses here to silence such reports. + static void dispatchHandlersForCause( + uint8_t *rdram, PS2Runtime *runtime, uint32_t cause, + const std::unordered_map &handlerMap, + uint32_t enabledMask, const char *tag) { if (!rdram || !runtime) { @@ -105,24 +131,16 @@ namespace ps2_syscalls std::vector handlers; { std::lock_guard lock(g_irq_handler_mutex); - if (cause < 32u && (g_enabled_intc_mask & (1u << cause)) == 0u) + if (cause < 32u && (enabledMask & (1u << cause)) == 0u) { return; } - handlers.reserve(g_intcHandlers.size()); - for (const auto &[id, info] : g_intcHandlers) + handlers.reserve(handlerMap.size()); + for (const auto &kv : handlerMap) { - (void)id; - if (!info.enabled) - { - continue; - } - if (info.cause != cause) - { - continue; - } - if (info.handler == 0u) + const IrqHandlerInfo &info = kv.second; + if (!info.enabled || info.cause != cause || info.handler == 0u) { continue; } @@ -132,150 +150,105 @@ namespace ps2_syscalls { return a.order < b.order; }); } - for (const IrqHandlerInfo &info : handlers) + auto runHandlers = [&](uint32_t stackTop) { - if (!runtime->hasFunction(info.handler)) + for (const IrqHandlerInfo &info : handlers) { - if (cause == kIntcVblankStart) + if (!runtime->hasFunction(info.handler)) { - PS2_IF_AGRESSIVE_LOGS({ - static std::atomic s_missingHandlerLogCount{0u}; - const uint32_t logIndex = s_missingHandlerLogCount.fetch_add(1u, std::memory_order_relaxed); - if (logIndex < 32u) - { - auto flags = std::cout.flags(); - std::cout << "[INTC:missing] cause=" << cause - << " handler=0x" << std::hex << info.handler - << std::dec - << " id=" << info.id - << std::endl; - std::cout.flags(flags); - } - }); + continue; } - continue; - } - try - { - R5900Context irqCtx{}; - SET_GPR_U32(&irqCtx, 28, info.gp); - SET_GPR_U32(&irqCtx, 29, getAsyncHandlerStackTop(runtime)); - SET_GPR_U32(&irqCtx, 31, 0u); - SET_GPR_U32(&irqCtx, 4, cause); - SET_GPR_U32(&irqCtx, 5, info.arg); - SET_GPR_U32(&irqCtx, 6, 0u); - SET_GPR_U32(&irqCtx, 7, 0u); - irqCtx.pc = info.handler; - - while (irqCtx.pc != 0u && runtime && !runtime->isStopRequested()) + try { - PS2Runtime::RecompiledFunction step = runtime->lookupFunction(irqCtx.pc); - if (!step) + R5900Context irqCtx{}; + SET_GPR_U32(&irqCtx, 28, info.gp); + SET_GPR_U32(&irqCtx, 29, stackTop); + SET_GPR_U32(&irqCtx, 31, 0u); + SET_GPR_U32(&irqCtx, 4, cause); + SET_GPR_U32(&irqCtx, 5, info.arg); + SET_GPR_U32(&irqCtx, 6, 0u); + SET_GPR_U32(&irqCtx, 7, 0u); + irqCtx.pc = info.handler; + + while (irqCtx.pc != 0u && runtime && !runtime->isStopRequested()) { - break; + PS2Runtime::RecompiledFunction step = runtime->lookupFunction(irqCtx.pc); + if (!step) + { + break; + } + step(rdram, &irqCtx, runtime); } - // Interrupt handlers must be able to preempt a guest thread that is - // spinning on interrupt-produced state, such as a vblank counter. - step(rdram, &irqCtx, runtime); } - } - catch (const ThreadExitException &) - { - } - catch (const std::exception &e) - { - static uint32_t warnCount = 0; - if (warnCount < 8u) + catch (const ThreadExitException &) { - std::cerr << "[INTC] handler 0x" << std::hex << info.handler - << " threw exception: " << e.what() << std::dec << std::endl; - ++warnCount; + } + catch (const std::exception &e) + { + static uint32_t warnCount = 0; + if (warnCount < 8u) + { + std::cerr << "[" << tag << "] handler 0x" << std::hex << info.handler + << " threw exception: " << e.what() << std::dec << std::endl; + ++warnCount; + } } } - } - } + }; - void dispatchDmacHandlersForCause(uint8_t *rdram, PS2Runtime *runtime, uint32_t cause) - { - if (!rdram || !runtime) + // Nothing to run: skip the token borrow and the scratch reservation. + if (handlers.empty()) { return; } - std::vector handlers; + // The INTC path only ever runs on the IRQ worker host thread, but the + // DMAC path is ALSO reachable synchronously from guest code: + // sceSifSetDma (Stubs/SIF.cpp), sceDmaSend (Stubs/Helpers/Support.h), + // and drainCompletedDmacHandlers (ps2_runtime.cpp) all call + // dispatchDmacHandlersForCause inline while servicing a guest syscall, + // i.e. while the calling fiber IS the guest execution slot. + // AsyncGuestScope's async_guest_begin() aborts by design if invoked + // from the guest executor thread (that guard exists to catch host + // workers mistakenly running there) - so only borrow the token when + // this call is NOT already running on the guest thread. + if (ps2sched::is_guest_thread()) + { + // Inline on the calling fiber: a handler body can yield at a + // back-edge, so a shared stack would be clobbered by another fiber + // dispatching inline (or by a nested inline DMAC on this same + // fiber). Reserve a fresh per-invocation scratch stack — NOT the + // async pool: a per-fiber pool reservation would exhaust the pool's + // 32 slots and fall back onto a shared stack, reintroducing the bug. + // Handlers in this loop run sequentially (never nested), so one + // reservation for the whole dispatch is correct; guestFree fires + // here when the dispatch (and any yield inside it) completes. + GuestScratchStack handlerStack(runtime, kAsyncHandlerStackSize); + runHandlers(handlerStack.valid() ? handlerStack.top() + : getAsyncHandlerStackTop(runtime)); + } + else { - std::lock_guard lock(g_irq_handler_mutex); - if (cause < 32u && (g_enabled_dmac_mask & (1u << cause)) == 0u) - { - return; - } - - handlers.reserve(g_dmacHandlers.size()); - for (const auto &[id, info] : g_dmacHandlers) - { - (void)id; - if (!info.enabled) - { - continue; - } - if (info.cause != cause) - { - continue; - } - if (info.handler == 0u) - { - continue; - } - handlers.push_back(info); - } - std::sort(handlers.begin(), handlers.end(), [](const IrqHandlerInfo &a, const IrqHandlerInfo &b) - { return a.order < b.order; }); + // Host worker (INTC) under AsyncGuestScope: cannot yield, one + // callback runs to completion, so the per-OS-thread pool cache is + // safe, including its failure fallback (kAsyncCallbackFallbackSp + // when the pool is exhausted or runtime is null). + AsyncGuestScope guestScope; // token released on any exit path + runHandlers(getAsyncHandlerStackTop(runtime)); } + } - for (const IrqHandlerInfo &info : handlers) - { - if (!runtime->hasFunction(info.handler)) - { - continue; - } + static void dispatchIntcHandlersForCause(uint8_t *rdram, PS2Runtime *runtime, uint32_t cause) + { + dispatchHandlersForCause(rdram, runtime, cause, g_intcHandlers, + g_enabled_intc_mask.load(std::memory_order_acquire), "INTC"); + } - try - { - R5900Context irqCtx{}; - SET_GPR_U32(&irqCtx, 28, info.gp); - SET_GPR_U32(&irqCtx, 29, getAsyncHandlerStackTop(runtime)); - SET_GPR_U32(&irqCtx, 31, 0u); - SET_GPR_U32(&irqCtx, 4, cause); - SET_GPR_U32(&irqCtx, 5, info.arg); - SET_GPR_U32(&irqCtx, 6, 0u); - SET_GPR_U32(&irqCtx, 7, 0u); - irqCtx.pc = info.handler; - - while (irqCtx.pc != 0u && runtime && !runtime->isStopRequested()) - { - PS2Runtime::RecompiledFunction step = runtime->lookupFunction(irqCtx.pc); - if (!step) - { - break; - } - step(rdram, &irqCtx, runtime); - } - } - catch (const ThreadExitException &) - { - } - catch (const std::exception &e) - { - static uint32_t warnCount = 0; - if (warnCount < 8u) - { - std::cerr << "[DMAC] handler 0x" << std::hex << info.handler - << " threw exception: " << e.what() << std::dec << std::endl; - ++warnCount; - } - } - } + void dispatchDmacHandlersForCause(uint8_t *rdram, PS2Runtime *runtime, uint32_t cause) + { + dispatchHandlersForCause(rdram, runtime, cause, g_dmacHandlers, + g_enabled_dmac_mask.load(std::memory_order_acquire), "DMAC"); } static void updateGsCsrFieldForVSync(PS2Runtime *runtime, uint64_t tickValue) @@ -307,9 +280,23 @@ namespace ps2_syscalls tickValue = ++g_vsync_tick_counter; } - g_vsync_cv.notify_all(); + // Wake all guest threads waiting for the next vsync tick. + // Called from the IRQ worker (a non-guest host thread). Use the identity- + // validated wakeup so a recycled tid cannot deliver this tick to the wrong + // fiber: each entry carries the parking fiber's generation token. + wakeWaiters(g_vsync_flag_mutex, g_vsync_waitList); updateGsCsrFieldForVSync(runtime, tickValue); + // These two writes race, by design, with guest/test code polling the + // same rdram words on another thread (real PS2 hardware exposes the + // vsync flag/tick exactly this way: a plain memory-mapped word the + // application polls, with no interlock). TSan reports this as a data + // race because it is one under the C++ memory model, but adding a + // mutex here would not match real hardware semantics and would still + // require the poller to take the same lock (it can't: it's guest code + // reading raw rdram, or test code via readGuestU32/readGuestU64, + // neither of which we can — or should — change). Left unsynchronized + // intentionally; do not wrap in a lock. if (reg.flagAddr != 0u) { writeGuestU32NoThrow(rdram, reg.flagAddr, 1u); @@ -378,11 +365,16 @@ namespace ps2_syscalls return; } + if (g_irq_worker_thread.joinable()) + { + g_irq_worker_thread.join(); + } + g_irq_worker_stop.store(false, std::memory_order_release); g_irq_worker_running.store(true, std::memory_order_release); try { - std::thread(interruptWorkerMain, rdram, runtime).detach(); + g_irq_worker_thread = std::thread(interruptWorkerMain, rdram, runtime); } catch (...) { @@ -401,35 +393,138 @@ namespace ps2_syscalls return g_vsync_tick_counter; } + void signalInterruptWorkerStop() + { + // Signal-only: no join (see Interrupt.h). The worker observes the stop + // flag on its next CV wait / loop check and exits; scheduler_shutdown() + // performs the join on the main thread via stopInterruptWorker(). + g_irq_worker_stop.store(true, std::memory_order_release); + g_irq_worker_cv.notify_all(); + } + void stopInterruptWorker() { g_irq_worker_stop.store(true, std::memory_order_release); g_irq_worker_cv.notify_all(); - std::unique_lock lock(g_irq_worker_mutex); - g_irq_worker_cv.wait_for(lock, std::chrono::milliseconds(500), []() - { return !g_irq_worker_running.load(std::memory_order_acquire); }); - g_vsync_cv.notify_all(); + + // Join the worker to a clean stop; it checks g_irq_worker_stop on both + // its CV wait and its while-condition, so it exits promptly. We must + // NOT hold g_irq_worker_mutex while joining (the worker takes that + // mutex on its CV wait — joining under it would deadlock). + std::thread workerToJoin; + { + std::lock_guard lock(g_irq_worker_mutex); + if (g_irq_worker_thread.joinable()) + { + workerToJoin = std::move(g_irq_worker_thread); + } + } + if (workerToJoin.joinable()) + { + workerToJoin.join(); + } + + // Wake any guest threads waiting on vsync during shutdown. + wakeWaiters(g_vsync_flag_mutex, g_vsync_waitList); } uint64_t WaitForNextVSyncTick(uint8_t *rdram, PS2Runtime *runtime) { ensureInterruptWorkerRunning(rdram, runtime); - std::unique_lock lock(g_vsync_flag_mutex); - uint64_t current = g_vsync_tick_counter; - uint64_t result = current; - waitWithGuestExecutionReleasedUntilUnlocked( - runtime, - lock, - [&]() + + // Opaque identity of the fiber that is about to park. Non-fiber host + // workers get token 0 and never publish to the wait-list. + const uint64_t selfToken = ps2sched::current_fiber_token(); + const bool onFiber = (selfToken != 0u); + + // Snapshot the tick we are waiting to advance past. A non-fiber worker + // never publishes to g_vsync_waitList (it cannot park), so it cannot + // rely on a single wake meaning "signalVSyncFlag ran"; it must instead + // poll this counter directly until it changes. + uint64_t entryTick; + { + std::lock_guard lock(g_vsync_flag_mutex); + entryTick = g_vsync_tick_counter; + } + + if (!onFiber) + { + // Non-fiber path (IRQ/alarm worker calling back into vsync wait, or + // a borrowed host worker): loop with bounded exponential backoff + // (mirrors WaitSema/WaitEventFlag's non-fiber Mesa loop) until + // g_vsync_tick_counter actually advances past entryTick or runtime + // stop is requested. A single block_current()+backoff step could + // return before the IRQ worker's next tick fired, handing back the + // SAME tick the caller already observed instead of truly waiting + // for the next one. + NonFiberBackoff nfBackoff; + for (;;) { - g_vsync_cv.wait(lock, [current, runtime]() - { return g_vsync_tick_counter > current || (runtime != nullptr && runtime->isStopRequested()); }); - }, - [&]() + const ps2sched::BlockResult br = ps2sched::block_current(); + nfBackoff.step(br); + + std::lock_guard lock(g_vsync_flag_mutex); + if (g_vsync_tick_counter != entryTick) + { + return g_vsync_tick_counter; + } + if (runtime == nullptr || runtime->isStopRequested()) + { + return g_vsync_tick_counter; + } + } + } + + // Publish under g_vsync_flag_mutex; arm_park after the lock is + // released so g_sched_mutex is never nested under it. + { + std::lock_guard lock(g_vsync_flag_mutex); + g_vsync_waitList.emplace_back(g_currentThreadId, selfToken); + } + ps2sched::arm_park(); + + // Block the current fiber; signalVSyncFlag calls the validated wakeup + // from the IRQ worker thread to wake us. + const ps2sched::BlockResult br = ps2sched::block_current(); + + // A fiber woken from a real park (Parked) may have been woken by + // scheduler_shutdown / TerminateThread rather than a vsync tick. If so, + // unwind instead of returning a tick value. Mirrors WaitSema's terminate + // check after wake. + if (br == ps2sched::BlockResult::Parked) + { + std::shared_ptr info = lookupThreadInfo(g_currentThreadId); + if (info && info->terminated.load()) { - result = g_vsync_tick_counter; - }); - return result; + // Drop our wait-list entry before unwinding so a recycled tid + // cannot inherit a stale token. + { + std::lock_guard clLock(g_vsync_flag_mutex); + auto &wl = g_vsync_waitList; + auto it = std::find_if(wl.begin(), wl.end(), + [selfToken](const std::pair &e) + { return e.second == selfToken; }); + if (it != wl.end()) wl.erase(it); + } + throw ThreadExitException(); + } + } + + // If we were woken by something other than a vsync tick (shutdown, + // TerminateThread, or a wakeup during the parking window), signalVSyncFlag + // never drained us, so our entry is still queued. Remove it by fiber-token + // identity (NOT by tid, which can recycle). A real vsync wake already + // swapped us out, so this erase is a harmless no-op on that path. + std::lock_guard lock(g_vsync_flag_mutex); + auto &wl = g_vsync_waitList; + auto it = std::find_if(wl.begin(), wl.end(), + [selfToken](const std::pair &e) + { return e.second == selfToken; }); + if (it != wl.end()) + { + wl.erase(it); + } + return g_vsync_tick_counter; } void WaitVSyncTick(uint8_t *rdram, PS2Runtime *runtime) @@ -459,19 +554,10 @@ namespace ps2_syscalls const uint32_t cause = getRegU32(ctx, 4); if (cause < 32u) { - std::lock_guard lock(g_irq_handler_mutex); - g_enabled_intc_mask |= (1u << cause); - } - if (cause == kIntcVblankStart || cause == kIntcVblankEnd) - { - PS2_IF_AGRESSIVE_LOGS({ - static std::atomic s_enableLogCount{0u}; - const uint32_t logIndex = s_enableLogCount.fetch_add(1u, std::memory_order_relaxed); - if (logIndex < 32u) - { - RUNTIME_LOG("[EnableIntc] cause=" << cause); - } - }); + // Atomic RMW: g_enabled_intc_mask is read lock-free from the IRQ + // worker thread (see declaration comment), so it must also be + // written lock-free rather than under g_irq_handler_mutex. + g_enabled_intc_mask.fetch_or(1u << cause, std::memory_order_acq_rel); } setReturnS32(ctx, KE_OK); } @@ -486,19 +572,7 @@ namespace ps2_syscalls const uint32_t cause = getRegU32(ctx, 4); if (cause < 32u) { - std::lock_guard lock(g_irq_handler_mutex); - g_enabled_intc_mask &= ~(1u << cause); - } - if (cause == kIntcVblankStart || cause == kIntcVblankEnd) - { - PS2_IF_AGRESSIVE_LOGS({ - static std::atomic s_disableLogCount{0u}; - const uint32_t logIndex = s_disableLogCount.fetch_add(1u, std::memory_order_relaxed); - if (logIndex < 32u) - { - RUNTIME_LOG("[DisableIntc] cause=" << cause); - } - }); + g_enabled_intc_mask.fetch_and(~(1u << cause), std::memory_order_acq_rel); } setReturnS32(ctx, KE_OK); } @@ -528,27 +602,6 @@ namespace ps2_syscalls g_intcHandlers[handlerId] = info; } - if (info.cause == kIntcVblankStart) - { - PS2_IF_AGRESSIVE_LOGS({ - static std::atomic s_addHandlerLogCount{0u}; - const uint32_t logIndex = s_addHandlerLogCount.fetch_add(1u, std::memory_order_relaxed); - if (logIndex < 32u) - { - auto flags = std::cout.flags(); - std::cout << "[AddIntcHandler] cause=" << info.cause - << " handler=0x" << std::hex << info.handler - << " arg=0x" << info.arg - << " gp=0x" << info.gp - << " sp=0x" << info.sp - << std::dec - << " id=" << handlerId - << std::endl; - std::cout.flags(flags); - } - }); - } - ensureInterruptWorkerRunning(rdram, runtime); setReturnS32(ctx, handlerId); } @@ -674,8 +727,8 @@ namespace ps2_syscalls const uint32_t cause = getRegU32(ctx, 4); if (cause < 32u) { - std::lock_guard lock(g_irq_handler_mutex); - g_enabled_dmac_mask |= (1u << cause); + // See EnableIntc: g_enabled_dmac_mask is atomic for the same reason. + g_enabled_dmac_mask.fetch_or(1u << cause, std::memory_order_acq_rel); } setReturnS32(ctx, KE_OK); } @@ -690,8 +743,7 @@ namespace ps2_syscalls const uint32_t cause = getRegU32(ctx, 4); if (cause < 32u) { - std::lock_guard lock(g_irq_handler_mutex); - g_enabled_dmac_mask &= ~(1u << cause); + g_enabled_dmac_mask.fetch_and(~(1u << cause), std::memory_order_acq_rel); } setReturnS32(ctx, KE_OK); } diff --git a/ps2xRuntime/src/lib/Kernel/Syscalls/Interrupt.h b/ps2xRuntime/src/lib/Kernel/Syscalls/Interrupt.h index eb9ba5f03..fecb9b423 100644 --- a/ps2xRuntime/src/lib/Kernel/Syscalls/Interrupt.h +++ b/ps2xRuntime/src/lib/Kernel/Syscalls/Interrupt.h @@ -1,6 +1,11 @@ #pragma once +#include +#include +#include #include +#include +#include #include "ps2_syscalls.h" namespace ps2_syscalls @@ -17,11 +22,22 @@ namespace ps2_syscalls extern std::mutex g_irq_worker_mutex; extern std::condition_variable g_irq_worker_cv; extern std::mutex g_vsync_flag_mutex; - extern std::condition_variable g_vsync_cv; + // Each entry pairs the guest tid with the parking fiber's identity token + // (from ps2sched::current_fiber_token(), which encodes the fiber's + // generation). signalVSyncFlag delivers the wakeup only to the exact + // fiber that parked, so a recycled tid cannot receive a stale tick. + // Borrowed host workers (g_currentThreadId == -1, token == 0) never park + // here, so every stored entry has a non-zero token. + extern std::vector> g_vsync_waitList; extern std::atomic g_irq_worker_stop; extern std::atomic g_irq_worker_running; - extern uint32_t g_enabled_intc_mask; - extern uint32_t g_enabled_dmac_mask; + extern std::thread g_irq_worker_thread; // joinable worker handle + // Written by Enable/DisableIntc(Dmac) and read by dispatchIntcHandlersForCause / + // dispatchDmacHandlersForCause from the IRQ worker thread. Atomic (rather + // than g_irq_handler_mutex) because the dispatch call sites read the mask + // while evaluating a function argument, i.e. before any lock is taken. + extern std::atomic g_enabled_intc_mask; + extern std::atomic g_enabled_dmac_mask; extern uint64_t g_vsync_tick_counter; extern VSyncFlagRegistration g_vsync_registration; } @@ -30,6 +46,14 @@ namespace ps2_syscalls void EnsureVSyncWorkerRunning(uint8_t *rdram, PS2Runtime *runtime); uint64_t GetCurrentVSyncTick(); void stopInterruptWorker(); + // Signal-only variant: sets the stop flag and wakes the worker but does + // NOT join. For callers on the guest executor thread (a fiber calling + // requestStop): joining there can deadlock against a worker blocked in + // async_guest_begin(), whose wait predicate (g_running_fiber == nullptr) + // cannot become true while the joining fiber is itself the running fiber. + // The join happens later in scheduler_shutdown() on the main thread + // (stopInterruptWorker is idempotent). + void signalInterruptWorkerStop(); uint64_t WaitForNextVSyncTick(uint8_t *rdram, PS2Runtime *runtime); void WaitVSyncTick(uint8_t *rdram, PS2Runtime *runtime); void SetVSyncFlag(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime); diff --git a/ps2xRuntime/src/lib/Kernel/Syscalls/Lifecycle.cpp b/ps2xRuntime/src/lib/Kernel/Syscalls/Lifecycle.cpp index d5a31ea74..f2726bede 100644 --- a/ps2xRuntime/src/lib/Kernel/Syscalls/Lifecycle.cpp +++ b/ps2xRuntime/src/lib/Kernel/Syscalls/Lifecycle.cpp @@ -1,6 +1,7 @@ #include "Common.h" #include "Interrupt.h" #include "Lifecycle.h" +#include "Sync.h" namespace ps2_syscalls { @@ -8,7 +9,23 @@ namespace ps2_syscalls void notifyRuntimeStop() { - stopInterruptWorker(); + // requestStop can be invoked from GUEST context (e.g. the + // unimplemented-function default handler runs on the guest executor + // thread, inside a fiber). Joining a host worker from there can + // deadlock: a worker blocked in async_guest_begin() waits for + // g_running_fiber == nullptr, which cannot happen while the joining + // fiber IS the running fiber. In that case only signal the workers to + // stop; scheduler_shutdown() joins them later on the main thread + // (both stop functions are idempotent). + const bool fromGuestExecutor = ps2sched::is_guest_thread(); + if (fromGuestExecutor) + { + signalInterruptWorkerStop(); + } + else + { + stopInterruptWorker(); + } { std::lock_guard lock(g_irq_handler_mutex); g_intcHandlers.clear(); @@ -24,7 +41,7 @@ namespace ps2_syscalls g_vsync_tick_counter = 0u; } - std::vector> threads; + std::vector>> threads; threads.reserve(32); { std::lock_guard lock(g_thread_map_mutex); @@ -32,22 +49,48 @@ namespace ps2_syscalls { if (entry.second) { - threads.push_back(entry.second); + threads.emplace_back(entry.first, entry.second); } } g_threads.clear(); g_nextThreadId = 2; // Reserve id 1 for main thread. } - g_currentThreadId = 1; + // Consume each reaped thread's activeCounted token rather than + // force-storing g_activeThreads to 0: a worker fiber that is still + // genuinely running is a concurrent, legitimate consumer of that SAME + // token through on_fiber_exit, since it holds the same ThreadInfo + // shared_ptr as this snapshot. The atomic exchange arbitrates the two + // sides so only the winner performs the matching fetch_sub. If + // on_fiber_exit wins, it decrements and this loop's exchange finds the + // token already false (no-op); if this loop wins, on_fiber_exit later + // sees the token gone (or a null ThreadInfo, since g_threads was + // cleared above) and skips its own decrement. + for (const auto &[tid, threadInfo] : threads) + { + if (threadInfo->activeCounted.exchange(false, std::memory_order_acq_rel)) + { + g_activeThreads.fetch_sub(1, std::memory_order_release); + } + } + // -1 is the "not a guest fiber" sentinel: a host thread running + // notifyRuntimeStop must never be mistaken for a real guest thread id + // by arm_park / wait-list operations. Do NOT clobber it when called + // from guest context, though — there g_currentThreadId identifies the + // still-running fiber on the executor thread, and zapping it would + // corrupt that fiber's identity for the rest of its unwind. + if (!fromGuestExecutor) + { + g_currentThreadId = -1; + } - for (const auto &threadInfo : threads) + for (const auto &[tid, threadInfo] : threads) { { std::lock_guard lock(threadInfo->m); threadInfo->forceRelease = true; threadInfo->terminated = true; } - threadInfo->cv.notify_all(); + ps2sched::request_terminate(tid); } std::vector> semas; @@ -66,7 +109,7 @@ namespace ps2_syscalls } for (const auto &sema : semas) { - sema->cv.notify_all(); + wakeWaiters(sema->m, sema->waitList); } std::vector> eventFlags; @@ -85,9 +128,22 @@ namespace ps2_syscalls } for (const auto &eventFlag : eventFlags) { - eventFlag->cv.notify_all(); + wakeWaiters(eventFlag->m, eventFlag->waitList); } + // Stop and JOIN the alarm worker BEFORE clearing alarms, so no callback can + // fire against rdram/runtime that is about to be destroyed. From guest + // context, signal-only (see the comment at the top of this function): + // the worker re-checks the stop flag before invoking any callback, and + // scheduler_shutdown() performs the join on the main thread. + if (fromGuestExecutor) + { + ps2_syscalls::signalAlarmWorkerStop(); + } + else + { + ps2_syscalls::stopAlarmWorker(); + } { std::lock_guard lock(g_alarm_mutex); g_alarms.clear(); @@ -104,13 +160,4 @@ namespace ps2_syscalls } } - void joinAllGuestHostThreads() - { - joinAllHostThreads(); - } - - void detachAllGuestHostThreads() - { - detachAllHostThreads(); - } } diff --git a/ps2xRuntime/src/lib/Kernel/Syscalls/Lifecycle.h b/ps2xRuntime/src/lib/Kernel/Syscalls/Lifecycle.h index 749ca1e29..e60b25b38 100644 --- a/ps2xRuntime/src/lib/Kernel/Syscalls/Lifecycle.h +++ b/ps2xRuntime/src/lib/Kernel/Syscalls/Lifecycle.h @@ -5,6 +5,4 @@ namespace ps2_syscalls { void notifyRuntimeStop(); - void joinAllGuestHostThreads(); - void detachAllGuestHostThreads(); } diff --git a/ps2xRuntime/src/lib/Kernel/Syscalls/RPC.cpp b/ps2xRuntime/src/lib/Kernel/Syscalls/RPC.cpp index 4f0fc7fb9..3dcd9da5e 100644 --- a/ps2xRuntime/src/lib/Kernel/Syscalls/RPC.cpp +++ b/ps2xRuntime/src/lib/Kernel/Syscalls/RPC.cpp @@ -611,18 +611,28 @@ namespace ps2_syscalls } bool signaled = false; + int wokenTid = 0; + uint64_t wokenToken = 0; { std::lock_guard lock(sema->m); if (!sema->deleted && sema->count < sema->maxCount) { sema->count++; signaled = true; + if (!sema->waitList.empty()) + { + wokenTid = sema->waitList.front().first; + wokenToken = sema->waitList.front().second; + sema->waitList.erase(sema->waitList.begin()); + } } } - if (signaled) + if (wokenTid != 0) { - sema->cv.notify_one(); + // Called from the RPC worker (non-guest host thread). Use the + // validated variant to avoid stale wakeups if the tid was recycled. + ps2sched::enqueue_external_wakeup_validated(wokenTid, wokenToken); } return signaled; } diff --git a/ps2xRuntime/src/lib/Kernel/Syscalls/Sync.cpp b/ps2xRuntime/src/lib/Kernel/Syscalls/Sync.cpp index 8243c48b1..2c41b76f3 100644 --- a/ps2xRuntime/src/lib/Kernel/Syscalls/Sync.cpp +++ b/ps2xRuntime/src/lib/Kernel/Syscalls/Sync.cpp @@ -1,5 +1,6 @@ #include "Common.h" #include "Sync.h" +#include "ps2_fiber.h" namespace ps2_syscalls { @@ -191,7 +192,6 @@ namespace ps2_syscalls g_semas.emplace(id, info); } - RUNTIME_LOG("[CreateSema] id=" << id << " init=" << init << " max=" << max); setReturnS32(ctx, id); } @@ -212,11 +212,21 @@ namespace ps2_syscalls g_semas.erase(it); } + // Collect all waiting threads, then wake each with token validation. + std::vector> waiters; { - std::lock_guard lock(sema->m); + std::lock_guard lk(sema->m); sema->deleted = true; + waiters.swap(sema->waitList); + } + for (const auto& [tid, token] : waiters) + { + ps2sched::enqueue_external_wakeup_validated(tid, token); + } + if (!waiters.empty()) + { + ps2sched::maybe_yield(); } - sema->cv.notify_all(); // PS2 EE BIOS returns sid on success. setReturnS32(ctx, sid); @@ -239,12 +249,10 @@ namespace ps2_syscalls // PS2 EE BIOS returns sid on success; KE_SEMA_OVF overrides on overflow. int ret = sid; - int beforeCount = 0; - int afterCount = 0; - bool wokeWaiter = false; + int wokenTid = 0; + uint64_t wokenToken = 0; { - std::lock_guard lock(sema->m); - beforeCount = sema->count; + std::unique_lock lock(sema->m); if (sema->count >= sema->maxCount) { ret = KE_SEMA_OVF; @@ -252,28 +260,23 @@ namespace ps2_syscalls else { sema->count++; - wokeWaiter = sema->waiters > 0; - sema->cv.notify_one(); + // Pop one waiter and wake it. + if (!sema->waitList.empty()) + { + wokenTid = sema->waitList.front().first; + wokenToken = sema->waitList.front().second; + sema->waitList.erase(sema->waitList.begin()); + } } - afterCount = sema->count; + lock.unlock(); } - - static std::atomic s_signalSemaLogs{0}; - const uint32_t sigLog = s_signalSemaLogs.fetch_add(1, std::memory_order_relaxed); - if (sigLog < 256u) + if (wokenTid != 0) { - RUNTIME_LOG("[SignalSema] tid=" << g_currentThreadId - << " sid=" << sid - << " count=" << beforeCount << "->" << afterCount - << " ret=" << ret - << std::endl); + ps2sched::enqueue_external_wakeup_validated(wokenTid, wokenToken); + ps2sched::maybe_yield(); } setReturnS32(ctx, ret); - if (wokeWaiter) - { - yieldGuestExecutionAfterWake(runtime); - } } void iSignalSema(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) @@ -291,31 +294,47 @@ namespace ps2_syscalls return; } - auto info = ensureCurrentThreadInfo(ctx); - throwIfTerminated(info); - + // Borrowed host workers (g_currentThreadId == -1) are not PS2 threads; + // they must never create or mutate a g_threads entry (all such workers + // would alias tid -1 and race each other). Guest identity is keyed off + // g_currentThreadId, NOT fiber-ness: a non-fiber host thread carrying a + // real guest tid (e.g. a raw std::thread standing in for a guest thread + // in tests, or the main thread — which the runtime ctor seeds with tid 1) + // still needs ThreadInfo bookkeeping so ReleaseWaitThread/ReferSemaStatus + // can observe and target it. onFiber remains the gate for the parts of + // this call that are genuinely scheduler-only (arm_park / the fiber Mesa + // wait below): a non-fiber, even with a valid info, cannot be parked on + // the fiber scheduler and still takes the bounded-backoff retry loop + // (block_current handles it). info == nullptr (borrowed worker) drives + // the non-bookkeeping retry path below; all ThreadInfo accesses are + // already guarded by `if (info)`. + const bool onFiber = (ps2fiber_current() != nullptr); + std::shared_ptr info = + (g_currentThreadId != -1) ? ensureCurrentThreadInfo(ctx) : nullptr; + throwIfTerminated(info); // throwIfTerminated is null-safe std::unique_lock lock(sema->m); // PS2 EE BIOS returns sid on success. int ret = sid; - int countAfter = sema->count; - bool terminated = false; - bool consumed = false; - if (sema->count == 0) + if (sema->count > 0) { - static std::atomic s_waitSemaBlockLogs{0}; - const uint32_t blockLog = s_waitSemaBlockLogs.fetch_add(1, std::memory_order_relaxed); - if (blockLog < 256u) - { - RUNTIME_LOG("[WaitSema:block] tid=" << g_currentThreadId - << " sid=" << sid - << " pc=0x" << std::hex << ctx->pc - << " ra=0x" << getRegU32(ctx, 31) - << std::dec - << std::endl); - } - + sema->count--; + } + else + { + // Slow path: wait until we can consume a permit (Mesa monitor semantics). + // Re-check count > 0 after each wake; re-block if stolen by PollSema. + NonFiberBackoff nfBackoff; // unused for fibers; ramps for borrowed workers + + // Establish wait state ONCE, here, at the point the thread first + // transitions into THS_WAIT/THS_WAITSUSPEND for this call. This is + // also the only safe place to clear forceRelease: ReleaseWaitThread + // only ever sets it while observing status == WAIT/WAITSUSPEND, and + // that transition (plus the clear) happens atomically under info->m + // here, so a stale true left over from a prior, unrelated wait on + // this ThreadInfo cannot leak into this wait, and nothing can race + // the clear itself. if (info) { std::lock_guard tLock(info->m); @@ -325,80 +344,165 @@ namespace ps2_syscalls info->forceRelease = false; } - sema->waiters++; - waitWithGuestExecutionReleasedUntilUnlocked( - runtime, - lock, - [&]() - { - sema->cv.wait(lock, [&]() - { - const bool forced = info ? info->forceRelease.load() : false; - const bool isTerminated = info ? info->terminated.load() : false; - return sema->count > 0 || sema->deleted || forced || isTerminated; - }); - }, - [&]() + for (;;) + { + // Retry-check: status stays WAIT/WAITSUSPEND for the *entire* + // loop (it is only reset to RUN/SUSPEND once, after the loop, + // below), so ReleaseWaitThread may legitimately fire and set + // forceRelease at any point while we're spinning here - + // including right after the post-wake check below found it + // false and decided to re-block. Clearing forceRelease + // unconditionally here would silently clobber that pending + // release, so consume it and take the same released-exit path + // as the post-wake check. + if (info) { - sema->waiters--; - if (sema->deleted) + bool release = false; + { + std::lock_guard tLock(info->m); + if (info->forceRelease) + { + info->forceRelease = false; + release = true; + } + } + if (release) { - ret = KE_WAIT_DELETE; + ret = KE_RELEASE_WAIT; + break; } + } + + // sema->waiters is a plain count of every thread genuinely + // blocked here — fiber, non-fiber-with-identity, or fully + // borrowed (g_currentThreadId == -1) — so ReferSemaStatus's + // wait_threads reports accurately for ALL of them (PS2 EE BIOS + // semantics: any blocked thread counts, not just ones we can + // individually target for wakeup). It carries no identity, so + // unlike waitList it has no tid-aliasing hazard and is safe to + // bump unconditionally. + sema->waiters++; + + // Only a thread with a valid guest identity (info != nullptr, + // i.e. g_currentThreadId != -1) publishes itself to the + // *wait-list* (as opposed to the waiters count above), whether + // or not it is a real fiber. This is what lets ReleaseWaitThread's + // target lookup and SignalSema/DeleteSema's targeted wakeup see a + // non-fiber host thread carrying a real guest tid (e.g. a raw + // std::thread standing in for a guest thread in tests). Off-fiber, + // current_fiber_token() is 0; SignalSema/DeleteSema's + // enqueue_external_wakeup_validated drops token==0 entries, so + // publishing here is harmless for such a waiter — it doesn't need + // the wake, it re-polls via the backoff loop below. A fully + // borrowed host worker (info==nullptr) would alias every other + // borrowed worker as tid -1 if it published a waitList entry, so + // it skips that (but is still counted in sema->waiters above) and + // relies on the block_current() non-fiber retry loop (which + // re-checks sema->count below). + if (info) + { + // Publish to the wait-list under sema->m so a SignalSema is + // serialized against our enqueue. arm_park() runs AFTER we + // drop sema->m, so g_sched_mutex is never nested under an + // object mutex. A SignalSema that fires in the publish/arm + // window sees g_running_fiber == this fiber and records + // wake_pending (consumed by block_current). + sema->waitList.emplace_back(g_currentThreadId, ps2sched::current_fiber_token()); + } + + // Drop sema->m BEFORE any scheduler operation. + lock.unlock(); + + if (onFiber) + { + ps2sched::arm_park(); + } - if (info) + const ps2sched::BlockResult br = ps2sched::block_current(); + + // Non-fiber (borrowed host worker) path: bounded exponential backoff + // so a never-satisfied condition cannot busy-spin the CPU. + if (br == ps2sched::BlockResult::NonFiberOwner || + br == ps2sched::BlockResult::NonFiberNoTok) + { + nfBackoff.step(br); + } + + // === Woke up here === + lock.lock(); + + // Any thread that published a wait-list entry above (info != + // nullptr) must remove it here if still present (SignalSema/ + // DeleteSema may have already popped it). A borrowed host worker + // (info == nullptr) never published a wait-list entry, so there + // is nothing to remove there — but every thread (identified or + // borrowed) decrements sema->waiters to match its unconditional + // increment above. + if (info) + { + auto& wl = sema->waitList; + auto it = std::find_if(wl.begin(), wl.end(), + [](const std::pair& e){ return e.first == g_currentThreadId; }); + if (it != wl.end()) wl.erase(it); + } + sema->waiters--; + + // Wake reasons that abort the wait without consuming a permit: + if (sema->deleted) + { + ret = KE_WAIT_DELETE; + break; + } + + if (info) + { + bool release = false; { std::lock_guard tLock(info->m); - terminated = info->terminated.load(); - info->status = (info->suspendCount > 0) ? THS_SUSPEND : THS_RUN; - info->waitType = TSW_NONE; - info->waitId = 0; if (info->forceRelease) { info->forceRelease = false; - ret = KE_RELEASE_WAIT; + release = true; } } - - if (!terminated && ret == sid && sema->count > 0) + if (release) { - sema->count--; - consumed = true; + ret = KE_RELEASE_WAIT; + break; } - countAfter = sema->count; - }); - } + } - if (terminated) - { - throw ThreadExitException(); - } + if (info && info->terminated.load()) + { + throw ThreadExitException(); + } - if (!consumed && lock.owns_lock()) - { - if (ret == sid && sema->count > 0) - { - sema->count--; - consumed = true; + // Mesa re-check: only consume if a permit is actually available. + // If PollSema stole the count between SignalSema's unlock and our + // re-lock, count == 0 and we loop to re-block. + if (sema->count > 0) + { + sema->count--; + // PS2 EE BIOS returns sid on success. + ret = sid; + break; + } + // Spurious wake or permit stolen — loop and block again. } - countAfter = sema->count; } - static std::atomic s_waitSemaWakeLogs{0}; - const uint32_t wakeLog = s_waitSemaWakeLogs.fetch_add(1, std::memory_order_relaxed); - if (wakeLog < 256u) + // Reset thread status on all non-exception exit paths (fast path, slow path success, + // and error breaks). The throw-ThreadExitException path unwinds without reaching here. + if (info) { - RUNTIME_LOG("[WaitSema:wake] tid=" << g_currentThreadId - << " sid=" << sid - << " ret=" << ret - << " count=" << countAfter - << std::endl); + std::lock_guard tLock(info->m); + info->status = (info->suspendCount > 0) ? THS_SUSPEND : THS_RUN; + info->waitType = TSW_NONE; + info->waitId = 0; } - if (lock.owns_lock()) - { - lock.unlock(); - } - waitWhileSuspended(info, runtime); + + lock.unlock(); + waitWhileSuspended(info); setReturnS32(ctx, ret); } @@ -471,14 +575,20 @@ namespace ps2_syscalls void CreateEventFlag(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) { uint32_t paramAddr = getRegU32(ctx, 4); // $a0 - const uint32_t *param = reinterpret_cast(getConstMemPtr(rdram, paramAddr)); auto info = std::make_shared(); - if (param) - { - info->attr = param[0]; - info->option = param[1]; - info->initBits = param[2]; + if (paramAddr != 0u) + { + // Read attr / option / initBits with full RDRAM range checks (matching + // CreateSema), instead of dereferencing a raw guest pointer. A field + // whose word falls outside RDRAM stays at its default of 0. + uint32_t attr = 0u, option = 0u, initBits = 0u; + readGuestU32Safe(rdram, paramAddr + 0u, attr); + readGuestU32Safe(rdram, paramAddr + 4u, option); + readGuestU32Safe(rdram, paramAddr + 8u, initBits); + info->attr = attr; + info->option = option; + info->initBits = initBits; info->bits = info->initBits; } @@ -513,11 +623,20 @@ namespace ps2_syscalls return; } + std::vector> evfWaiters; { - std::lock_guard lock(info->m); + std::lock_guard lk(info->m); info->deleted = true; + evfWaiters.swap(info->waitList); + } + for (const auto& [tid, token] : evfWaiters) + { + ps2sched::enqueue_external_wakeup_validated(tid, token); + } + if (!evfWaiters.empty()) + { + ps2sched::maybe_yield(); } - info->cv.notify_all(); setReturnS32(ctx, 0); } @@ -538,31 +657,25 @@ namespace ps2_syscalls return; } - uint32_t newBits = 0u; - bool hadWaiters = false; + std::vector> setEvfWaiters; { - std::lock_guard lock(info->m); + std::unique_lock lock(info->m); info->bits |= bits; - newBits = info->bits; - hadWaiters = info->waiters > 0; + // Collect all waiting threads (they'll re-evaluate the condition on wake). + // Don't clear waitList yet — each waiter removes itself on wake. + setEvfWaiters = info->waitList; + lock.unlock(); } - static std::atomic s_setEventFlagLogs{0}; - const uint32_t setLog = s_setEventFlagLogs.fetch_add(1, std::memory_order_relaxed); - if (setLog < 256u) + for (const auto& [tid, token] : setEvfWaiters) { - RUNTIME_LOG("[SetEventFlag] tid=" << g_currentThreadId - << " eid=" << eid - << " bits=0x" << std::hex << bits - << " newBits=0x" << newBits - << std::dec << std::endl); + ps2sched::enqueue_external_wakeup_validated(tid, token); } - info->cv.notify_all(); - setReturnS32(ctx, 0); - if (hadWaiters) + if (!setEvfWaiters.empty()) { - yieldGuestExecutionAfterWake(runtime); + ps2sched::maybe_yield(); } + setReturnS32(ctx, 0); } void iSetEventFlag(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) @@ -585,7 +698,6 @@ namespace ps2_syscalls std::lock_guard lock(info->m); info->bits &= bits; } - info->cv.notify_all(); setReturnS32(ctx, KE_OK); } @@ -629,7 +741,14 @@ namespace ps2_syscalls return; } - auto tInfo = ensureCurrentThreadInfo(ctx); + // Guest identity is keyed off g_currentThreadId, NOT fiber-ness (see + // WaitSema for the full rationale). onFiber remains the gate only for + // the genuinely scheduler-only parts below (arm_park / the fiber Mesa + // loop); a non-fiber thread with a valid tInfo still takes the + // bounded-backoff retry loop. + const bool onFiber = (ps2fiber_current() != nullptr); + std::shared_ptr tInfo = + (g_currentThreadId != -1) ? ensureCurrentThreadInfo(ctx) : nullptr; throwIfTerminated(tInfo); int ret = KE_OK; @@ -650,80 +769,182 @@ namespace ps2_syscalls return (info->bits & waitBits) == waitBits; }; - bool waitedWithGuestRelease = false; - bool terminated = false; - uint32_t bitsAfter = info->bits; - - auto finishEventFlagWaitLocked = [&]() + if (!satisfied()) { - if (ret == KE_OK && info->deleted) - { - ret = KE_WAIT_DELETE; - } - - if (ret == KE_OK) + if (!onFiber) { - if (resBitsPtr) + // Non-fiber path (IRQ/alarm worker, or — since tInfo is now keyed + // off g_currentThreadId rather than onFiber — any host thread + // carrying a real guest tid, e.g. a raw std::thread standing in + // for a guest thread in tests): Mesa-style re-block loop, + // mirroring WaitSema's non-fiber handling. We cannot park a host + // worker on the fiber scheduler, so bounded exponential backoff + // (NonFiberBackoff) stands in for arm_park/block_current's real + // parking, re-checking satisfied() under info->m every iteration. + // + // A thread with a valid guest identity (tInfo != nullptr) also + // publishes itself to info->waitList and keeps its ThreadInfo + // status in THS_WAIT/THS_WAITSUSPEND for the duration, so + // ReferEventFlagStatus's numThreads count and ReleaseWaitThread's + // target lookup both see it — matching WaitSema. Off-fiber, + // current_fiber_token() is 0; SetEventFlag's wake fan-out + // tolerates token==0 (enqueue_external_wakeup_validated drops it): + // this waiter doesn't need the wake, it re-polls every backoff + // step. A true borrowed worker (tInfo == nullptr) never publishes + // and relies solely on satisfied() re-checks, same as before. + if (tInfo) { - *resBitsPtr = info->bits; + std::lock_guard tLock(tInfo->m); + tInfo->status = (tInfo->suspendCount > 0) ? THS_WAITSUSPEND : THS_WAIT; + tInfo->waitType = TSW_EVENT; + tInfo->waitId = eid; + tInfo->forceRelease = false; } - if (mode & WEF_CLEAR_ALL) + NonFiberBackoff nfBackoff; + for (;;) { - info->bits = 0; + if (tInfo) + { + bool release = false; + { + std::lock_guard tLock(tInfo->m); + if (tInfo->forceRelease) + { + tInfo->forceRelease = false; + release = true; + } + } + if (release) + { + ret = KE_RELEASE_WAIT; + break; + } + } + + // info->waiters counts every thread genuinely blocked here — + // fiber, non-fiber-with-identity, or fully borrowed + // (g_currentThreadId == -1) — so ReferEventFlagStatus's + // numThreads and the EA_MULTI exclusivity gate above are + // accurate for all of them. It carries no identity, so unlike + // waitList it has no tid-aliasing hazard and is safe to bump + // unconditionally (mirrors WaitSema's sema->waiters). + info->waiters++; + if (tInfo) + { + info->waitList.emplace_back(g_currentThreadId, ps2sched::current_fiber_token()); + } + + lock.unlock(); + const ps2sched::BlockResult br = ps2sched::block_current(); + nfBackoff.step(br); + lock.lock(); + + if (tInfo) + { + auto &wl = info->waitList; + auto it = std::find_if(wl.begin(), wl.end(), + [](const std::pair& e){ return e.first == g_currentThreadId; }); + if (it != wl.end()) wl.erase(it); + } + info->waiters--; + + if (tInfo) + { + std::lock_guard tLock(tInfo->m); + if (tInfo->forceRelease) + { + tInfo->forceRelease = false; + ret = KE_RELEASE_WAIT; + } + } + + if (tInfo && tInfo->terminated.load()) + { + throw ThreadExitException(); + } + + if (ret != KE_OK || info->deleted || satisfied()) + break; } - else if (mode & WEF_CLEAR) + + if (tInfo) { - info->bits &= ~waitBits; + std::lock_guard tLock(tInfo->m); + tInfo->status = (tInfo->suspendCount > 0) ? THS_SUSPEND : THS_RUN; + tInfo->waitType = TSW_NONE; + tInfo->waitId = 0; } } + else + { + // Mesa-style re-block loop for fibers. SetEventFlag wakes ALL + // waiters including those that waited AND bits that were only + // partially satisfied; re-check the condition on every wake and + // re-publish to the wait-list if still unsatisfied. + for (;;) + { + // Update thread wait state, unless a ReleaseWaitThread already + // raced in since our last check. Status flips back to RUN/SUSPEND + // at the bottom of every iteration (below) and only becomes WAIT + // again right here, so in principle no legitimate forceRelease + // can be pending at this exact point (see WaitSema for the + // general hazard this guards against). Check-and-consume first + // anyway, defensively: if it is somehow already set, take the + // same released-exit path as the post-wake check below instead + // of blindly clearing it and re-parking. + bool release = false; + if (tInfo) + { + std::lock_guard tLock(tInfo->m); + if (tInfo->forceRelease) + { + tInfo->forceRelease = false; + release = true; + } + else + { + tInfo->status = (tInfo->suspendCount > 0) ? THS_WAITSUSPEND : THS_WAIT; + tInfo->waitType = TSW_EVENT; + tInfo->waitId = eid; + } + } + if (release) + { + ret = KE_RELEASE_WAIT; + break; + } - bitsAfter = info->bits; - }; + // Publish under info->m; arm_park after unlock (no nested locks). + info->waiters++; + info->waitList.emplace_back(g_currentThreadId, ps2sched::current_fiber_token()); - if (!satisfied()) - { - static std::atomic s_waitEventBlockLogs{0}; - const uint32_t evBlockLog = s_waitEventBlockLogs.fetch_add(1, std::memory_order_relaxed); - if (evBlockLog < 256u) - { - RUNTIME_LOG("[WaitEventFlag:block] tid=" << g_currentThreadId - << " eid=" << eid - << " waitBits=0x" << std::hex << waitBits - << " mode=0x" << mode - << " bits=0x" << info->bits - << " pc=0x" << ctx->pc - << " ra=0x" << getRegU32(ctx, 31) - << std::dec - << std::endl); - } + // UNLOCK before any scheduler operation. + lock.unlock(); + ps2sched::arm_park(); + const ps2sched::BlockResult br = ps2sched::block_current(); + if (br != ps2sched::BlockResult::Parked && + br != ps2sched::BlockResult::WokenInWindow) + { + std::fprintf(stderr, + "FATAL [WaitEventFlag]: unexpected NonFiber result in fiber Mesa loop\n"); + std::terminate(); + } - if (tInfo) - { - std::lock_guard tLock(tInfo->m); - tInfo->status = (tInfo->suspendCount > 0) ? THS_WAITSUSPEND : THS_WAIT; - tInfo->waitType = TSW_EVENT; - tInfo->waitId = eid; - tInfo->forceRelease = false; - } + lock.lock(); - info->waiters++; - waitedWithGuestRelease = true; - waitWithGuestExecutionReleasedUntilUnlocked( - runtime, - lock, - [&]() - { - info->cv.wait(lock, satisfied); - }, - [&]() - { - info->waiters--; + // Remove self from wait-list (re-added at top of loop if not satisfied). + { + auto &wl = info->waitList; + auto it = std::find_if(wl.begin(), wl.end(), + [](const std::pair& e){ return e.first == g_currentThreadId; }); + if (it != wl.end()) wl.erase(it); + info->waiters--; + } if (tInfo) { std::lock_guard tLock(tInfo->m); - terminated = tInfo->terminated.load(); tInfo->status = (tInfo->suspendCount > 0) ? THS_SUSPEND : THS_RUN; tInfo->waitType = TSW_NONE; tInfo->waitId = 0; @@ -734,44 +955,46 @@ namespace ps2_syscalls } } - if (!terminated) + if (tInfo && tInfo->terminated.load()) { - finishEventFlagWaitLocked(); + throw ThreadExitException(); } - else - { - bitsAfter = info->bits; - } - }); - } - if (terminated) - { - throw ThreadExitException(); - } + // Exit if forceRelease/terminate/delete broke us out, OR + // if the condition is now satisfied. + if (ret != KE_OK || info->deleted || satisfied()) + break; - if (!waitedWithGuestRelease) - { - finishEventFlagWaitLocked(); + // Spurious wake (SetEventFlag set only a subset of AND bits): + // loop back to re-publish and re-block. + } + } } - static std::atomic s_waitEventWakeLogs{0}; - const uint32_t evWakeLog = s_waitEventWakeLogs.fetch_add(1, std::memory_order_relaxed); - if (evWakeLog < 256u) + if (ret == KE_OK && info->deleted) { - RUNTIME_LOG("[WaitEventFlag:wake] tid=" << g_currentThreadId - << " eid=" << eid - << " ret=" << ret - << " bits=0x" << std::hex << bitsAfter - << std::dec - << std::endl); + ret = KE_WAIT_DELETE; } - if (lock.owns_lock()) + if (ret == KE_OK) { - lock.unlock(); + if (resBitsPtr) + { + *resBitsPtr = info->bits; + } + + if (mode & WEF_CLEAR_ALL) + { + info->bits = 0; + } + else if (mode & WEF_CLEAR) + { + info->bits &= ~waitBits; + } } - waitWhileSuspended(tInfo, runtime); + + lock.unlock(); + waitWhileSuspended(tInfo); setReturnS32(ctx, ret); } @@ -894,21 +1117,159 @@ namespace ps2_syscalls ReferEventFlagStatus(rdram, ctx, runtime); } + static void alarmWorkerMain() + { + g_currentThreadId = -1; // host worker, not a fiber + for (;;) + { + std::shared_ptr readyAlarm; + { + std::unique_lock lock(g_alarm_mutex); + while (!readyAlarm) + { + if (g_alarm_stop_flag.load(std::memory_order_acquire)) + return; // stop requested -> exit, fire nothing more + if (g_alarms.empty()) + { + g_alarm_cv.wait(lock); + continue; + } + + auto nextIt = std::min_element( + g_alarms.begin(), g_alarms.end(), + [](const auto &a, const auto &b) + { return a.second->dueAt < b.second->dueAt; }); + if (nextIt == g_alarms.end()) + { + g_alarm_cv.wait(lock); + continue; + } + + const auto now = std::chrono::steady_clock::now(); + // Copy the deadline out of the AlarmInfo BEFORE waiting. + // condition_variable::wait_until() releases `lock` internally + // while it sleeps, and CancelAlarm/ReleaseAlarm only need + // g_alarm_mutex to erase (and destroy) this same AlarmInfo. A + // reference into `nextIt->second->dueAt` held across that + // unlocked window would dangle the instant CancelAlarm's erase + // runs concurrently — this local copy is ours, not the map's. + const auto deadline = nextIt->second->dueAt; + if (deadline > now) + { + g_alarm_cv.wait_until(lock, deadline); + continue; + } + + readyAlarm = nextIt->second; + g_alarms.erase(nextIt); + } + } + + // After dropping the lock, re-check stop before touching rdram. + // If stop was requested we must NOT invoke the callback (rdram / + // runtime may be torn down). + if (g_alarm_stop_flag.load(std::memory_order_acquire)) + return; + + if (!readyAlarm || !readyAlarm->runtime || !readyAlarm->rdram || + !readyAlarm->handler) + continue; + if (!readyAlarm->runtime->hasFunction(readyAlarm->handler)) + continue; + + try + { + constexpr uint32_t kAlarmCallbackStackSize = 0x4000u; + thread_local PS2Runtime *s_alarmStackRuntime = nullptr; + thread_local uint32_t s_alarmStackTop = 0u; + if (s_alarmStackRuntime != readyAlarm->runtime || s_alarmStackTop == 0u) + { + s_alarmStackRuntime = readyAlarm->runtime; + s_alarmStackTop = readyAlarm->runtime->reserveAsyncCallbackStack( + kAlarmCallbackStackSize, 16u); + } + + R5900Context callbackCtx{}; + setRegU32(&callbackCtx, 28, readyAlarm->gp); + // Failure fallback: see kAsyncCallbackFallbackSp. + setRegU32(&callbackCtx, 29, + (s_alarmStackTop != 0u) ? s_alarmStackTop + : kAsyncCallbackFallbackSp); + setRegU32(&callbackCtx, 31, 0); + setRegU32(&callbackCtx, 4, static_cast(readyAlarm->id)); + setRegU32(&callbackCtx, 5, static_cast(readyAlarm->ticks)); + setRegU32(&callbackCtx, 6, readyAlarm->commonArg); + setRegU32(&callbackCtx, 7, 0); + callbackCtx.pc = readyAlarm->handler; + + PS2Runtime::RecompiledFunction func = + readyAlarm->runtime->lookupFunction(readyAlarm->handler); + { + AsyncGuestScope guestScope; // token released even if func throws + func(readyAlarm->rdram, &callbackCtx, readyAlarm->runtime); + } + } + catch (const ThreadExitException &) + { + } + catch (const std::exception &e) + { + static int alarmExceptionLogs = 0; + if (alarmExceptionLogs < 8) + { + std::cerr << "[SetAlarm] callback exception: " << e.what() << std::endl; + ++alarmExceptionLogs; + } + } + } + } + + void ensureAlarmWorkerRunning() + { + std::lock_guard lock(g_alarm_mutex); + if (!g_alarm_worker_running) + { + g_alarm_stop_flag.store(false, std::memory_order_release); + g_alarm_thread = std::thread(alarmWorkerMain); + g_alarm_worker_running = true; + } + } + + void signalAlarmWorkerStop() + { + { + std::lock_guard lock(g_alarm_mutex); + if (!g_alarm_worker_running) + return; // never started or already stopped + } + g_alarm_stop_flag.store(true, std::memory_order_release); + g_alarm_cv.notify_all(); + // No join, and g_alarm_worker_running stays true: the worker re-checks + // the stop flag before invoking any callback, and scheduler_shutdown()'s + // stopAlarmWorker() performs the join on the main thread (see Sync.h). + } + + void stopAlarmWorker() + { + { + std::lock_guard lock(g_alarm_mutex); + if (!g_alarm_worker_running) + return; // never started or already stopped + } + g_alarm_stop_flag.store(true, std::memory_order_release); + g_alarm_cv.notify_all(); + if (g_alarm_thread.joinable()) + g_alarm_thread.join(); // wait for the worker to fully exit + std::lock_guard lock(g_alarm_mutex); + g_alarm_worker_running = false; + } + void SetAlarm(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) { uint16_t ticks = static_cast(getRegU32(ctx, 4) & 0xFFFFu); uint32_t handler = getRegU32(ctx, 5); uint32_t arg = getRegU32(ctx, 6); - static int logCount = 0; - if (logCount < 5) - { - RUNTIME_LOG("[SetAlarm] ticks=" << ticks - << " handler=0x" << std::hex << handler - << " arg=0x" << arg << std::dec << std::endl); - ++logCount; - } - if (!runtime || !handler || !runtime->hasFunction(handler)) { setReturnS32(ctx, KE_ERROR); diff --git a/ps2xRuntime/src/lib/Kernel/Syscalls/Sync.h b/ps2xRuntime/src/lib/Kernel/Syscalls/Sync.h index 0d66da1bc..2c68ab3f4 100644 --- a/ps2xRuntime/src/lib/Kernel/Syscalls/Sync.h +++ b/ps2xRuntime/src/lib/Kernel/Syscalls/Sync.h @@ -32,4 +32,18 @@ namespace ps2_syscalls void iCancelAlarm(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime); void ReleaseAlarm(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime); void iReleaseAlarm(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime); + + // Alarm worker lifecycle. ensureAlarmWorkerRunning() lazily starts a + // joinable worker thread. stopAlarmWorker() requests stop, wakes it, and + // joins it. stopAlarmWorker() is called from notifyRuntimeStop(). + void ensureAlarmWorkerRunning(); + void stopAlarmWorker(); + // Signal-only variant: sets the stop flag and wakes the worker but does + // NOT join. For callers on the guest executor thread (a fiber calling + // requestStop): joining there can deadlock against the alarm worker + // blocked in AsyncGuestScope/async_guest_begin(), whose wait predicate + // (g_running_fiber == nullptr) cannot become true while the joining fiber + // is itself the running fiber. The join happens later in + // scheduler_shutdown() on the main thread (stopAlarmWorker is idempotent). + void signalAlarmWorkerStop(); } diff --git a/ps2xRuntime/src/lib/Kernel/Syscalls/System.cpp b/ps2xRuntime/src/lib/Kernel/Syscalls/System.cpp index e6ec46663..4e289a10c 100644 --- a/ps2xRuntime/src/lib/Kernel/Syscalls/System.cpp +++ b/ps2xRuntime/src/lib/Kernel/Syscalls/System.cpp @@ -1,5 +1,6 @@ #include "Common.h" #include "System.h" +#include "ps2_syscall_override_state.h" namespace ps2_syscalls { @@ -435,7 +436,17 @@ namespace ps2_syscalls const uint32_t overridePc = ctx->pc; const uint32_t overrideRa = getRegU32(ctx, 31); - thread_local std::vector s_activeSyscallOverrides; + // Reentrancy guard scoped to the CALLING guest context. On a fiber (the + // N=1 executor) this resolves to the fiber's OWN stack, so a fiber parked + // mid-override never marks the syscall active for another fiber, and the + // pop_back below always removes THIS fiber's entry (push/pop is LIFO on + // one fiber's call chain). Host workers and direct non-fiber callers + // (e.g. the kernel-test main thread) use a PERSISTENT per-OS-thread + // fallback: each runs its override chain to completion without yielding, + // so a per-OS-thread stack is exactly right and still catches genuine + // single-context self-recursion (the 0x83 recursive-override case). + SyscallOverrideStack &overrideState = ps2sched::current_active_syscall_overrides(); + std::vector &s_activeSyscallOverrides = overrideState.active; if (std::find(s_activeSyscallOverrides.begin(), s_activeSyscallOverrides.end(), syscallNumber) != s_activeSyscallOverrides.end()) { static std::atomic s_reentrantLogs{0u}; diff --git a/ps2xRuntime/src/lib/Kernel/Syscalls/Thread.cpp b/ps2xRuntime/src/lib/Kernel/Syscalls/Thread.cpp index 7807fc767..7940401bc 100644 --- a/ps2xRuntime/src/lib/Kernel/Syscalls/Thread.cpp +++ b/ps2xRuntime/src/lib/Kernel/Syscalls/Thread.cpp @@ -1,5 +1,6 @@ #include "Common.h" #include "Thread.h" +#include "ps2_scheduler_internal.h" namespace ps2_syscalls { @@ -15,26 +16,6 @@ namespace ps2_syscalls } } - static void notifyThreadWaitObject(int waitType, int waitId) - { - if (waitType == TSW_SEMA) - { - auto sema = lookupSemaInfo(waitId); - if (sema) - { - sema->cv.notify_all(); - } - } - else if (waitType == TSW_EVENT) - { - auto eventFlag = lookupEventFlagInfo(waitId); - if (eventFlag) - { - eventFlag->cv.notify_all(); - } - } - } - static void runExitHandlersForThread(int tid, uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) { if (!runtime || !ctx) @@ -68,6 +49,66 @@ namespace ps2_syscalls } } + // ----------------------------------------------------------------------- + // on_fiber_exit — called by fiber_trampoline (via g_fiber_exit_hook) + // after dispatchLoop returns. Runs exit handlers and resets ThreadInfo. + // ----------------------------------------------------------------------- + static void on_fiber_exit(int tid, uint8_t* rdram, R5900Context* ctx, PS2Runtime* runtime) + { + auto info = lookupThreadInfo(tid); + + runExitHandlersForThread(tid, rdram, ctx, runtime); + + uint32_t detachedAutoStack = 0; + if (info) { + std::lock_guard lock(info->m); + info->started = false; + info->status = THS_DORMANT; + info->waitType = TSW_NONE; + info->waitId = 0; + info->wakeupCount = 0; + info->suspendCount = 0; + info->forceRelease = false; + info->terminated = false; + } + + bool stillRegistered = false; + { + std::lock_guard lock(g_thread_map_mutex); + stillRegistered = (g_threads.find(tid) != g_threads.end()); + } + if (!stillRegistered && info) { + std::lock_guard lock(info->m); + if (info->ownsStack && info->stack != 0) { + detachedAutoStack = info->stack; + info->stack = 0; + info->stackSize = 0; + info->ownsStack = false; + } + } + if (detachedAutoStack != 0 && runtime) { + runtime->guestFree(detachedAutoStack); + } + + // Consume this thread's active-thread token exactly once. If `info` is + // null the g_threads entry was already removed by whoever also took the + // token (notifyRuntimeStop reaping residual guest threads, or + // ExitDeleteThread erasing its own entry), so this exit must NOT + // decrement again. When `info` is present the atomic exchange is the + // sole arbiter: a concurrent notifyRuntimeStop() holding the same + // shared_ptr races here and exactly one side wins the true->false + // transition and performs the single fetch_sub. + if (info && info->activeCounted.exchange(false, std::memory_order_acq_rel)) + g_activeThreads.fetch_sub(1, std::memory_order_release); + } + + static std::once_flag s_fiber_exit_hook_once; + static void ensureFiberExitHookRegistered() { + std::call_once(s_fiber_exit_hook_once, [](){ + g_fiber_exit_hook = on_fiber_exit; + }); + } + void FlushCache(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) { setReturnS32(ctx, KE_OK); @@ -211,13 +252,6 @@ namespace ps2_syscalls g_threads[id] = info; } - RUNTIME_LOG("[CreateThread] id=" << id - << " entry=0x" << std::hex << info->entry - << " stack=0x" << info->stack - << " size=0x" << info->stackSize - << " gp=0x" << info->gp - << " prio=" << std::dec << info->priority << std::endl); - setReturnS32(ctx, id); } @@ -293,7 +327,8 @@ namespace ps2_syscalls if (!runtime || !runtime->hasFunction(info->entry)) { - std::cerr << "[StartThread] entry 0x" << std::hex << info->entry << std::dec << " is not registered" << std::endl; + std::cerr << "[StartThread] entry 0x" << std::hex << info->entry << std::dec + << " is not registered" << std::endl; setReturnS32(ctx, KE_ERROR); return; } @@ -303,8 +338,6 @@ namespace ps2_syscalls return; } - joinHostThreadById(tid); - const uint32_t callerSp = getRegU32(ctx, 29); const uint32_t callerGp = getRegU32(ctx, 28); @@ -332,224 +365,78 @@ namespace ps2_syscalls { info->stack = autoStack; info->ownsStack = true; - RUNTIME_LOG("[StartThread] id=" << tid - << " auto-stack=0x" << std::hex << autoStack - << " size=0x" << info->stackSize << std::dec << std::endl); } } - if (info->stack != 0 && info->stackSize == 0) { - // Some games leave size zero in the thread param even though a stack - // buffer is supplied; use a conservative default instead of caller SP. info->stackSize = 0x800u; } } - g_activeThreads.fetch_add(1, std::memory_order_relaxed); - try + uint32_t threadSp = callerSp; + if (info->stack) { - std::thread worker([=]() mutable - { - { - std::string name = "PS2Thread_" + std::to_string(tid); - ThreadNaming::SetCurrentThreadName(name); - } - R5900Context threadCtxCopy{}; - R5900Context *threadCtx = &threadCtxCopy; - - { - std::lock_guard lock(info->m); - info->status = THS_RUN; - } - - uint32_t threadSp = callerSp; - if (info->stack) - { - const uint32_t stackSize = (info->stackSize != 0) ? info->stackSize : 0x800u; - threadSp = (info->stack + stackSize) & ~0xFu; - } - uint32_t threadGp = info->gp; - const uint32_t normalizedGp = threadGp & 0x1FFFFFFFu; - if (threadGp == 0 || normalizedGp < 0x10000u || normalizedGp >= PS2_RAM_SIZE) - { - threadGp = callerGp; - } - - SET_GPR_U32(threadCtx, 29, threadSp); - SET_GPR_U32(threadCtx, 28, threadGp); - SET_GPR_U32(threadCtx, 4, info->arg); - SET_GPR_U32(threadCtx, 31, 0); - threadCtx->pc = info->entry; - - g_currentThreadId = tid; - - RUNTIME_LOG("[StartThread] id=" << tid - << " entry=0x" << std::hex << info->entry - << " sp=0x" << GPR_U32(threadCtx, 29) - << " gp=0x" << GPR_U32(threadCtx, 28) - << " arg=0x" << info->arg << std::dec << std::endl); - - bool exited = false; - try - { - uint32_t lastPc = 0xFFFFFFFFu; - uint32_t samePcCount = 0; - constexpr uint32_t kSamePcYieldMask = 0xFFu; - constexpr uint32_t kSamePcWarnInterval = 0x20000u; - uint64_t stepCount = 0u; - - while (runtime && !runtime->isStopRequested()) - { - ++stepCount; - if (info->terminated.load(std::memory_order_relaxed)) - { - throw ThreadExitException(); - } - - waitWhileSuspended(info, runtime); - - const uint32_t pc = threadCtx->pc; - info->currentPc.store(pc, std::memory_order_relaxed); - if (pc == 0u) - { - break; - } - - if ((stepCount & 0x1FFFFFu) == 0u) - { - RUNTIME_LOG("[StartThread] id=" << tid - << " heartbeat pc=0x" << std::hex << pc - << " ra=0x" << GPR_U32(threadCtx, 31) - << " sp=0x" << GPR_U32(threadCtx, 29) - << " gp=0x" << GPR_U32(threadCtx, 28) - << std::dec << std::endl); - } - - if (pc == lastPc) - { - ++samePcCount; - if ((samePcCount & kSamePcYieldMask) == 0u) - { - std::this_thread::sleep_for(std::chrono::milliseconds(1)); - } - if (samePcCount > kSamePcWarnInterval) - { - // If a thread is spinning for an extremely long time (e.g. idle thread), - // force a 1ms sleep to prevent host CPU starvation. - if ((samePcCount % (kSamePcWarnInterval * 8u)) == 0u) - { - std::this_thread::sleep_for(std::chrono::milliseconds(1)); - } - else if ((samePcCount % (kSamePcWarnInterval)) == 0u) - { - std::this_thread::yield(); - } - } - } - else - { - samePcCount = 0; - lastPc = pc; - } - - PS2Runtime::RecompiledFunction step = runtime->lookupFunction(pc); - if (!step) - { - std::cerr << "[StartThread] id=" << tid << " missing function for pc=0x" - << std::hex << pc << std::dec << std::endl; - throw ThreadExitException(); - } - { - PS2Runtime::GuestExecutionScope guestExecution(runtime); - step(rdram, threadCtx, runtime); - } - } - } - catch (const ThreadExitException &) - { - exited = true; - } - catch (const std::exception &e) - { - std::cerr << "[StartThread] id=" << tid << " exception: " << e.what() << std::endl; - } - - if (!exited) - { - RUNTIME_LOG("[StartThread] id=" << tid << " returned (pc=0x" - << std::hex << threadCtx->pc << std::dec << ")" << std::endl); - } + const uint32_t stackSize = (info->stackSize != 0) ? info->stackSize : 0x800u; + threadSp = (info->stack + stackSize) & ~0xFu; + } + uint32_t threadGp = info->gp; + const uint32_t normalizedGp = threadGp & 0x1FFFFFFFu; + if (threadGp == 0 || normalizedGp < 0x10000u || normalizedGp >= PS2_RAM_SIZE) + { + threadGp = callerGp; + } - runExitHandlersForThread(tid, rdram, threadCtx, runtime); + ensureFiberExitHookRegistered(); + // Mint this thread's active-thread token. Order matters: bump the + // counter FIRST, then publish the token with release. A consumer that + // observes activeCounted==true is therefore guaranteed to also see the + // matching +1, so the exchange/fetch_sub in a consumer can never run + // ahead of this fetch_add and transiently drive g_activeThreads negative. + g_activeThreads.fetch_add(1, std::memory_order_relaxed); + info->activeCounted.store(true, std::memory_order_release); - uint32_t detachedAutoStack = 0; + // Create the fiber and enqueue it Ready. Throws on allocation failure or if the scheduler is shutting down; the catch below reports it as KE_NO_MEMORY. + try + { + ps2sched::create_fiber(tid, + info->currentPriority > 0 ? info->currentPriority + : static_cast(info->priority), + info->entry, threadSp, threadGp, arg, runtime, rdram); + } + catch (const std::exception& e) + { + // Undo the g_activeThreads increment and reset thread state. Consume + // the token we just minted; the exchange guards against a concurrent + // notifyRuntimeStop() that reaped this same ThreadInfo. + if (info->activeCounted.exchange(false, std::memory_order_acq_rel)) + g_activeThreads.fetch_sub(1, std::memory_order_release); { std::lock_guard lock(info->m); info->started = false; - info->status = THS_DORMANT; - info->waitType = TSW_NONE; - info->waitId = 0; - info->wakeupCount = 0; - info->suspendCount = 0; - info->forceRelease = false; - info->terminated = false; - } - - bool stillRegistered = false; - { - std::lock_guard lock(g_thread_map_mutex); - stillRegistered = (g_threads.find(tid) != g_threads.end()); + info->status = THS_DORMANT; } - if (!stillRegistered) - { - // ExitDeleteThread removes the record immediately; reclaim auto stack here. - std::lock_guard lock(info->m); - if (info->ownsStack && info->stack != 0) - { - detachedAutoStack = info->stack; - info->stack = 0; - info->stackSize = 0; - info->ownsStack = false; - } - } - - if (detachedAutoStack != 0 && runtime) - { - runtime->guestFree(detachedAutoStack); - } - - // Notify anybody waiting for termination (like TerminateThread) - info->cv.notify_all(); - - g_activeThreads.fetch_sub(1, std::memory_order_relaxed); }); - registerHostThread(tid, std::move(worker)); + std::cerr << "[StartThread] create_fiber failed: " << e.what() << std::endl; + setReturnS32(ctx, KE_NO_MEMORY); + return; } - catch (const std::exception &e) + + // Update ThreadInfo status to READY now that the fiber is enqueued. + // Guard against a race where TerminateThread ran between create_fiber + // and here: if the thread was already transitioned to THS_DORMANT by + // the terminate path, do not revert it to THS_READY. { - std::cerr << "[StartThread] failed to spawn host thread for tid=" << tid << ": " << e.what() << std::endl; - g_activeThreads.fetch_sub(1, std::memory_order_relaxed); std::lock_guard lock(info->m); - info->started = false; - info->status = THS_DORMANT; - info->waitType = TSW_NONE; - info->waitId = 0; - info->wakeupCount = 0; - info->suspendCount = 0; - info->forceRelease = false; - info->terminated = false; - setReturnS32(ctx, KE_ERROR); - return; + if (info->status != THS_DORMANT) + info->status = THS_READY; } + // Yield if the new thread has higher or equal priority. + ps2sched::maybe_yield(); setReturnS32(ctx, KE_OK); } void ExitThread(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) { - RUNTIME_LOG("[ExitThread] Game requested thread exit! PC=0x" << std::hex << ctx->pc - << " RA=0x" << getRegU32(ctx, 31) << std::dec << " tid=" << g_currentThreadId << std::endl); - runExitHandlersForThread(g_currentThreadId, rdram, ctx, runtime); auto info = ensureCurrentThreadInfo(ctx); if (info) @@ -561,19 +448,12 @@ namespace ps2_syscalls info->waitId = 0; info->wakeupCount = 0; } - if (info) - { - info->cv.notify_all(); - } throw ThreadExitException(); } void ExitDeleteThread(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) { int tid = g_currentThreadId; - RUNTIME_LOG("[ExitDeleteThread] Game requested thread exit & delete! PC=0x" << std::hex << ctx->pc - << " RA=0x" << getRegU32(ctx, 31) << std::dec << " tid=" << tid << std::endl); - runExitHandlersForThread(tid, rdram, ctx, runtime); auto info = ensureCurrentThreadInfo(ctx); if (info) @@ -585,14 +465,16 @@ namespace ps2_syscalls info->waitId = 0; info->wakeupCount = 0; } - if (info) - { - info->cv.notify_all(); - } { std::lock_guard lock(g_thread_map_mutex); g_threads.erase(tid); } + // We just erased our own g_threads entry, so the on_fiber_exit that runs + // after this throw will see a null ThreadInfo and skip the token consume. + // Consume the active-thread token here instead, so g_activeThreads is + // decremented exactly once for this started thread. + if (info && info->activeCounted.exchange(false, std::memory_order_acq_rel)) + g_activeThreads.fetch_sub(1, std::memory_order_release); throw ThreadExitException(); } @@ -600,7 +482,14 @@ namespace ps2_syscalls { int tid = static_cast(getRegU32(ctx, 4)); if (tid == 0) + { + if (g_currentThreadId == -1) + { + setReturnS32(ctx, KE_ILLEGAL_THID); + return; + } tid = g_currentThreadId; + } auto info = (tid == g_currentThreadId) ? ensureCurrentThreadInfo(ctx) : lookupThreadInfo(tid); if (!info) @@ -609,8 +498,6 @@ namespace ps2_syscalls return; } - int waitType = TSW_NONE; - int waitId = 0; { std::lock_guard lock(info->m); if (info->status == THS_DORMANT) @@ -618,13 +505,9 @@ namespace ps2_syscalls setReturnS32(ctx, KE_DORMANT); return; } - waitType = info->waitType; - waitId = info->waitId; info->terminated = true; info->forceRelease = true; } - info->cv.notify_all(); - notifyThreadWaitObject(waitType, waitId); if (tid == g_currentThreadId) { @@ -633,17 +516,8 @@ namespace ps2_syscalls } else { - // Block until the target thread actually finishes unwinding and becomes dormant. - // Drop the thread mutex before reacquiring GuestExecutionScope to avoid lock inversion. - std::unique_lock lock(info->m); - waitWithGuestExecutionReleasedUntilUnlocked( - runtime, - lock, - [&]() - { - info->cv.wait(lock, [&]() - { return !info->started && info->status == THS_DORMANT; }); - }); + ps2sched::request_terminate(tid); + ps2sched::join_fiber(tid); } setReturnS32(ctx, KE_OK); @@ -653,7 +527,14 @@ namespace ps2_syscalls { int tid = static_cast(getRegU32(ctx, 4)); if (tid == 0) + { + if (g_currentThreadId == -1) + { + setReturnS32(ctx, KE_ILLEGAL_THID); + return; + } tid = g_currentThreadId; + } auto info = (tid == g_currentThreadId) ? ensureCurrentThreadInfo(ctx) : lookupThreadInfo(tid); if (!info) @@ -672,34 +553,27 @@ namespace ps2_syscalls info->suspendCount++; applySuspendStatusLocked(*info); } - info->cv.notify_all(); if (tid == g_currentThreadId) { - std::unique_lock lock(info->m); - bool terminated = false; - waitWithGuestExecutionReleasedUntilUnlocked( - runtime, - lock, - [&]() - { - info->cv.wait(lock, [&]() - { return info->suspendCount == 0 || info->terminated.load(); }); - }, - [&]() - { - terminated = info->terminated.load(); - if (!terminated) - { - info->status = THS_RUN; - } - }); - - if (terminated) + // Drive the scheduler gate through fc->suspendCount via + // suspend_self(), NOT block_current() directly. suspend_self() + // increments FiberContext::suspendCount and parks the fiber; the + // matching ResumeThread -> clear_suspend() zeroes it and re- + // enqueues when it reaches 0. info->suspendCount (incremented above) + // remains the PS2-visible count for status reporting. + if (info->terminated.load()) throw ThreadExitException(); + ps2sched::suspend_self(); // parks until clear_suspend() wakes us + if (info->terminated.load()) throw ThreadExitException(); { - throw ThreadExitException(); + std::lock_guard lock(info->m); + info->status = THS_RUN; } } + else + { + ps2sched::suspend_other(tid); + } setReturnS32(ctx, KE_OK); } @@ -708,7 +582,14 @@ namespace ps2_syscalls { int tid = static_cast(getRegU32(ctx, 4)); if (tid == 0) + { + if (g_currentThreadId == -1) + { + setReturnS32(ctx, KE_ILLEGAL_THID); + return; + } tid = g_currentThreadId; + } auto info = (tid == g_currentThreadId) ? ensureCurrentThreadInfo(ctx) : lookupThreadInfo(tid); if (!info) @@ -742,7 +623,23 @@ namespace ps2_syscalls } } } - info->cv.notify_all(); + + // ThreadInfo::suspendCount is the PS2-visible nesting count. + // FiberContext::suspendCount is the scheduler parking gate. When the + // PS2 count reaches 0 the thread must run again, so force the scheduler + // gate to 0 in one shot (handles nested SuspendThread correctly). + { + int sc; + { + std::lock_guard lock(info->m); + sc = info->suspendCount; + } + if (sc == 0) { + ps2sched::clear_suspend(tid); // fc->suspendCount = 0 + wake if Blocked + ps2sched::maybe_yield(); + } + } + setReturnS32(ctx, KE_OK); } @@ -758,6 +655,11 @@ namespace ps2_syscalls if (tid == 0) // TH_SELF { + if (g_currentThreadId == -1) + { + setReturnS32(ctx, KE_ILLEGAL_THID); + return; + } tid = g_currentThreadId; } @@ -798,18 +700,43 @@ namespace ps2_syscalls void SleepThread(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) { - auto info = ensureCurrentThreadInfo(ctx); - if (!info) - { + // Guest identity is keyed off g_currentThreadId, NOT fiber-ness (see + // WaitSema/WaitEventFlag in Sync.cpp for the full rationale). onFiber + // remains the gate only for the genuinely scheduler-only parts below + // (arm_park / the fiber wait loop's parking); a non-fiber thread + // carrying a real guest tid (g_currentThreadId != -1) still gets + // THS_WAIT bookkeeping via a ThreadInfo, so it is targetable by + // WakeupThread/ReleaseWaitThread, and takes the bounded-backoff retry + // loop below instead of parking, re-checking wakeupCount exactly like + // the fiber path. + const bool onFiber = (ps2fiber_current() != nullptr); + std::shared_ptr info = + (g_currentThreadId != -1) ? ensureCurrentThreadInfo(ctx) : nullptr; + if (onFiber && !info) + { + // A real fiber must have a ThreadInfo; failure to create one is a + // genuine error. setReturnS32(ctx, KE_UNKNOWN_THID); return; } - throwIfTerminated(info); + throwIfTerminated(info); // null-safe int ret = 0; - int wakeupCountAfter = 0; - bool terminated = false; + + if (!info) + { + // Fully borrowed host worker (g_currentThreadId == -1): PS2 + // interrupt context cannot sleep on a PS2 thread it does not own. + // There is no ThreadInfo / wakeupCount to consult. Park-and-retry + // once with bounded backoff, then return OK so the worker does + // not livelock the emulator. + ps2sched::BlockResult br = ps2sched::block_current(); + nonFiberBlockBackoff(br); + setReturnS32(ctx, 0); + return; + } + std::unique_lock lock(info->m); if (info->wakeupCount > 0) @@ -819,82 +746,74 @@ namespace ps2_syscalls info->waitType = TSW_NONE; info->waitId = 0; ret = 0; - wakeupCountAfter = info->wakeupCount; } else { - static std::atomic s_sleepBlockLogs{0}; - const uint32_t sleepBlockLog = s_sleepBlockLogs.fetch_add(1, std::memory_order_relaxed); - if (sleepBlockLog < 256u) - { - RUNTIME_LOG("[SleepThread:block] tid=" << g_currentThreadId - << " pc=0x" << std::hex << ctx->pc - << " ra=0x" << getRegU32(ctx, 31) - << std::dec << std::endl); - } - info->status = THS_WAIT; info->waitType = TSW_SLEEP; info->waitId = 0; info->forceRelease = false; - waitWithGuestExecutionReleasedUntilUnlocked( - runtime, - lock, - [&]() + NonFiberBackoff nfBackoff; // unused for fibers; ramps for non-fiber waiters + + for (;;) + { + // Drop info->m before ANY scheduler operation so g_sched_mutex is + // never nested under info->m. + lock.unlock(); + // Arm on every iteration: block_current() consumes wake_pending, + // so a wake arriving in the new publish/arm window would be missed + // if we skipped re-arming on subsequent iterations. + if (onFiber) { - info->cv.wait(lock, [&]() - { return info->wakeupCount > 0 || info->forceRelease.load() || info->terminated.load(); }); - }, - [&]() + ps2sched::arm_park(); + } + const ps2sched::BlockResult br = ps2sched::block_current(); + + // Non-fiber (but identified) waiter: bounded exponential backoff + // so a never-satisfied condition cannot busy-spin the CPU. + if (br == ps2sched::BlockResult::NonFiberOwner || + br == ps2sched::BlockResult::NonFiberNoTok) { - terminated = info->terminated.load(); - if (terminated) - { - return; - } - - info->status = THS_RUN; - info->waitType = TSW_NONE; - info->waitId = 0; - - if (info->forceRelease.load()) - { - info->forceRelease = false; - ret = KE_RELEASE_WAIT; - } - else - { - if (info->wakeupCount > 0) - { - info->wakeupCount--; - } - ret = 0; - } - wakeupCountAfter = info->wakeupCount; - }); - } - - if (terminated) - { - throw ThreadExitException(); - } + nfBackoff.step(br); + } - static std::atomic s_sleepWakeLogs{0}; - const uint32_t sleepWakeLog = s_sleepWakeLogs.fetch_add(1, std::memory_order_relaxed); - if (sleepWakeLog < 256u) - { - RUNTIME_LOG("[SleepThread:wake] tid=" << g_currentThreadId - << " ret=" << ret - << " wakeupCount=" << wakeupCountAfter - << std::endl); - } + lock.lock(); - if (lock.owns_lock()) - { - lock.unlock(); + // 1. Terminate wins unconditionally (shutdown / TerminateThread). + if (info->terminated.load()) + throw ThreadExitException(); + + // 2. ReleaseWaitThread forced us out of the wait. + if (info->forceRelease.load()) + { + info->forceRelease = false; + ret = KE_RELEASE_WAIT; + break; + } + + // 3. Genuine WakeupThread: a permit is available. + if (info->wakeupCount > 0) + { + --info->wakeupCount; + ret = 0; + break; + } + + // 4. Spurious wake (e.g. ResumeThread / clear_suspend with no + // pending wakeup): stay asleep. Re-affirm wait state and loop. + info->status = THS_WAIT; + info->waitType = TSW_SLEEP; + info->waitId = 0; + } + + info->status = THS_RUN; + info->waitType = TSW_NONE; + info->waitId = 0; } - waitWhileSuspended(info, runtime); + + lock.unlock(); + waitWhileSuspended(info); setReturnS32(ctx, ret); } @@ -919,9 +838,7 @@ namespace ps2_syscalls return; } - int newWakeupCount = 0; - int statusAfter = THS_DORMANT; - bool wokeSleeper = false; + bool wasWaiting = false; { std::lock_guard lock(info->m); if (info->status == THS_DORMANT) @@ -931,6 +848,7 @@ namespace ps2_syscalls } if (info->status == THS_WAIT && info->waitType == TSW_SLEEP) { + wasWaiting = true; if (info->suspendCount > 0) { info->status = THS_SUSPEND; @@ -942,32 +860,20 @@ namespace ps2_syscalls info->waitType = TSW_NONE; info->waitId = 0; info->wakeupCount++; - info->cv.notify_one(); - wokeSleeper = true; } else { info->wakeupCount++; } - newWakeupCount = info->wakeupCount; - statusAfter = info->status; } - static std::atomic s_wakeupLogs{0}; - const uint32_t wakeupLog = s_wakeupLogs.fetch_add(1, std::memory_order_relaxed); - if (wakeupLog < 256u) - { - RUNTIME_LOG("[WakeupThread] tid=" << g_currentThreadId - << " target=" << tid - << " status=" << statusAfter - << " wakeupCount=" << newWakeupCount - << std::endl); + // If the thread was sleeping, make it ready and yield if higher priority. + if (wasWaiting) { + ps2sched::make_ready(tid); + ps2sched::maybe_yield(); } + setReturnS32(ctx, KE_OK); - if (wokeSleeper) - { - yieldGuestExecutionAfterWake(runtime); - } } void iWakeupThread(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) @@ -979,7 +885,14 @@ namespace ps2_syscalls { int tid = static_cast(getRegU32(ctx, 4)); if (tid == 0) + { + if (g_currentThreadId == -1) + { + setReturnS32(ctx, KE_ILLEGAL_THID); + return; + } tid = g_currentThreadId; + } auto info = (tid == g_currentThreadId) ? ensureCurrentThreadInfo(ctx) : lookupThreadInfo(tid); if (!info) @@ -1028,7 +941,14 @@ namespace ps2_syscalls int newPrio = static_cast(getRegU32(ctx, 5)); if (tid == 0) + { + if (g_currentThreadId == -1) + { + setReturnS32(ctx, KE_ILLEGAL_THID); + return; + } tid = g_currentThreadId; + } auto info = (tid == g_currentThreadId) ? ensureCurrentThreadInfo(ctx) : lookupThreadInfo(tid); if (!info) @@ -1058,6 +978,9 @@ namespace ps2_syscalls info->currentPriority = newPrio; } + ps2sched::update_priority(tid, newPrio); + ps2sched::maybe_yield(); + setReturnS32(ctx, KE_OK); } @@ -1068,10 +991,14 @@ namespace ps2_syscalls void RotateThreadReadyQueue(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) { - static int logCount = 0; int prio = static_cast(getRegU32(ctx, 4)); if (prio == 0) { + if (g_currentThreadId == -1) + { + setReturnS32(ctx, KE_ILLEGAL_THID); + return; + } auto current = ensureCurrentThreadInfo(ctx); if (current) { @@ -1079,19 +1006,17 @@ namespace ps2_syscalls prio = (current->currentPriority > 0) ? current->currentPriority : 1; } } - if (logCount < 16) - { - RUNTIME_LOG("[RotateThreadReadyQueue] prio=" << prio); - ++logCount; - } if (prio <= 0 || prio >= 128) { setReturnS32(ctx, KE_ILLEGAL_PRIORITY); return; } + // Rotate the equal-priority group in the fiber run queue. + ps2sched::rotate_ready_queue(prio); + ps2sched::maybe_yield(); + setReturnS32(ctx, KE_OK); - yieldGuestExecutionAfterWake(runtime); } void iRotateThreadReadyQueue(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) @@ -1116,16 +1041,12 @@ namespace ps2_syscalls } bool wasWaiting = false; - int waitType = 0; - int waitId = 0; { std::lock_guard lock(info->m); if (info->status == THS_WAIT || info->status == THS_WAITSUSPEND) { wasWaiting = true; - waitType = info->waitType; - waitId = info->waitId; info->forceRelease = true; info->waitType = TSW_NONE; info->waitId = 0; @@ -1146,10 +1067,11 @@ namespace ps2_syscalls return; } - info->cv.notify_all(); - notifyThreadWaitObject(waitType, waitId); + // Make the released thread ready and yield if it has higher priority. + ps2sched::make_ready(tid); + ps2sched::maybe_yield(); + setReturnS32(ctx, KE_OK); - yieldGuestExecutionAfterWake(runtime); } void iReleaseWaitThread(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) diff --git a/ps2xRuntime/src/lib/ps2_debug_panel.cpp b/ps2xRuntime/src/lib/ps2_debug_panel.cpp index 2876e1289..043ee75bf 100644 --- a/ps2xRuntime/src/lib/ps2_debug_panel.cpp +++ b/ps2xRuntime/src/lib/ps2_debug_panel.cpp @@ -686,7 +686,6 @@ namespace const uint32_t gp = runtime.m_debugGp.load(std::memory_order_relaxed); ImGui::Text("Runtime: %s", runtime.isStopRequested() ? "stop requested" : "running"); - ImGui::Text("Guest execution waiters: %u", runtime.guestExecutionWaiterCountForTesting()); ImGui::Separator(); textHex32("PC", pc); ImGui::SameLine(); diff --git a/ps2xRuntime/src/lib/ps2_fiber.cpp b/ps2xRuntime/src/lib/ps2_fiber.cpp new file mode 100644 index 000000000..03c111e70 --- /dev/null +++ b/ps2xRuntime/src/lib/ps2_fiber.cpp @@ -0,0 +1,675 @@ +// Backend dispatch: PLATFORM_VITA -> SceFiber; PS2X_FIBER_PTHREAD -> pthread semaphore; +// _WIN32 -> Win32 Fibers; else -> POSIX ucontext +#include "ps2_fiber.h" +#include +#include +#include +#include +#include + +// Abort if a fiber context switch is attempted off the guest executor thread. +// Used by the backends (ucontext, SceFiber, Win32 Fibers) where exactly one +// thread ever switches between fiber contexts; doing it from another thread +// is undefined behaviour (ucontext_t) or a documented single-thread-use +// requirement (SceFiber, Win32 Fibers). The pthread backend runs each fiber +// on its own OS thread instead of a shared executor thread, so it does not +// use this guard. +static inline void ps2fiber_require_executor_thread(const char* who) +{ + if (!ps2fiber_on_executor_thread()) + { + std::fprintf(stderr, "FATAL: %s called off the guest executor thread\n", who); + std::abort(); + } +} + +#if !defined(PLATFORM_VITA) && !defined(PS2X_FIBER_PTHREAD) && !defined(_WIN32) +// ============================================================================ +// POSIX path — ucontext_t +// ============================================================================ +#include +#include +#include // sysconf(_SC_PAGESIZE) +#include +#include + +static_assert(sizeof(void*) == 8, "ucontext pointer split requires exactly 64-bit pointers (LP64)"); +static_assert(sizeof(unsigned int) == 4, "makecontext int args must be 32 bits"); + +// Per-guest-executor-thread state. These are thread_local but in practice +// only ever touched by the single g_guest_thread. +static thread_local ucontext_t tls_guest_main_ctx; +static thread_local PS2Fiber* tls_current_fiber_ptr = nullptr; + +struct PS2Fiber +{ + ucontext_t ctx; + uint8_t* map_base = nullptr; // mmap base (guard page first) + size_t map_size = 0; // total mapping incl. guard page + uint8_t* stack = nullptr; // usable stack region (after guard page) + size_t stackSize = 0; + void (*fn)(void*) = nullptr; + void* arg = nullptr; +}; + +// makecontext can only pass int-sized arguments portably. Split the 64-bit +// PS2Fiber* into two 32-bit halves and reassemble on entry. fn/arg are stored +// in the struct, so we only need to recover the PS2Fiber* here. +static void ps2fiber_trampoline(unsigned int self_hi, unsigned int self_lo) +{ + uintptr_t raw = (static_cast(self_hi) << 32) | static_cast(self_lo); + PS2Fiber* self = reinterpret_cast(raw); + self->fn(self->arg); + // fn returned. Switch back to the guest executor main context. The executor + // observes fiber state == Finished and frees it. We must NOT return from + // this function (there is no uc_link); swap out unconditionally. + swapcontext(&self->ctx, &tls_guest_main_ctx); + std::abort(); +} + +PS2Fiber* ps2fiber_alloc(void (*fn)(void*), void* arg, size_t stack_bytes) +{ + if (stack_bytes == 0) { std::fprintf(stderr, "ps2fiber_alloc: zero stack size\n"); return nullptr; } + long pageQuery = sysconf(_SC_PAGESIZE); + const size_t page = (pageQuery > 0) ? static_cast(pageQuery) : 4096u; + size_t usable = (stack_bytes + page - 1) & ~(page - 1); + size_t total = usable + page; // 1 guard page at low address + + void* base = mmap(nullptr, total, PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + if (base == MAP_FAILED) + { + std::fprintf(stderr, "[ps2fiber] mmap failed\n"); + return nullptr; + } + if (mprotect(base, page, PROT_NONE) != 0) + { + munmap(base, total); + std::fprintf(stderr, "[ps2fiber] mprotect guard failed\n"); + return nullptr; + } + + PS2Fiber* f = new (std::nothrow) PS2Fiber(); + if (!f) + { + munmap(base, total); + return nullptr; + } + + f->map_base = static_cast(base); + f->map_size = total; + f->stack = static_cast(base) + page; + f->stackSize = usable; + f->fn = fn; + f->arg = arg; + + if (getcontext(&f->ctx) != 0) + { + std::fprintf(stderr, "[ps2fiber] getcontext failed: %s\n", std::strerror(errno)); + munmap(base, total); + delete f; + return nullptr; + } + f->ctx.uc_stack.ss_sp = f->stack; + f->ctx.uc_stack.ss_size = f->stackSize; + f->ctx.uc_link = nullptr; // trampoline never returns via link + + uintptr_t raw = reinterpret_cast(f); + makecontext(&f->ctx, reinterpret_cast(ps2fiber_trampoline), 2, + static_cast(raw >> 32), + static_cast(raw & 0xFFFFFFFFu)); + return f; +} + +void ps2fiber_free(PS2Fiber* f) +{ + if (!f) return; + if (f->map_base) munmap(f->map_base, f->map_size); + delete f; +} + +void ps2fiber_resume(PS2Fiber* f) +{ + ps2fiber_require_executor_thread("ps2fiber_resume"); + PS2Fiber* prev = tls_current_fiber_ptr; + tls_current_fiber_ptr = f; + if (prev == nullptr) + { + // Resuming from the guest executor main context. + swapcontext(&tls_guest_main_ctx, &f->ctx); + } + else + { + // Unreachable in the N=1 design: the executor only ever resumes a fiber + // from the main context (prev == nullptr). A non-null prev would mean a + // fiber resumed another fiber directly, which the scheduler never does. + std::fprintf(stderr, "FATAL: nested ps2fiber_resume (prev != nullptr)\n"); + std::abort(); + } + tls_current_fiber_ptr = prev; +} + +void ps2fiber_yield() +{ + ps2fiber_require_executor_thread("ps2fiber_yield"); + PS2Fiber* self = tls_current_fiber_ptr; + if (!self) + { + std::fprintf(stderr, "FATAL: ps2fiber_yield with no current fiber\n"); + std::abort(); + } + tls_current_fiber_ptr = nullptr; + swapcontext(&self->ctx, &tls_guest_main_ctx); + tls_current_fiber_ptr = self; +} + +PS2Fiber* ps2fiber_current() +{ + return tls_current_fiber_ptr; +} + +bool ps2fiber_finished(PS2Fiber* /*f*/) +{ + // ucontext backend: no separate OS thread to outlive the PS2Fiber. The + // executor relies on FiberContext::state == Finished instead. + return false; +} + +#elif defined(PLATFORM_VITA) && !defined(PS2X_FIBER_PTHREAD) +// ============================================================================ +// SceFiber backend — Vita (PLATFORM_VITA and not PS2X_FIBER_PTHREAD) +// ============================================================================ +#include +#include // sceKernelAllocMemBlock/GetMemBlockBase/FreeMemBlock +#include "ps2_scheduler.h" // extern thread_local int g_currentThreadId + +static thread_local PS2Fiber* tls_current_fiber_ptr = nullptr; +// Set by ps2fiber_resume immediately before sceFiberRun so the fiber entry +// can recover its PS2Fiber* without relying on the 32-bit argOnRunTo alone. +static thread_local PS2Fiber* tls_pending_self = nullptr; + +struct PS2Fiber +{ + SceFiber fiber; // SDK control block (value, not pointer) + SceUID memblock = -1; // sceKernelAllocMemBlock uid backing ctx (< 0 == none) + void* ctx = nullptr; // usable context buffer = memblock base + size_t ctxSize = 0; + void (*fn)(void*) = nullptr; + void* arg = nullptr; + int tid = -1; +}; + +static void ps2fiber_entry(SceUInt32 /*argOnInitialize*/, SceUInt32 /*argOnRunTo*/) +{ + PS2Fiber* self = tls_pending_self; + tls_current_fiber_ptr = self; + g_currentThreadId = self->tid; + self->fn(self->arg); + // fn returned — hand control back to the thread. The entry must not return. + sceFiberReturnToThread(0, nullptr); + std::abort(); // unreachable +} + +PS2Fiber* ps2fiber_alloc(void (*fn)(void*), void* arg, size_t stack_bytes) +{ + if (stack_bytes == 0) { std::fprintf(stderr, "ps2fiber_alloc: zero stack size\n"); return nullptr; } + // sceKernelAllocMemBlock requires the size to be a multiple of 4 KiB for + // SCE_KERNEL_MEMBLOCK_TYPE_USER_RW (documented VitaSDK behaviour; there is + // no vitasdk-exposed named constant for the alignment itself). Round up. + const size_t page = 4096u; + size_t usable = (stack_bytes + page - 1) & ~(page - 1); + + // NOTE: unlike mmap+mprotect on POSIX, there is no VitaSDK equivalent of a + // guard page for a plain USER_RW memblock (no per-page protection API for + // a sub-range of a block), so a Vita fiber stack has NO guard page. A + // stack overflow here will silently corrupt adjacent memory rather than + // fault — this is an accepted platform limitation, not an oversight. + SceUID memblock = sceKernelAllocMemBlock("ps2fiber_stack", SCE_KERNEL_MEMBLOCK_TYPE_USER_RW, + static_cast(usable), nullptr); + if (memblock < 0) + { + std::fprintf(stderr, "[ps2fiber] sceKernelAllocMemBlock failed: 0x%08x\n", memblock); + return nullptr; + } + void* base = nullptr; + int rcBase = sceKernelGetMemBlockBase(memblock, &base); + if (rcBase != SCE_OK || !base) + { + sceKernelFreeMemBlock(memblock); + std::fprintf(stderr, "[ps2fiber] sceKernelGetMemBlockBase failed: 0x%08x\n", rcBase); + return nullptr; + } + + PS2Fiber* f = new (std::nothrow) PS2Fiber(); + if (!f) { sceKernelFreeMemBlock(memblock); return nullptr; } + + f->memblock = memblock; + f->ctx = base; + f->ctxSize = usable; + f->fn = fn; + f->arg = arg; + + // vitasdk only declares _sceFiberInitializeImpl (confirmed against + // vita-headers db/360/SceFiber.yml — there is no sceFiberInitialize NID), + // so call the Impl entry point directly. It takes a non-const char* name. + int rc = _sceFiberInitializeImpl(&f->fiber, const_cast("ps2fiber"), ps2fiber_entry, 0, + f->ctx, static_cast(f->ctxSize), nullptr); + if (rc != 0) + { + sceKernelFreeMemBlock(memblock); + delete f; + std::fprintf(stderr, "[ps2fiber] _sceFiberInitializeImpl failed: 0x%08x\n", rc); + return nullptr; + } + return f; +} + +void ps2fiber_set_tid(PS2Fiber* f, int tid) +{ + // No-op stub for SceFiber: guest_executor_main already publishes + // g_currentThreadId on the executor thread (N=1 — no cross-thread TLS hazard). + // The tid is stored so ps2fiber_entry can publish it when the fiber first runs. + if (f) f->tid = tid; +} + +void ps2fiber_resume(PS2Fiber* f) +{ + ps2fiber_require_executor_thread("ps2fiber_resume"); + tls_pending_self = f; + PS2Fiber* prev = tls_current_fiber_ptr; + tls_current_fiber_ptr = f; + SceInt32 rc = sceFiberRun(&f->fiber, 0, nullptr); + if (rc != SCE_OK) + { + std::fprintf(stderr, "[ps2fiber] sceFiberRun failed: 0x%08x\n", rc); + std::abort(); + } + tls_current_fiber_ptr = prev; +} + +void ps2fiber_yield() +{ + ps2fiber_require_executor_thread("ps2fiber_yield"); + PS2Fiber* self = tls_current_fiber_ptr; + if (!self) { std::fprintf(stderr, "FATAL: ps2fiber_yield with no current fiber\n"); std::abort(); } + tls_current_fiber_ptr = nullptr; + SceInt32 rcRet = sceFiberReturnToThread(0, nullptr); + if (rcRet != SCE_OK) + { + std::fprintf(stderr, "[ps2fiber] sceFiberReturnToThread failed: 0x%08x\n", rcRet); + std::abort(); + } + tls_current_fiber_ptr = self; +} + +PS2Fiber* ps2fiber_current() +{ + return tls_current_fiber_ptr; +} + +bool ps2fiber_finished(PS2Fiber* /*f*/) +{ + // SceFiber backend: matches ucontext semantics — the scheduler relies on + // FiberContext::state == Finished rather than polling this function. + return false; +} + +void ps2fiber_free(PS2Fiber* f) +{ + if (!f) return; + // sceFiberInitialize always succeeds before ps2fiber_alloc returns, so + // every allocated fiber must be finalized — even one that was never started. + SceInt32 rcFin = sceFiberFinalize(&f->fiber); + if (rcFin != SCE_OK) + { + std::fprintf(stderr, "[ps2fiber] sceFiberFinalize failed: 0x%08x\n", rcFin); + std::abort(); + } + if (f->memblock >= 0) sceKernelFreeMemBlock(f->memblock); + delete f; +} + +#elif defined(PS2X_FIBER_PTHREAD) +// ============================================================================ +// pthread backend — Linux TSan testing (PS2X_FIBER_PTHREAD) +// ============================================================================ +#if defined(PLATFORM_VITA) +#error "Both PLATFORM_VITA and PS2X_FIBER_PTHREAD defined — use Vita toolchain without PS2X_FIBER_PTHREAD" +#endif +#include +#include +#include +#include +#include "ps2_scheduler.h" // extern thread_local int g_currentThreadId + +static thread_local PS2Fiber* tls_current_fiber_ptr = nullptr; + +struct PS2Fiber +{ + pthread_t thread; + sem_t resume_sem; // posted by ps2fiber_resume; waited by fiber thread + sem_t yield_sem; // posted by ps2fiber_yield/finish; waited by resume + void (*fn)(void*) = nullptr; + void* arg = nullptr; + bool started = false; // pthread created + std::atomic finished{false}; // fn returned; written on the fiber thread and read on the executor, so must be atomic + std::atomic ever_resumed{false}; // set by ps2fiber_resume; distinguishes "never resumed" (free is legal) from "parked mid-run" (free is a caller bug) + std::atomic abandoned{false}; // set by ps2fiber_free when freeing a never-resumed fiber; tells the worker to exit without running fn + uint8_t* map_base = nullptr; // mmap base (guard page first); freed after pthread_join + size_t map_size = 0; // total mapping incl. guard page + // Guest thread id for this fiber. Set by the scheduler via + // ps2fiber_set_tid() after alloc; published onto the fiber's own pthread in + // ps2fiber_thread_main so blocking syscalls see the right g_currentThreadId. + // Defaults to -1 until the scheduler assigns it. + int tid = -1; +}; + +static void* ps2fiber_thread_main(void* raw) +{ + PS2Fiber* self = reinterpret_cast(raw); + sem_wait(&self->resume_sem); // wait for first resume + if (self->abandoned.load(std::memory_order_acquire)) + { + // ps2fiber_free abandoned this fiber before the scheduler ever resumed + // it (legal per ps2_fiber.h's contract: free is allowed "before it was + // ever resumed"). Exit immediately without running fn or touching + // guest TLS; ps2fiber_free is parked in pthread_join waiting for us. + self->finished.store(true, std::memory_order_release); + return nullptr; + } + // Publish guest identity ON THE FIBER'S OWN THREAD. On Vita guest code + // runs here, not on the executor thread, so blocking syscalls must read the + // fiber's tid / current-fiber pointer from this thread's TLS. + tls_current_fiber_ptr = self; + g_currentThreadId = self->tid; + self->fn(self->arg); + self->finished.store(true, std::memory_order_release); + g_currentThreadId = -1; + tls_current_fiber_ptr = nullptr; + sem_post(&self->yield_sem); // hand control back to executor; thread ends + return nullptr; +} + +PS2Fiber* ps2fiber_alloc(void (*fn)(void*), void* arg, size_t stack_bytes) +{ + PS2Fiber* f = new (std::nothrow) PS2Fiber(); + if (!f) return nullptr; + f->fn = fn; + f->arg = arg; + + // Allocate our own stack with an explicit guard page at the low address so + // a stack overflow faults (SIGSEGV) rather than silently corrupting memory. + long pg_long = sysconf(_SC_PAGESIZE); + size_t page = (pg_long > 0) ? static_cast(pg_long) : 4096u; + size_t usable = stack_bytes ? stack_bytes : (512 * 1024); // smaller on Vita + size_t total = usable + page; + void* base = mmap(nullptr, total, PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + if (base == MAP_FAILED) + { + delete f; + return nullptr; + } + if (mprotect(base, page, PROT_NONE) != 0) + { + munmap(base, total); + delete f; + return nullptr; + } + f->map_base = static_cast(base); + f->map_size = total; + + sem_init(&f->resume_sem, 0, 0); + sem_init(&f->yield_sem, 0, 0); + + pthread_attr_t attr; + pthread_attr_init(&attr); + // Belt-and-suspenders: ask pthread to set its own guard too (may be a no-op + // on some platforms, but the mmap guard above is the authoritative one). + pthread_attr_setguardsize(&attr, page); + // Hand our pre-allocated, guard-paged stack to pthread. + pthread_attr_setstack(&attr, static_cast(base) + page, usable); + int rc = pthread_create(&f->thread, &attr, ps2fiber_thread_main, f); + pthread_attr_destroy(&attr); + if (rc != 0) + { + sem_destroy(&f->resume_sem); + sem_destroy(&f->yield_sem); + munmap(base, total); + delete f; + return nullptr; + } + f->started = true; + return f; +} + +void ps2fiber_set_tid(PS2Fiber* f, int tid) +{ + if (f) f->tid = tid; +} + +void ps2fiber_resume(PS2Fiber* f) +{ + f->ever_resumed.store(true, std::memory_order_release); + sem_post(&f->resume_sem); // wake fiber thread + sem_wait(&f->yield_sem); // block until it yields or finishes +} + +void ps2fiber_yield() +{ + PS2Fiber* self = tls_current_fiber_ptr; + if (!self) { std::abort(); } + sem_post(&self->yield_sem); // hand back to executor + sem_wait(&self->resume_sem); // wait for next resume +} + +PS2Fiber* ps2fiber_current() +{ + return tls_current_fiber_ptr; +} + +bool ps2fiber_finished(PS2Fiber* f) +{ + // ps2fiber_thread_main sets finished=true just before the pthread exits. + // The executor uses this to avoid resuming a dead fiber thread. + return f && f->finished.load(std::memory_order_acquire); +} + +void ps2fiber_free(PS2Fiber* f) +{ + if (!f) return; + if (f->started && !f->finished.load(std::memory_order_acquire)) + { + if (!f->ever_resumed.load(std::memory_order_acquire)) + { + // Never resumed: legal per ps2_fiber.h's contract ("before it was + // ever resumed"). The worker thread is parked in its first + // sem_wait(resume_sem); wake it with 'abandoned' set so it exits + // immediately instead of running fn / publishing guest TLS. + f->abandoned.store(true, std::memory_order_release); + sem_post(&f->resume_sem); + } + else + { + // Resumed at least once and currently parked mid-run (yielded, not + // finished): a genuine caller bug — the scheduler must only free + // fibers that are Finished (or were never started). + std::fprintf(stderr, "FATAL: ps2fiber_free on unfinished fiber\n"); + std::abort(); + } + } + if (f->started) pthread_join(f->thread, nullptr); // joinable: no leak + // Unmap our guard-paged stack only after the thread is fully joined (exited). + if (f->map_base) munmap(f->map_base, f->map_size); + sem_destroy(&f->resume_sem); + sem_destroy(&f->yield_sem); + delete f; +} + +#elif defined(_WIN32) +// ============================================================================ +// Win32 Fibers backend — Windows (_WIN32, not PLATFORM_VITA/PS2X_FIBER_PTHREAD) +// ============================================================================ +#ifndef WIN32_LEAN_AND_MEAN +#define WIN32_LEAN_AND_MEAN +#endif +#ifndef NOMINMAX +#define NOMINMAX +#endif +#include + +// Per-guest-executor-thread state. These are thread_local but in practice +// only ever touched by the single g_guest_thread. +static thread_local LPVOID tls_guest_main_fiber = nullptr; // executor's own fiber (the "main context") +static thread_local PS2Fiber* tls_current_fiber_ptr = nullptr; + +struct PS2Fiber +{ + LPVOID fiber = nullptr; // CreateFiberEx handle + void (*fn)(void*) = nullptr; + void* arg = nullptr; +}; + +// Lazily turn the executor thread itself into a fiber so there is a "main" +// fiber to SwitchToFiber back to. ConvertThreadToFiberEx fails with +// ERROR_ALREADY_FIBER when the thread was already converted (e.g. by host +// code), so detect that case via IsThreadAFiber/GetCurrentFiber instead of +// calling it twice. +// +// FIBER_FLAG_FLOAT_SWITCH matters here and in CreateFiberEx: without it, +// x87/SSE floating-point state is NOT saved across fiber switches, and guest +// code relies heavily on FPU/SSE state (COP1, VU macro mode). +static LPVOID ps2fiber_main_fiber() +{ + if (tls_guest_main_fiber) return tls_guest_main_fiber; + if (IsThreadAFiber()) + { + tls_guest_main_fiber = GetCurrentFiber(); + return tls_guest_main_fiber; + } + LPVOID mainFiber = ConvertThreadToFiberEx(nullptr, FIBER_FLAG_FLOAT_SWITCH); + if (!mainFiber) + { + std::fprintf(stderr, "FATAL: ConvertThreadToFiberEx failed: %lu\n", + static_cast(GetLastError())); + std::abort(); + } + tls_guest_main_fiber = mainFiber; + return mainFiber; +} + +static void CALLBACK ps2fiber_trampoline(void* raw) +{ + PS2Fiber* self = static_cast(raw); + self->fn(self->arg); + // fn returned. Switch back to the guest executor main fiber. The executor + // observes fiber state == Finished and frees it. We must NOT return from + // this start routine: a Win32 fiber start routine that returns calls + // ExitThread, silently killing the entire executor thread. + SwitchToFiber(tls_guest_main_fiber); + std::fprintf(stderr, "FATAL: Win32 fiber resumed after its fn returned\n"); + std::abort(); +} + +PS2Fiber* ps2fiber_alloc(void (*fn)(void*), void* arg, size_t stack_bytes) +{ + if (stack_bytes == 0) { std::fprintf(stderr, "ps2fiber_alloc: zero stack size\n"); return nullptr; } + PS2Fiber* f = new (std::nothrow) PS2Fiber(); + if (!f) return nullptr; + f->fn = fn; + f->arg = arg; + + // Reserve the caller's full stack_bytes but commit only 64 KiB up front: + // Windows grows the committed region on demand through its guard-page + // mechanism, so a large reserve costs address space, not RAM, and 64 KiB + // comfortably covers the trampoline + scheduler frames before guest code + // pushes deeper. (If stack_bytes is smaller than the commit, the system + // rounds the reserve up to cover it.) Unlike the ucontext backend's + // explicit mprotect(PROT_NONE) page, Windows fiber stacks get an + // OS-managed guard page natively, so overflow raises + // STATUS_STACK_OVERFLOW instead of silently corrupting adjacent memory. + const SIZE_T commit = 64 * 1024; + const SIZE_T reserve = stack_bytes; + f->fiber = CreateFiberEx(commit, reserve, FIBER_FLAG_FLOAT_SWITCH, ps2fiber_trampoline, f); + if (!f->fiber) + { + std::fprintf(stderr, "[ps2fiber] CreateFiberEx failed: %lu\n", + static_cast(GetLastError())); + delete f; + return nullptr; + } + return f; +} + +void ps2fiber_free(PS2Fiber* f) +{ + if (!f) return; + if (f->fiber) + { + // DeleteFiber is safe both for a Finished fiber (parked on the main + // fiber after its fn returned) and for one that was never resumed. + // It must NOT run for the currently executing fiber — DeleteFiber on + // the running fiber calls ExitThread. The ps2_fiber.h contract already + // forbids freeing a running fiber; enforce it anyway. + if (IsThreadAFiber() && GetCurrentFiber() == f->fiber) + { + std::fprintf(stderr, "FATAL: ps2fiber_free called on the running fiber\n"); + std::abort(); + } + DeleteFiber(f->fiber); + } + delete f; +} + +void ps2fiber_resume(PS2Fiber* f) +{ + ps2fiber_require_executor_thread("ps2fiber_resume"); + ps2fiber_main_fiber(); // lazily convert the executor thread on first use + PS2Fiber* prev = tls_current_fiber_ptr; + tls_current_fiber_ptr = f; + if (prev == nullptr) + { + // Resuming from the guest executor main fiber. + SwitchToFiber(f->fiber); + } + else + { + // Unreachable in the N=1 design: the executor only ever resumes a fiber + // from the main context (prev == nullptr). A non-null prev would mean a + // fiber resumed another fiber directly, which the scheduler never does. + std::fprintf(stderr, "FATAL: nested ps2fiber_resume (prev != nullptr)\n"); + std::abort(); + } + tls_current_fiber_ptr = prev; +} + +void ps2fiber_yield() +{ + ps2fiber_require_executor_thread("ps2fiber_yield"); + PS2Fiber* self = tls_current_fiber_ptr; + if (!self) + { + std::fprintf(stderr, "FATAL: ps2fiber_yield with no current fiber\n"); + std::abort(); + } + tls_current_fiber_ptr = nullptr; + SwitchToFiber(tls_guest_main_fiber); + tls_current_fiber_ptr = self; +} + +PS2Fiber* ps2fiber_current() +{ + return tls_current_fiber_ptr; +} + +bool ps2fiber_finished(PS2Fiber* /*f*/) +{ + // Win32 Fibers backend: matches ucontext semantics — no separate OS thread + // to outlive the PS2Fiber, so the executor relies on + // FiberContext::state == Finished instead of polling this function. + return false; +} + +#else +#error "Unknown fiber backend: define PLATFORM_VITA (SceFiber), PS2X_FIBER_PTHREAD (pthread), build for _WIN32 (Win32 Fibers), or use the default POSIX ucontext backend" +#endif // fiber backend dispatch diff --git a/ps2xRuntime/src/lib/ps2_fiber.h b/ps2xRuntime/src/lib/ps2_fiber.h new file mode 100644 index 000000000..f68ed202e --- /dev/null +++ b/ps2xRuntime/src/lib/ps2_fiber.h @@ -0,0 +1,49 @@ +#pragma once +#include + +// Opaque fiber handle. +struct PS2Fiber; + +// Allocate a fiber that will run fn(arg) on its own stack of stack_bytes. +// Returns nullptr on allocation failure (caller must handle — see create_fiber). +PS2Fiber* ps2fiber_alloc(void (*fn)(void*), void* arg, size_t stack_bytes); + +#if defined(PLATFORM_VITA) || defined(PS2X_FIBER_PTHREAD) +// Vita and pthread backends: associate a guest thread id with a fiber so +// blocking syscalls see the correct g_currentThreadId on the fiber's own OS +// thread. Call after ps2fiber_alloc, before first resume. +// The POSIX ucontext and Win32 Fibers backends have no per-fiber OS thread and +// set g_currentThreadId in guest_executor_main, so this does not exist there. +void ps2fiber_set_tid(PS2Fiber* f, int tid); +#endif + +// Free the fiber's stack and struct. Must NOT be called while the fiber is +// running or is mid-yield (i.e. only after fn returned / it is Finished, or +// before it was ever resumed). +void ps2fiber_free(PS2Fiber* f); + +// Resume (switch TO) fiber f. MUST be called only from the guest executor +// thread (the thread that owns the "main" context). Returns when f yields or +// when f's fn returns. +void ps2fiber_resume(PS2Fiber* f); + +// Yield (switch FROM the current fiber back to the guest executor main context). +// MUST be called only from within a fiber running on the guest executor thread. +void ps2fiber_yield(); + +// The fiber currently running on this thread, or nullptr if not on a fiber. +PS2Fiber* ps2fiber_current(); + +// True if the fiber's function has already returned. On the POSIX ucontext +// and Win32 Fibers backends this is always false: those fibers have no +// separate OS thread, so they cannot be "dead" while their PS2Fiber still +// exists. Their executors rely on FiberContext::state == Finished (set in the +// fiber trampoline) to detect completion rather than polling this function. +bool ps2fiber_finished(PS2Fiber* f); + +// True when called on the registered guest executor thread. Defined in +// ps2_scheduler.cpp. Used by the ucontext, SceFiber, and Win32 Fibers +// backends to assert that fiber context switches only happen on the one +// executor thread (the N=1 invariant), which is what makes saving/restoring +// a fiber context safe. +bool ps2fiber_on_executor_thread(); diff --git a/ps2xRuntime/src/lib/ps2_runtime.cpp b/ps2xRuntime/src/lib/ps2_runtime.cpp index 667a6d38a..5e139a31b 100644 --- a/ps2xRuntime/src/lib/ps2_runtime.cpp +++ b/ps2xRuntime/src/lib/ps2_runtime.cpp @@ -1,4 +1,6 @@ #include "ps2_runtime.h" +#include "ps2_dispatch_history.h" +#include "ps2_scheduler.h" #include "ps2_log.h" #include "ps2_stubs.h" #include "ps2_syscalls.h" @@ -21,7 +23,6 @@ #include #include #include -#include #include namespace ps2_stubs @@ -89,6 +90,31 @@ namespace constexpr uint32_t kGuestHeapSafetyPad = 0x1000u; constexpr uint32_t kGuestHeapHardLimit = 0x01F00000u; + // ----------------------------------------------------------------------- + // Async callback stack pool: [kAsyncCallbackStackFloor, kAsyncCallbackStackTop) + // — KERNEL-RESERVED memory, carved downward from the top. + // + // Placed below the ELF load base so it is disjoint from the game-chosen + // main stack at top-of-RAM — SDK crt0 calls SetupThread to place that + // stack at the top of user RAM — so + // host-dispatched guest callbacks (the sceGsSyncVCallback chain, INTC + // handlers, alarms) never interleave, via the N=1 scheduler's token + // handoff, with the game's own live stack frames. + // + // On real hardware these contexts run on KERNEL stacks in kernel-reserved + // memory — never on the interrupted user thread's stack. The EE kernel + // owns phys [0, 0x00100000): this runtime's kernel-mirror state sits + // below 0x00012000, the ELF loads at 0x00100000+, the guest heap starts + // at the ELF's bss end, and guest thread stacks are game-chosen addresses + // >= 0x00100000. [kAsyncCallbackStackFloor, kAsyncCallbackStackTop) is untouched by both sides, so + // callback stacks there are disjoint from ALL guest memory by + // construction. + // + // kAsyncCallbackStackFloor/Top themselves live in ps2_runtime.h (next to + // kAsyncCallbackFallbackSp) so the member default initializers below and + // this file's re-arm site share one definition instead of three. + // ----------------------------------------------------------------------- + constexpr uint32_t COP0_CAUSE_EXCCODE_MASK = 0x0000007Cu; constexpr uint32_t COP0_CAUSE_BD = 0x80000000u; constexpr uint32_t COP0_STATUS_EXL = 0x00000002u; @@ -97,19 +123,16 @@ namespace constexpr uint32_t EXCEPTION_VECTOR_TLB_REFILL = 0x80000000u; constexpr uint32_t EXCEPTION_VECTOR_BOOT = 0xBFC00200u; - struct DispatchHistory + // Fiber-owned when running inside a fiber; per-OS-thread fallback otherwise + // (borrowed host workers, executor between fibers, direct non-fiber callers). + DispatchHistory ¤tDispatchHistory() { - std::array pcs{}; - uint32_t next = 0u; - bool wrapped = false; - }; - - thread_local DispatchHistory g_dispatchHistory; - thread_local std::unordered_map g_guestExecutionDepths; + return ps2sched::current_dispatch_history(); + } void pushDispatchPc(uint32_t pc) { - DispatchHistory &h = g_dispatchHistory; + DispatchHistory &h = currentDispatchHistory(); h.pcs[h.next] = pc; h.next = (h.next + 1u) % static_cast(h.pcs.size()); if (h.next == 0u) @@ -120,7 +143,7 @@ namespace std::string formatDispatchHistory() { - const DispatchHistory &h = g_dispatchHistory; + const DispatchHistory &h = currentDispatchHistory(); const uint32_t count = h.wrapped ? static_cast(h.pcs.size()) : h.next; if (count == 0u) { @@ -323,40 +346,6 @@ namespace } } -PS2Runtime::GuestExecutionScope::GuestExecutionScope(PS2Runtime *runtime) noexcept - : m_runtime(runtime) -{ - if (m_runtime) - { - m_runtime->enterGuestExecution(); - } -} - -PS2Runtime::GuestExecutionScope::~GuestExecutionScope() -{ - if (m_runtime) - { - m_runtime->leaveGuestExecution(); - } -} - -PS2Runtime::GuestExecutionReleaseScope::GuestExecutionReleaseScope(PS2Runtime *runtime) noexcept - : m_runtime(runtime) -{ - if (m_runtime) - { - m_depth = m_runtime->releaseGuestExecution(); - } -} - -PS2Runtime::GuestExecutionReleaseScope::~GuestExecutionReleaseScope() -{ - if (m_runtime && m_depth != 0u) - { - m_runtime->reacquireGuestExecution(m_depth); - } -} - static void UploadFrame(Texture2D &tex, PS2Runtime *rt, uint32_t &outWidth, uint32_t &outHeight) { static uint64_t s_lastPresentationTick = std::numeric_limits::max(); @@ -484,8 +473,27 @@ PS2Runtime::PS2Runtime() m_guestHeapLimit = std::min(kGuestHeapHardLimit, PS2_RAM_SIZE); m_guestHeapSuggestedBase = kGuestHeapDefaultBase; m_guestHeapConfigured = false; - m_asyncCallbackStackFloor = std::min(kGuestHeapHardLimit, PS2_RAM_SIZE); - m_asyncCallbackStackTop = PS2_RAM_SIZE; + // m_asyncCallbackStackFloor/Top keep their kernel-pool default member + // initializers here; loadELF() re-arms them for the pool's next load + // (see the layout comment at kAsyncCallbackStackFloor). + + // Claim the reserved main-thread identity (tid 1 — see State.h's + // g_nextThreadId starting at 2, and run()'s create_fiber(1, 1, ...) for the + // guest boot fiber, both of which reserve this same id) for the host thread + // that constructs this runtime. g_currentThreadId == -1 here means this + // host thread has never been assigned a guest identity: it is neither a + // running guest fiber (which carries its own tid, set by the scheduler on + // its own dedicated executor thread — a different OS thread from this one) + // nor a borrowed IRQ/alarm/RPC worker (those self-assign -1 as the first + // statement of their thread function, before any PS2Runtime is reachable). + // ensureCurrentThreadInfo() lazily creates tid 1's ThreadInfo (THS_RUN, + // wakeupCount 0) the first time a syscall needs it, and the guest boot + // fiber (also tid 1, but on the separate executor thread) later finds and + // reuses that same g_threads entry — so tid 1 never has two ThreadInfos. + if (g_currentThreadId == -1) + { + g_currentThreadId = 1; + } } void PS2Runtime::setDebugUiCallbacks(DebugUiCallback initCallback, @@ -510,7 +518,7 @@ PS2Runtime::~PS2Runtime() try { requestStop(); - ps2_syscalls::detachAllGuestHostThreads(); + // Fiber pool is cleaned up by scheduler_shutdown() in run(). #if defined(PLATFORM_VITA) m_audioBackend.stopAll(); m_audioBackend.setAudioReady(false); @@ -687,7 +695,6 @@ bool PS2Runtime::loadELF(const std::string &elfPath) } m_cpuContext.pc = header.entry; - m_debugPc.store(m_cpuContext.pc, std::memory_order_relaxed); uint32_t maxLoadedRdramEnd = kGuestHeapDefaultBase; uint32_t moduleBase = std::numeric_limits::max(); @@ -840,9 +847,10 @@ bool PS2Runtime::loadELF(const std::string &elfPath) } { std::lock_guard lock(m_asyncCallbackStackMutex); - const uint32_t hardLimit = std::min(kGuestHeapHardLimit, PS2_RAM_SIZE); - m_asyncCallbackStackFloor = std::min(std::max(hardLimit, suggestedHeapBase), PS2_RAM_SIZE); - m_asyncCallbackStackTop = PS2_RAM_SIZE; + // Re-arm the kernel pool for this load (see the layout comment at + // kAsyncCallbackStackFloor); a prior run may have carved it down. + m_asyncCallbackStackFloor = kAsyncCallbackStackFloor; + m_asyncCallbackStackTop = kAsyncCallbackStackTop; } LoadedModule module; @@ -1030,6 +1038,11 @@ void PS2Runtime::resetMissingFunctionReportOnce() m_missingFunctionReported.store(false, std::memory_order_release); } +std::string PS2Runtime::debugCurrentDispatchTrace() const +{ + return formatDispatchHistory(); +} + void PS2Runtime::reportMissingFunction(uint8_t *rdram, R5900Context *ctx, uint32_t targetPc, @@ -1814,6 +1827,12 @@ uint32_t PS2Runtime::reserveAsyncCallbackStack(uint32_t size, uint32_t alignment } m_asyncCallbackStackTop = base; + // One line per reservation (a handful per boot): permanent evidence of + // where host-dispatched callback stacks live, so any future overlap with + // guest memory is visible in the boot log. + std::cerr << "[async-stack] reserved [0x" << std::hex << base + << ", 0x" << top << ") stackTop=0x" << (top - 0x10u) + << std::dec << '\n'; return top - 0x10u; } @@ -1825,6 +1844,19 @@ void PS2Runtime::dispatchLoop(uint8_t *rdram, R5900Context *ctx) while (!isStopRequested()) { + // Cooperative scheduling point. The recompiler emits the + // shouldPreemptGuestExecution() hook only at INTRA-function back-edges; + // a guest loop that spins ACROSS function dispatches (call/return + // chains, recover-pc storms) has its back-edge HERE, not inside any + // recompiled function, so without this call such a loop never reaches + // yield_point() and holds the guest token forever, starving host + // workers (interrupt worker VBlank/INTC delivery) parked in + // async_guest_begin(). The fast path is a counter test, so this is as + // cheap as the emitted per-back-edge checks. The return value is + // irrelevant: whether or not we yielded, ctx->pc is a clean + // function-boundary resume point. + (void)shouldPreemptGuestExecution(); + const uint32_t pc = ctx->pc; if (pc == lastPc) @@ -1852,11 +1884,7 @@ void PS2Runtime::dispatchLoop(uint8_t *rdram, R5900Context *ctx) RecompiledFunction fn = lookupFunction(pc); const uint32_t dispatchedPc = pc; const uint32_t dispatchedRa = static_cast(_mm_extract_epi32(ctx->r[31], 0)); - - { - GuestExecutionScope guestExecution(this); - fn(rdram, ctx, this); - } + fn(rdram, ctx, this); if (ctx->pc == 0u) { @@ -1880,105 +1908,9 @@ void PS2Runtime::dispatchLoop(uint8_t *rdram, R5900Context *ctx) } } -void PS2Runtime::enterGuestExecution() -{ - m_guestExecutionWaiters.fetch_add(1u, std::memory_order_acq_rel); - m_guestExecutionMutex.lock(); - m_guestExecutionWaiters.fetch_sub(1u, std::memory_order_acq_rel); - ++g_guestExecutionDepths[this]; - markGuestExecutionAcquired(); -} - -void PS2Runtime::leaveGuestExecution() -{ - auto it = g_guestExecutionDepths.find(this); - if (it == g_guestExecutionDepths.end() || it->second == 0u) - { - return; - } - - --it->second; - m_guestExecutionMutex.unlock(); - if (it->second == 0u) - { - g_guestExecutionDepths.erase(it); - } -} - -uint32_t PS2Runtime::releaseGuestExecution() -{ - auto it = g_guestExecutionDepths.find(this); - if (it == g_guestExecutionDepths.end() || it->second == 0u) - { - return 0u; - } - - const uint32_t depth = it->second; - for (uint32_t i = 0; i < depth; ++i) - { - m_guestExecutionMutex.unlock(); - } - g_guestExecutionDepths.erase(it); - return depth; -} - -void PS2Runtime::reacquireGuestExecution(uint32_t depth) -{ - if (depth == 0u) - { - return; - } - - uint32_t &heldDepth = g_guestExecutionDepths[this]; - for (uint32_t i = 0; i < depth; ++i) - { - m_guestExecutionWaiters.fetch_add(1u, std::memory_order_acq_rel); - m_guestExecutionMutex.lock(); - m_guestExecutionWaiters.fetch_sub(1u, std::memory_order_acq_rel); - ++heldDepth; - markGuestExecutionAcquired(); - } -} - -void PS2Runtime::markGuestExecutionAcquired() -{ - { - std::lock_guard lock(m_guestExecutionHandoffMutex); - m_guestExecutionHandoffEpoch.fetch_add(1u, std::memory_order_acq_rel); - } - m_guestExecutionHandoffCv.notify_all(); -} - -void PS2Runtime::yieldGuestExecutionAfterWake() -{ - auto it = g_guestExecutionDepths.find(this); - if (it == g_guestExecutionDepths.end() || it->second == 0u) - { - std::this_thread::yield(); - return; - } - - const uint64_t handoffEpoch = m_guestExecutionHandoffEpoch.load(std::memory_order_acquire); - { - GuestExecutionReleaseScope releaseGuestExecution(this); - std::unique_lock lock(m_guestExecutionHandoffMutex); - m_guestExecutionHandoffCv.wait_for(lock, std::chrono::milliseconds(2), [&]() - { return m_guestExecutionHandoffEpoch.load(std::memory_order_acquire) != handoffEpoch; }); - } -} - bool PS2Runtime::shouldPreemptGuestExecution() { - thread_local uint32_t s_backEdgeYieldCounter = 0u; - const uint32_t waiterCount = m_guestExecutionWaiters.load(std::memory_order_acquire); - const uint32_t yieldInterval = (waiterCount != 0u) ? 64u : 100u; - if (++s_backEdgeYieldCounter < yieldInterval) - { - return false; - } - - s_backEdgeYieldCounter = 0u; - return true; + return ps2sched::yield_point(); } uint8_t PS2Runtime::Load8(uint8_t *rdram, R5900Context *ctx, uint32_t vaddr) @@ -2154,6 +2086,11 @@ void PS2Runtime::requestStop() ps2_syscalls::notifyRuntimeStop(); } +void PS2Runtime::requestStopFlagOnly() +{ + m_stopRequested.store(true, std::memory_order_relaxed); +} + bool PS2Runtime::isStopRequested() const { return m_stopRequested.load(std::memory_order_relaxed); @@ -2164,6 +2101,10 @@ void PS2Runtime::HandleIntegerOverflow(R5900Context *ctx) raiseCop0Exception(ctx, EXCEPTION_INTEGER_OVERFLOW); } +// Trampoline pointer used by the scheduler stop callback (non-capturing lambda). +// Set in run() before scheduler_init(); cleared after scheduler_shutdown() returns. +static PS2Runtime* g_stopRuntime = nullptr; + void PS2Runtime::run() { m_stopRequested.store(false, std::memory_order_relaxed); @@ -2175,11 +2116,12 @@ void PS2Runtime::run() ps2_syscalls::initializeGuestKernelState(m_memory.getRDRAM()); m_cpuContext.r[4] = _mm_setzero_si128(); m_cpuContext.r[5] = _mm_setzero_si128(); + // Bootstrap $sp at top of RAM, as the hardware loader does; the guest's + // crt0 immediately replaces it via SetupThread, which points $sp at the + // top of the game-chosen main stack. Callback stacks live in the + // kernel-reserved pool (see kAsyncCallbackStackFloor), not here, so this + // cannot collide with them. m_cpuContext.r[29] = _mm_set_epi64x(0, static_cast(PS2_RAM_SIZE - 0x10u)); - m_debugPc.store(m_cpuContext.pc, std::memory_order_relaxed); - m_debugRa.store(static_cast(_mm_extract_epi32(m_cpuContext.r[31], 0)), std::memory_order_relaxed); - m_debugSp.store(static_cast(_mm_extract_epi32(m_cpuContext.r[29], 0)), std::memory_order_relaxed); - m_debugGp.store(static_cast(_mm_extract_epi32(m_cpuContext.r[28], 0)), std::memory_order_relaxed); RUNTIME_LOG("Starting execution at address 0x" << std::hex << m_cpuContext.pc << std::dec); @@ -2188,34 +2130,24 @@ void PS2Runtime::run() Texture2D frameTex = LoadTextureFromImage(blank); UnloadImage(blank); - g_activeThreads.store(1, std::memory_order_relaxed); - std::atomic gameThreadFinished{false}; + // Initialize the fiber/pool scheduler. + ps2sched::scheduler_init(); + g_stopRuntime = this; + ps2sched::scheduler_set_stop_callback(+[]{ if (g_stopRuntime) g_stopRuntime->requestStopFlagOnly(); }); - std::thread gameThread([&]() - { - ThreadNaming::SetCurrentThreadName("GameThread"); - try - { - dispatchLoop(m_memory.getRDRAM(), &m_cpuContext); - uint32_t pc = m_debugPc.load(std::memory_order_relaxed); - RUNTIME_LOG("Game thread returned. PC=0x" << std::hex << pc - << " RA=0x" << static_cast(_mm_extract_epi32(m_cpuContext.r[31], 0)) << std::dec << std::endl); - } - catch (const std::exception &e) - { - std::cerr << "Error during program execution: " << e.what() << std::endl; - } - catch (...) - { - std::cerr << "Error during program execution: unknown exception" << std::endl; - } - g_activeThreads.fetch_sub(1, std::memory_order_relaxed); - gameThreadFinished.store(true, std::memory_order_release); }); + // Create the main guest fiber (tid=1). + uint8_t *rdram = m_memory.getRDRAM(); + { + const uint32_t entry = m_cpuContext.pc; + const uint32_t sp = static_cast(_mm_extract_epi32(m_cpuContext.r[29], 0)); + const uint32_t gp = static_cast(_mm_extract_epi32(m_cpuContext.r[28], 0)); + ps2sched::create_fiber(1, 1, entry, sp, gp, 0u, this, rdram); + } ps2_syscalls::EnsureVSyncWorkerRunning(m_memory.getRDRAM(), this); uint64_t tick = 0; - while (!isStopRequested() && g_activeThreads.load(std::memory_order_relaxed) > 0) + while (!isStopRequested()) { PS2_IF_AGRESSIVE_LOGS({ tick++; @@ -2284,54 +2216,10 @@ void PS2Runtime::run() requestStop(); - const auto joinDeadline = std::chrono::steady_clock::now() + std::chrono::seconds(2); - while (!gameThreadFinished.load(std::memory_order_acquire) && - std::chrono::steady_clock::now() < joinDeadline) - { - std::this_thread::sleep_for(std::chrono::milliseconds(1)); - } - - if (gameThread.joinable()) - { - if (gameThreadFinished.load(std::memory_order_acquire)) - { - gameThread.join(); - } - else - { - std::cerr << "[run] game thread did not stop within timeout; detaching" << std::endl; - gameThread.detach(); - } - } - - const auto workerDeadline = std::chrono::steady_clock::now() + std::chrono::milliseconds(1000); - while (g_activeThreads.load(std::memory_order_relaxed) > 0 && - std::chrono::steady_clock::now() < workerDeadline) - { - std::this_thread::sleep_for(std::chrono::milliseconds(1)); - } - - if (g_activeThreads.load(std::memory_order_relaxed) > 0) - { - requestStop(); - const auto finalWorkerDeadline = std::chrono::steady_clock::now() + std::chrono::milliseconds(1000); - while (g_activeThreads.load(std::memory_order_relaxed) > 0 && - std::chrono::steady_clock::now() < finalWorkerDeadline) - { - std::this_thread::sleep_for(std::chrono::milliseconds(1)); - } - } - - if (g_activeThreads.load(std::memory_order_relaxed) == 0) - { - ps2_syscalls::joinAllGuestHostThreads(); - } - else - { - std::cerr << "[run] guest host threads did not stop within timeout; detaching remaining worker threads" - << std::endl; - ps2_syscalls::detachAllGuestHostThreads(); - } + // Signal all guest fibers to stop and join the pool threads. + ps2sched::scheduler_shutdown(); + ps2sched::scheduler_set_stop_callback(nullptr); + g_stopRuntime = nullptr; if (m_debugUiInitialized && m_debugUiShutdownCallback) { @@ -2340,12 +2228,4 @@ void PS2Runtime::run() } UnloadTexture(frameTex); CloseWindow(); - - const int remainingThreads = g_activeThreads.load(std::memory_order_relaxed); - RUNTIME_LOG("[run] exiting loop, activeThreads=" << remainingThreads); - if (remainingThreads > 0) - { - std::cerr << "[run] warning: " << remainingThreads - << " guest worker thread(s) still active during shutdown." << std::endl; - } } diff --git a/ps2xRuntime/src/lib/ps2_scheduler.cpp b/ps2xRuntime/src/lib/ps2_scheduler.cpp new file mode 100644 index 000000000..6bcb715c6 --- /dev/null +++ b/ps2xRuntime/src/lib/ps2_scheduler.cpp @@ -0,0 +1,1169 @@ +// ps2_scheduler.cpp — N=1 fiber cooperative scheduler for PS2Recomp. +// +// Architecture: +// - Exactly ONE dedicated host OS thread (g_guest_thread, the "guest executor") +// runs all guest fibers. This eliminates cross-thread swapcontext UB +// structurally — a ucontext_t is only ever saved and resumed on the one thread. +// - Guest threads are mapped to FiberContext objects; each owns a PS2Fiber +// (ucontext_t on POSIX with guard page; joinable pthread on Vita). +// - Host threads (IRQ worker, alarm worker, RPC worker) that need to run guest +// code acquire the guest token via AsyncGuestScope (RAII). +// - Cooperative yield points sampled every 128 back-edges via yield_point(). + +#include "ps2_scheduler_internal.h" +#include "ps2_fiber.h" +#include +#include +#include "Kernel/Syscalls/Helpers/ThreadExit.h" // canonical ThreadExitException +#include "Kernel/Syscalls/Interrupt.h" // stopInterruptWorker +#include "Kernel/Syscalls/Sync.h" // stopAlarmWorker + +#include +#include +#include +#include +#include + +// g_currentThreadId — -1 means this host thread is not currently running a fiber. +thread_local int g_currentThreadId = -1; + +// Globals +std::mutex g_sched_mutex; +std::condition_variable g_sched_cv; +FiberContext* g_run_queue = nullptr; +FiberContext* g_running_fiber = nullptr; +bool g_guest_token_held_by_host = false; +// True on a host (non-fiber) worker thread that currently holds the guest token +// (acquired via async_guest_begin, released via async_guest_end). Used to assert +// the token is released by the same thread that acquired it, and so block_current +// can report whether the calling non-fiber worker owns the token. +static thread_local bool tls_holds_guest_token = false; +std::atomic g_stop{false}; +// Host workers currently blocked in async_guest_begin() waiting for the guest +// token. Read by the executor's resume predicate (it must sleep while a worker +// is parked) and by yield_point / diagnostics via ps2sched::host_token_waiters(). +static std::atomic g_host_token_waiters{0}; +// Number of times a host worker has been GRANTED the token via +// async_guest_begin(). Written under g_sched_mutex; read by the executor's +// resume predicate (also under g_sched_mutex) to implement a fair handoff: +// the executor defers to parked workers until at least one grant happens, +// then may resume fibers again even if more workers are waiting. +static uint64_t g_host_token_grants = 0; + +std::unordered_map> g_fiber_map; +std::thread g_guest_thread; + +// True only on the single guest executor thread (g_guest_thread). Set once at the +// top of guest_executor_main, before any fiber can be resumed. Read by +// ps2fiber_on_executor_thread(). A thread_local bool is lock-free and avoids the +// (typically lock-based) std::atomic on the context-switch hot +// path. Always correct: only the executor thread ever runs guest_executor_main. +static thread_local bool tls_is_executor_thread = false; + +// Thread-locals +thread_local FiberContext* tls_current_fiber = nullptr; +thread_local uint32_t tls_backedge_counter = 0; + +// True on any guest execution context: the executor thread (all backends), or +// a fiber's own OS thread on the pthread backend (which sets tls_current_fiber +// for the lifetime of the fiber, since that thread owns the execution slot +// whenever the fiber runs). Backs both is_guest_thread() and the +// async_guest_begin() abort guard, so the two can never drift apart. +static inline bool on_guest_execution_slot() +{ + return tls_is_executor_thread || tls_current_fiber != nullptr; +} + +// Fiber exit hook (set once by Thread.cpp) +void (*g_fiber_exit_hook)(int tid, uint8_t* rdram, R5900Context* ctx, PS2Runtime* rt) = nullptr; + +// Stop callback — set by scheduler_set_stop_callback(); invoked by +// scheduler_shutdown() before joining g_guest_thread. Read/written under +// g_sched_mutex. nullptr-safe. +static void (*g_request_runtime_stop_fn)() = nullptr; + +static constexpr size_t kFiberStackBytes = 1024 * 1024; // 1 MiB + +// Fatal helper — terminate on invariant violation +#define SCHED_REQUIRE(cond, msg) \ + do \ + { \ + if (!(cond)) \ + { \ + std::fprintf(stderr, "FATAL [ps2sched]: " msg "\n"); \ + std::terminate(); \ + } \ + } while (0) + +// --------------------------------------------------------------------------- +// fiber_trampoline — entry point of every PS2Fiber. +// +// Runs on g_guest_thread (the guest executor). Catches ALL exceptions so the +// executor can continue scheduling other fibers after this one exits. +// --------------------------------------------------------------------------- +static void fiber_trampoline(void* arg) +{ + FiberContext* fc = static_cast(arg); + SCHED_REQUIRE(fc != nullptr, "fiber_trampoline null arg"); + // guest_executor_main set tls_current_fiber on the executor thread before + // resuming us (the N=1 / ucontext design runs all guest code on that one + // thread). On the pthread backend each fiber runs on its own OS thread, so + // tls_current_fiber is only set on the executor thread and this check does + // not apply — ps2fiber_thread_main sets tls_current_fiber_ptr instead. +#if !defined(PS2X_FIBER_PTHREAD) + SCHED_REQUIRE(fc == tls_current_fiber, "fiber_trampoline fc mismatch"); +#endif + + // pthread backend: each fiber runs on its own OS thread. guest_executor_main + // sets tls_current_fiber only on the executor thread, so fiber OS threads + // never have it set — breaking arm_park, block_current, and + // current_fiber_token. Publish the FiberContext* on this fiber's thread now + // so the entire blocking protocol works correctly under PS2X_FIBER_PTHREAD. + // This is also what makes the fiber's OS thread a guest execution context + // for on_guest_execution_slot(): it owns the execution slot whenever it + // runs (the executor is blocked in ps2fiber_resume until the fiber yields + // back), so is_guest_thread() reads true here and shared helpers take the + // synchronous path instead of borrowing the token via async_guest_begin() + // — which would park waiting for this very fiber to stop: a self-deadlock. + // tls_current_fiber is per-OS-thread TLS and dies with the fiber's thread, + // so no reset is needed. +#if defined(PS2X_FIBER_PTHREAD) + tls_current_fiber = fc; +#endif + + PS2Runtime* rt = fc->rt; // from struct, not smuggled GPRs + uint8_t* rdram = fc->rdram; + + try + { + rt->dispatchLoop(rdram, &fc->cpu); + } + catch (const ThreadExitException&) + { + // Cooperative termination — fall through. + } + catch (const std::exception& e) + { + std::fprintf(stderr, "[ps2sched] fiber tid=%d uncaught std::exception: %s\n", + fc->tid, e.what()); + } + catch (...) + { + std::fprintf(stderr, "[ps2sched] fiber tid=%d uncaught unknown exception\n", fc->tid); + } + + // Mark Exiting BEFORE running the exit hook. If a guest exit handler + // makes a blocking syscall (block_current -> ps2fiber_yield), the executor's + // post-resume code sees Exiting and RE-ENQUEUES the fiber (treated like a + // cooperative yield) instead of freeing it — the trampoline still has a live + // stack frame here. When the wakeup arrives the fiber resumes inside the + // hook and continues. Only after the hook returns do we transition to + // Finished, which is the executor's signal to free the fiber. + { + std::lock_guard lk(g_sched_mutex); + fc->state = FiberContext::State::Exiting; + } + + if (g_fiber_exit_hook) + { + // The exit hook runs guest exit handlers (arbitrary recompiled code). If + // one calls ExitThread it throws ThreadExitException; any such throw must + // NOT cross the swapcontext boundary below (UB). Swallow it: the fiber is + // already exiting. + try + { + g_fiber_exit_hook(fc->tid, rdram, &fc->cpu, rt); + } + catch (const ThreadExitException&) + { + // Guest exit handler called ExitThread — normal exit completion. + } + catch (const std::exception& e) + { + std::fprintf(stderr, + "[ps2sched] fiber tid=%d exit hook threw std::exception: %s\n", + fc->tid, e.what()); + } + catch (...) + { + std::fprintf(stderr, + "[ps2sched] fiber tid=%d exit hook threw unknown exception\n", + fc->tid); + } + } + + { + std::lock_guard lk(g_sched_mutex); + fc->state = FiberContext::State::Finished; + } + g_sched_cv.notify_all(); // wake join_fiber waiters / executor + + // pthread backend: clear tls_current_fiber on the fiber's OS thread before + // handing control back to the executor. This mirrors the executor clearing it + // after ps2fiber_resume returns (line ~211) and ensures no dangling reference + // to fc remains in TLS after the fiber is freed. +#if defined(PS2X_FIBER_PTHREAD) + tls_current_fiber = nullptr; +#endif + +#if !defined(PS2X_FIBER_PTHREAD) + ps2fiber_yield(); // back to guest_executor_main; never returns + SCHED_REQUIRE(false, "fiber_trampoline resumed after Finished"); +#endif + // pthread backend: ps2fiber_thread_main returns from fn() here, sets + // PS2Fiber::finished=true, clears TLS, and posts yield_sem — the fiber OS + // thread then exits naturally and ps2fiber_free can pthread_join cleanly. +} + +// --------------------------------------------------------------------------- +// guest_executor_main — the N=1 loop +// --------------------------------------------------------------------------- +static void guest_executor_main() +{ + // Publish our identity BEFORE anything can resume a fiber. ps2fiber_resume + // (called below) asserts it runs on this thread via + // ps2fiber_on_executor_thread(), which reads this TLS flag; it also makes + // on_guest_execution_slot() (and so is_guest_thread()) true here. + tls_is_executor_thread = true; + + std::unique_lock lk(g_sched_mutex); + // The wait predicate MUST be exactly (canQuit || canRun). A predicate that + // short-circuits on bare g_stop can be instantly true while NEITHER the + // break condition nor the pop condition holds (g_stop set with a non-empty + // run queue and a parked host worker): the loop then continue's, cv.wait + // sees the predicate already true and returns WITHOUT releasing + // g_sched_mutex, and the executor livelocks — spinning under the lock and + // starving the very waiters it is gated on. + auto canQuit = [] + { + return g_stop && g_run_queue == nullptr; + }; + // Fair handoff to parked host workers. async_guest_begin()'s wakeup + // predicate is (g_running_fiber == nullptr && !token_held), but the only + // window where g_running_fiber is null is while THIS loop holds + // g_sched_mutex between resumes. Without a waiters gate the executor's own + // cv.wait predicate is instantly true again (the fiber is re-enqueued by + // the post-resume code), so it re-resumes the fiber without ever sleeping, + // and a parked host worker (e.g. the interrupt worker delivering VBlank) + // can never win the token: the guest spins waiting for ticks the starved + // worker cannot deliver. + // + // The gate must NOT be a bare (waiters == 0), though: sustained worker + // traffic (workers looping async_guest_begin/end, as in the X2 CV- + // contention test) would then starve fibers in the opposite direction. + // Instead the executor defers until AT LEAST ONE worker has been granted + // the token since it started deferring (g_host_token_grants advanced), + // after which it may resume fibers even if more workers are waiting — + // strict alternation under pressure, zero cost when no worker is parked. + bool deferringToWaiters = false; + uint64_t grantsAtDefer = 0; + auto canRun = [&] + { + if (g_run_queue == nullptr || + g_running_fiber != nullptr || + g_guest_token_held_by_host) + { + return false; + } + if (g_host_token_waiters.load(std::memory_order_relaxed) == 0) + { + deferringToWaiters = false; + return true; + } + if (!deferringToWaiters) + { + // A worker is parked and we have not yet deferred: start deferring + // (sleep until a grant happens). + deferringToWaiters = true; + grantsAtDefer = g_host_token_grants; + return false; + } + if (g_host_token_grants != grantsAtDefer) + { + // At least one worker got the token while we were deferring; take + // our turn even if more workers are waiting (fairness). + deferringToWaiters = false; + return true; + } + return false; + }; + while (true) + { + g_sched_cv.wait(lk, [&] + { + return canQuit() || canRun(); + }); + + if (canQuit()) break; + // canRun() held at the wait's final predicate evaluation and + // g_sched_mutex has been held continuously since, so the state cannot + // have changed. Do NOT re-call canRun() here: it is stateful (the + // deferral bookkeeping above), and a second evaluation while workers + // are still waiting would re-arm the deferral and skip our turn. + // canQuit and canRun are mutually exclusive (queue null vs non-null), + // so reaching this line means canRun held. + + FiberContext* fc = pop_head_locked(); + SCHED_REQUIRE(fc != nullptr, "executor popped null with non-empty predicate"); + + fc->state = FiberContext::State::Running; + g_running_fiber = fc; + // Guest code runs on THIS (executor) thread, so publish the guest + // identity here before resuming. + tls_current_fiber = fc; + g_currentThreadId = fc->tid; + + lk.unlock(); + ps2fiber_resume(fc->fiber); // runs until yield or finish + lk.lock(); + + tls_current_fiber = nullptr; + g_currentThreadId = -1; + g_running_fiber = nullptr; + + if (fc->state == FiberContext::State::Running || + fc->state == FiberContext::State::Exiting) + { + // Running: cooperative yield (maybe_yield / yield_point). The fiber + // normally enqueues itself before yielding; enqueue_locked is + // idempotent so this is a safe backstop. + // Exiting: the exit hook yielded (e.g. a blocking syscall in a + // guest exit handler). It is NOT done yet — re-enqueue so it runs + // to completion. Do NOT free: fiber_trampoline still has a live + // stack frame. + enqueue_locked(fc); + } + else if (fc->state == FiberContext::State::Blocked) + { + // A waker fired during the park window (between the fiber + // setting state=Blocked and ps2fiber_yield returning here). It could + // not enqueue safely (fc->next was owned by the running fiber), so it + // set wake_pending instead. Honour it now — UNLESS the fiber was + // suspended during the park window (a suspendCount > 0 means it + // must stay off the run queue). The matching clear_suspend() + // will wake it when the suspend is lifted. + if (fc->wake_pending && + fc->suspendCount.load(std::memory_order_relaxed) == 0) + { + fc->wake_pending = false; + enqueue_locked(fc); // sets state=Ready + } + // else: genuinely Blocked (or suspended); a future waker / resume + // will enqueue it. Leave wake_pending as-is so a later resume can + // honour it. + } + else if (fc->state == FiberContext::State::Finished) + { + // Erase the map entry while the lock is held so that a borrowed worker + // racing in async_guest_begin cannot call StartThread on this tid and + // insert a new FiberContext entry that we would then unconditionally clobber + // after dropping and re-acquiring the lock. Moving out the unique_ptr + // under the lock prevents the window between unlock and re-lock from + // allowing tid reuse that would be silently destroyed. + int tid = fc->tid; + std::unique_ptr dead; + auto it = g_fiber_map.find(tid); + if (it != g_fiber_map.end()) + dead = std::move(it->second); + g_fiber_map.erase(tid); // tid is now free for reuse under the lock + PS2Fiber* deadFiber = dead ? dead->fiber : nullptr; + if (dead) dead->fiber = nullptr; // suppress destructor double-free + lk.unlock(); + ps2fiber_free(deadFiber); // munmap outside the lock from a local + // dead destructs here (FiberContext body) — fiber already freed above + lk.lock(); // re-acquire before next wait or break + } + // else Fresh: impossible after a resume. + + g_sched_cv.notify_all(); + } + lk.unlock(); +} + +// --------------------------------------------------------------------------- +// wake_locked — race-safe wakeup. MUST be called with g_sched_mutex held. +// Idempotent and state-aware: safe to call whenever the caller has already +// applied whatever suspendCount/terminate gating IT cares about (make_ready / +// enqueue_external_wakeup_validated gate on suspendCount==0 before calling; +// request_terminate / clear_suspend force suspendCount to the +// value they intend first, then call this unconditionally -- see their +// comments for why gating on state==Blocked alone drops wakes that race the +// fiber's own publish/arm-park window). If fc is mid-park (state==Blocked or +// even still Running, but ps2fiber_yield has not yet returned to the +// executor), g_running_fiber still points at fc: in that window we MUST NOT +// touch fc->next, so we set wake_pending and let block_current() or the +// executor's post-resume code consume it. Otherwise (fc genuinely parked, +// off-queue, not the running fiber) enqueue normally; if already queued, +// no-op. +// --------------------------------------------------------------------------- +static void wake_locked(FiberContext* fc) +{ + // Called under g_sched_mutex. Idempotent: if already queued, nothing to do. + if (fc->in_run_queue) + { + return; + } + + if (g_running_fiber == fc) + { + // The fiber is still the running fiber. This covers two windows: + // 1. mid-park: state==Blocked set (by arm_park or block_current) but + // ps2fiber_yield has not yet returned control to the executor. + // 2. state==Blocked armed via arm_park(), the syscall published to + // an object wait-list and a waker fired before block_current(). + // In both cases fc->next is owned by the still-executing fiber, so we + // MUST NOT enqueue_locked. Record the wakeup; block_current() (case 2) + // or the executor post-resume code (case 1) honours wake_pending. + fc->wake_pending = true; + } + else + { + // Fully parked (state==Blocked, not the running fiber): enqueue directly. + enqueue_locked(fc); + } +} + +void ps2sched::scheduler_init() +{ + { + std::lock_guard lk(g_sched_mutex); + g_run_queue = nullptr; + g_running_fiber = nullptr; + g_guest_token_held_by_host = false; + g_stop.store(false, std::memory_order_relaxed); + g_fiber_map.clear(); + } + SCHED_REQUIRE(!g_guest_thread.joinable(), "scheduler_init while executor running"); + // Fresh scheduler epoch starts with zero counted guest threads; heals any + // counter drift from a prior epoch's shutdown races. g_activeThreads (declared + // extern in ps2_syscalls.h, already visible here via Interrupt.h/Sync.h) has + // no concurrent decrementer at this point: the prior executor is proven + // joined above and the new one has not been spawned yet, so a plain store + // is sound (unlike in notifyRuntimeStop, which runs concurrently with a + // possibly still-live fiber). + g_activeThreads.store(0, std::memory_order_relaxed); + // The executor thread publishes tls_is_executor_thread itself at the top of + // guest_executor_main, before any fiber resume. + g_guest_thread = std::thread(guest_executor_main); +} + +void ps2sched::scheduler_set_stop_callback(void (*fn)()) +{ + std::lock_guard lk(g_sched_mutex); + g_request_runtime_stop_fn = fn; +} + +void ps2sched::scheduler_shutdown() +{ + // Stop and JOIN the host workers (IRQ/vsync + alarm) FIRST, before we + // tear down the fiber map. Those workers call enqueue_external_wakeup_validated() + // into g_fiber_map; joining them here guarantees no such call can race the + // teardown below, regardless of whether notifyRuntimeStop() already ran. + // Both stop functions are idempotent (no-op if already stopped/joined). + ps2_syscalls::stopInterruptWorker(); + ps2_syscalls::stopAlarmWorker(); + + { + std::lock_guard lk(g_sched_mutex); + g_stop.store(true, std::memory_order_release); + // Re-terminate every fiber so blocked ones wake and unwind. + for (auto& [tid, fc] : g_fiber_map) + { + if (fc->state == FiberContext::State::Blocked) + { + fc->terminateRequested.store(true, std::memory_order_relaxed); + fc->suspendCount.store(0, std::memory_order_relaxed); + wake_locked(fc.get()); + } + else if (fc->state == FiberContext::State::Exiting) + { + // Exiting fibers yielded inside their exit hook (e.g. a blocking + // syscall in a guest exit handler). They are off-queue but still + // alive with a live trampoline stack frame. Do NOT set + // terminateRequested (the exit path is already underway and an + // unwind here would abandon the hook); just re-enqueue so the + // executor runs them to completion before the quit predicate + // (g_stop && run_queue==nullptr) is satisfied. + fc->suspendCount.store(0, std::memory_order_relaxed); + wake_locked(fc.get()); + } + else + { + // Fresh / Ready / Running / Finished: leave terminateRequested set + // so a still-running fiber unwinds at its next yield_point. + fc->terminateRequested.store(true, std::memory_order_relaxed); + } + } + } + g_sched_cv.notify_all(); + + // Request the runtime stop so dispatchLoop's while(!isStopRequested()) exits + // between dispatched functions. Invoked OUTSIDE g_sched_mutex: the callback + // runs runtime.requestStopFlagOnly(), which sets its own atomic and must not + // nest under g_sched_mutex. + void (*stopFn)() = nullptr; + { + std::lock_guard lk(g_sched_mutex); + stopFn = g_request_runtime_stop_fn; + } + if (stopFn) stopFn(); + + if (g_guest_thread.joinable()) g_guest_thread.join(); + + // Executor has exited. No guest code runs now. Free any remaining fibers. + std::lock_guard lk(g_sched_mutex); + if (!g_fiber_map.empty()) + { + std::fprintf(stderr, "[ps2sched] WARNING: %zu fiber(s) remain after executor exit\n", + g_fiber_map.size()); + } + for (auto& [tid, fc] : g_fiber_map) + { + if (fc->fiber) + { + ps2fiber_free(fc->fiber); + fc->fiber = nullptr; + } + } + g_fiber_map.clear(); + g_run_queue = nullptr; +} + +void ps2sched::create_fiber(int tid, int priority, uint32_t entry, + uint32_t sp, uint32_t gp, uint32_t arg, + PS2Runtime* rt, uint8_t* rdram) +{ + // Refuse to create fibers once shutdown has begun. A fiber running its + // terminate path can still reach StartThread -> create_fiber; if we allowed + // it, the executor could partially run the new fiber during teardown and + // leak its stack if it parks. Throwing here reuses StartThread's existing + // allocation-failure path (it resets the thread to dormant and returns + // KE_NO_MEMORY). + { + std::lock_guard lk(g_sched_mutex); + if (g_stop) + { + throw std::runtime_error("[ps2sched] create_fiber refused: scheduler stopping"); + } + } + + // The R5900Context constructor zeroes all registers and sets the documented + // reset defaults (cop0_random=47, vu0_q=1.0). + auto fc = std::make_unique(); + fc->tid = tid; + fc->priority = priority; + fc->rt = rt; + fc->rdram = rdram; + + fc->cpu.pc = entry; + SET_GPR_U32(&fc->cpu, 29, sp); // $sp + SET_GPR_U32(&fc->cpu, 28, gp); // $gp + SET_GPR_U32(&fc->cpu, 4, arg); // $a0 (passed directly, not smuggled through other registers) + SET_GPR_U32(&fc->cpu, 31, 0u); // $ra = 0 (returns -> pc==0 -> dispatchLoop break) + + PS2Fiber* fiber = ps2fiber_alloc(fiber_trampoline, fc.get(), kFiberStackBytes); + if (!fiber) + { + throw std::runtime_error("[ps2sched] fiber allocation failed"); + } + fc->fiber = fiber; +#if defined(PLATFORM_VITA) || defined(PS2X_FIBER_PTHREAD) + ps2fiber_set_tid(fiber, tid); +#endif + fc->state = FiberContext::State::Fresh; + + FiberContext* raw = fc.get(); + { + std::lock_guard lk(g_sched_mutex); + g_fiber_map[tid] = std::move(fc); + enqueue_locked(raw); // Fresh -> Ready + } + g_sched_cv.notify_all(); +} + +void ps2sched::request_terminate(int tid) +{ + std::lock_guard lk(g_sched_mutex); + FiberContext* fc = fiber_for(tid); + if (!fc) return; + fc->terminateRequested.store(true, std::memory_order_relaxed); + // Force the suspend gate open unconditionally (not only when already + // Blocked) so a suspended-and-parked target always becomes wakeable, and + // so a suspend that races concurrently cannot re-close the gate behind + // this terminate (mirrors clear_suspend's force-to-0). + fc->suspendCount.store(0, std::memory_order_relaxed); + // Always route through wake_locked, exactly like every other waker + // (make_ready / enqueue_external_wakeup_validated / clear_suspend). + // wake_locked is idempotent and state-aware: + // in_run_queue -> no-op (already runnable) + // g_running_fiber==fc -> wake_pending=true (the fiber published to an + // object wait-list and called arm_park(), state + // is already Blocked, but has not yet reached + // block_current()/ps2fiber_yield() -- OR it has + // published but not even called arm_park() yet, + // in which case state is still Running and this + // is the mid-park window too) + // else (Blocked) -> enqueue_locked + // Routing unconditionally (not only when state==Blocked) is required: a + // terminate landing in the window between a fiber publishing itself to an + // object wait-list (state still Running) and its own arm_park()/ + // block_current() call would otherwise be recorded in terminateRequested + // but never wake the fiber; if the object is never independently signaled + // afterward, the fiber parks forever and TerminateThread's join_fiber() + // hangs waiting for it to finish. A resulting spurious wake_pending on a + // fiber that was never actually about to park is harmless: every blocking + // wait in this codebase is a Mesa loop that re-checks its real condition + // after waking (and, here, also checks ThreadInfo::terminated -- see + // WaitSema/SleepThread/WaitEventFlag), and wake_pending is always consumed + // exactly once by block_current() or by the executor's post-resume Blocked + // handling. + wake_locked(fc); + g_sched_cv.notify_all(); +} + +void ps2sched::join_fiber(int tid) +{ + // Capture the target's identity as {tid, generation} once, under + // g_sched_mutex, before waiting at all. This defends against tid + // recycling (ABA): the target can finish, be erased from g_fiber_map by + // the executor, and have `tid` reused by a brand-new, unrelated fiber + // between our predicate evaluations below. Without the generation check, + // a joiner whose predicate re-derives "the fiber currently at tid" on + // every iteration would silently start waiting on (and re-applying the + // join-priority floor to) the NEW fiber instead of detecting that its + // original target is long gone -- an over-wait that can hang or block + // far longer than intended. targetDone() below re-derives the *current* + // occupant of tid every time and compares its generation against the one + // captured here; a mismatch (or a missing entry) means the original + // target is gone, full stop, regardless of what now lives at that tid. + uint32_t targetGeneration = 0; + { + std::lock_guard lk(g_sched_mutex); + FiberContext* t = fiber_for(tid); + if (!t) return; // already gone before we ever started waiting. + targetGeneration = t->generation; + } + + // Must be called with g_sched_mutex held. True once the ORIGINAL target + // (identified by {tid, targetGeneration}) is no longer joinable: gone + // from the map, tid recycled to a different fiber, or Finished. + auto targetDone = [&]() + { + FiberContext* t = fiber_for(tid); + return !t || t->generation != targetGeneration || + t->state == FiberContext::State::Finished; + }; + + // Non-fiber path: a host thread (test harness, RPC worker, etc.) called + // TerminateThread for another tid. We cannot cooperatively yield, so just + // wait on g_sched_cv until the target (by identity) is done. + if (ps2fiber_current() == nullptr) + { + std::unique_lock lk(g_sched_mutex); + g_sched_cv.wait(lk, [&]() { return targetDone(); }); + return; + } + + // Called from a fiber (TerminateThread of another tid). Cooperative: yield + // until the target (by identity) is done. + FiberContext* self = tls_current_fiber; + + while (true) + { + bool done = false; + { + std::lock_guard lk(g_sched_mutex); + FiberContext* t = fiber_for(tid); + done = (!t || t->generation != targetGeneration || + t->state == FiberContext::State::Finished); + // Once `done` is true (including the generation-mismatch case), + // the join-priority floor must stop being re-applied: t may now + // be a completely unrelated fiber that happens to occupy tid, and + // self's priority is restored below regardless. + if (!done && self && t) + { + // Per-iteration floor: ensure self runs strictly AFTER the target + // (higher priority number == lower scheduling priority). The + // target's priority may change between iterations, so re-apply. + const int floor = t->priority; + if (self->priority < floor) + { + if (!self->joinFloorActive) + { + // First lowering: remember the real priority to restore. + self->joinFloorActive = true; + self->joinSavedPriority = self->priority; + } + // self is the running fiber here, so it is not in the queue; + // just set the field for the next enqueue. + self->priority = floor; + } + } + } + if (done) break; + ps2fiber_yield(); // let the executor run the target; it will finish + } + + // Restore the joiner's real priority. joinSavedPriority reflects the latest + // value written by any concurrent ChangeThreadPriority (update_priority keeps + // it current while the floor is active), so we never lose a reprioritize. + if (self) + { + std::lock_guard lk(g_sched_mutex); + if (self->joinFloorActive) + { + const int restore = self->joinSavedPriority; + self->joinFloorActive = false; + if (self->priority != restore) + { + // self is the currently running fiber and cannot be in the + // run queue — assert defensively then update the priority. + // It will be re-enqueued with the new priority at the next + // cooperative yield. + SCHED_REQUIRE(!self->in_run_queue, + "join_fiber: running fiber should not be in run queue"); + self->priority = restore; + } + } + } +} + +void ps2sched::update_priority(int tid, int newPriority) +{ + std::lock_guard lk(g_sched_mutex); + FiberContext* fc = fiber_for(tid); + if (!fc) return; + + // If this fiber is currently inside join_fiber with a temporary priority floor + // in effect, the game-requested priority must be recorded as the value to + // restore on join exit, NOT written over the temporary floor. Otherwise either + // join_fiber's restore would lose this change, or this write would let the + // joiner outrun the target and spin the join. + if (fc->joinFloorActive) + { + fc->joinSavedPriority = newPriority; + return; + } + + if (fc->priority == newPriority) return; + bool wasQueued = fc->in_run_queue; + if (wasQueued) remove_locked(fc); + fc->priority = newPriority; + if (wasQueued) enqueue_locked(fc); + // If the target is the running fiber, its new priority takes effect at the next + // yield_point (ChangeThreadPriority calls maybe_yield right after). +} + +// --------------------------------------------------------------------------- +// arm_park — set state=Blocked and leaves any pending wakeup recorded by a +// waker intact (block_current consumes it). After this returns, the gates in +// make_ready / enqueue_external_wakeup_validated see state==Blocked and route through +// wake_locked, which (because the fiber is still g_running_fiber) records the +// wakeup in wake_pending. block_current() then observes wake_pending and does +// not park, so no wakeup is lost in the publish window. +// --------------------------------------------------------------------------- +void ps2sched::arm_park() +{ + FiberContext* fc = tls_current_fiber; + if (fc == nullptr) + { + // Borrowed host worker. There is no fiber to arm; block_current() returns + // a non-fiber result from this same thread. No-op here. (Callers gate this + // call on ps2fiber_current() != nullptr, so reaching here from a non-fiber + // should not happen; stay defensive.) + return; + } + std::lock_guard lk(g_sched_mutex); + // Do NOT clear wake_pending here. arm_park may run AFTER the fiber has been + // published to an object wait-list (see WaitSema), so a wakeup recorded by a + // waker between publish and arm_park must survive. block_current() consumes + // wake_pending; if none is pending it confirms Blocked. + fc->state = FiberContext::State::Blocked; +} + +// --------------------------------------------------------------------------- +// block_current — honour a wakeup that arrived in the arm/publish window. +// Returns a BlockResult enum describing the four possible outcomes. +// --------------------------------------------------------------------------- +ps2sched::BlockResult ps2sched::block_current() +{ + FiberContext* fc = tls_current_fiber; + if (fc == nullptr) + { + // A borrowed IRQ/alarm/RPC worker (running guest code under + // AsyncGuestScope) called a blocking syscall. We cannot park a host + // worker on the fiber scheduler. Report whether this worker actually holds + // the guest token so the caller only drops/reacquires a token it owns. + bool ownsToken; + { + std::lock_guard lk(g_sched_mutex); + ownsToken = g_guest_token_held_by_host && tls_holds_guest_token; + } + return ownsToken ? BlockResult::NonFiberOwner + : BlockResult::NonFiberNoTok; + } + bool throwTerminate = false; + bool wokenInWindow = false; + { + std::lock_guard lk(g_sched_mutex); + // Shutdown safety: a terminate-requested fiber must NEVER park again. + // Throw ThreadExitException (outside the lock, below) so it unwinds the + // fiber stack through any Mesa wait-loop. This handles the case where the + // syscall's own info->terminated flag was not set by a caller that used + // scheduler_shutdown() without notifyRuntimeStop(), which would otherwise + // leave the fiber spinning forever on a count-0 semaphore. + // + // Deliberately gated on g_stop, NOT on terminateRequested alone (a + // single-thread TerminateThread(otherTid) sets terminateRequested via + // request_terminate() without setting g_stop). Every Mesa-loop caller + // of block_current() (WaitSema, SleepThread/WakeupThread, WaitEventFlag, + // ...) does its OWN mandatory bookkeeping cleanup -- removing itself + // from an object's wait-list, decrementing a waiters count, etc. -- + // AFTER block_current() returns and BEFORE it checks + // ThreadInfo::terminated and throws itself. If block_current() threw + // directly here for a bare terminateRequested, that cleanup would be + // skipped, leaving stale wait-list entries / wrong waiters counts on + // an object that other, unrelated threads keep using afterward (e.g. + // ReferSemaStatus's wait_threads count, or a future SignalSema's + // wait-list scan). During a full g_stop shutdown this does not matter + // (the runtime is tearing down and such objects are about to be + // destroyed anyway), which is why the throw is safe -- and needed -- + // only in that case. request_terminate()'s unconditional wake_locked() + // call (see its comment) already guarantees a plain terminateRequested + // reaches every Mesa loop's own info->terminated check via a normal + // wake, so no correctness gap remains from keeping this gated. + if (g_stop.load(std::memory_order_relaxed) && fc->terminateRequested.load(std::memory_order_relaxed)) + { + fc->wake_pending = false; + fc->state = FiberContext::State::Running; // never parked + throwTerminate = true; + } + // A waker may have fired between arm_park() and here (after the syscall + // published to the object wait-list). wake_locked recorded it in + // wake_pending. If so, consume it and DO NOT park. + else if (fc->wake_pending) + { + fc->wake_pending = false; + fc->state = FiberContext::State::Running; // we never actually parked + wokenInWindow = true; + } + else + { + // No wakeup yet. Confirm Blocked (arm_park already set it; this is a + // harmless re-affirmation and also covers callers that did not arm). + fc->wake_pending = false; + fc->state = FiberContext::State::Blocked; + } + } + if (throwTerminate) throw ThreadExitException(); + if (wokenInWindow) return BlockResult::WokenInWindow; + ps2fiber_yield(); // executor sees Blocked; re-enqueues iff wake_pending. + // Resumes here when the executor pops fc again (state set to Running by it). + return BlockResult::Parked; +} + +// make_ready (suspendCount gate) +void ps2sched::make_ready(int tid) +{ + bool notify = false; + { + std::lock_guard lk(g_sched_mutex); + FiberContext* fc = fiber_for(tid); + // No state==Blocked gate: a waker that fires in the publish/arm window + // (state still Running, fiber still g_running_fiber) must reach + // wake_locked so it can record wake_pending. wake_locked itself routes: + // in_run_queue -> no-op (already queued) + // g_running_fiber==fc -> wake_pending=true (mid-park / publish window) + // else (Blocked) -> enqueue_locked + // The suspendCount gate stays: wake_locked does not check it, and a + // suspended fiber must not be enqueued; clear_suspend wakes it when + // the suspend is lifted. + if (fc && fc->suspendCount.load(std::memory_order_relaxed) == 0) + { + wake_locked(fc); + notify = true; + } + } + if (notify) g_sched_cv.notify_all(); +} + +static inline uint64_t make_fiber_token(const FiberContext* fc) +{ + return (static_cast(fc->generation) << 32) | + static_cast(static_cast(fc->tid)); +} + +uint64_t ps2sched::current_fiber_token() +{ + FiberContext* fc = tls_current_fiber; + return fc ? make_fiber_token(fc) : 0u; +} + +::DispatchHistory& ps2sched::current_dispatch_history() noexcept +{ + if (FiberContext* fc = tls_current_fiber) + { + return fc->dispatchHistory; + } + // Persistent per-OS-thread instance: borrowed host workers, the executor + // between fibers, and direct non-fiber callers all land here. + thread_local ::DispatchHistory s_nonFiberHistory; + return s_nonFiberHistory; +} + +::SyscallOverrideStack& ps2sched::current_active_syscall_overrides() noexcept +{ + if (FiberContext* fc = tls_current_fiber) + { + return fc->syscallOverrides; + } + // Persistent per-OS-thread stack: each non-fiber caller (borrowed host + // worker, direct non-fiber caller) runs its override chain to completion + // without yielding, so a per-OS-thread stack is exactly right and still + // catches genuine single-context self-recursion (the 0x83 case). + thread_local ::SyscallOverrideStack s_nonFiberOverrideState; + return s_nonFiberOverrideState; +} + +void ps2sched::enqueue_external_wakeup_validated(int tid, uint64_t token) +{ + { + std::lock_guard lk(g_sched_mutex); + FiberContext* fc = fiber_for(tid); + // Identity check by {generation, tid} token, NOT by pointer: if `tid` was + // recycled to a new fiber, that fiber has a different generation and the + // token mismatches, so the stale wakeup is dropped. No state==Blocked gate: + // WaitForNextVSyncTick has the same publish/arm window as WaitSema, so a + // tick that fires while the fiber is still Running must reach wake_locked. + if (fc != nullptr && + token != 0u && + make_fiber_token(fc) == token && + fc->suspendCount.load(std::memory_order_relaxed) == 0) + { + wake_locked(fc); + } + } + g_sched_cv.notify_all(); +} + +void ps2sched::maybe_yield() +{ + FiberContext* fc = tls_current_fiber; + if (!fc) return; // called from a host worker: no-op + bool yield = false; + { + std::lock_guard lk(g_sched_mutex); + if (g_run_queue && g_run_queue->priority < fc->priority) + { + enqueue_locked(fc); // enqueue Ready before yielding + yield = true; + } + } + if (yield) ps2fiber_yield(); +} + +void ps2sched::suspend_self() +{ + FiberContext* fc = tls_current_fiber; + if (fc == nullptr) + { + // Borrowed host worker; cannot suspend a non-fiber. No-op. + return; + } + fc->suspendCount.fetch_add(1, std::memory_order_relaxed); + // Use the SAME arm_park()/block_current() protocol as every object wait + // (WaitSema, WaitEventFlag, SleepThread, ...): a suspend CAN race a + // concurrent waker. Thread.cpp's SuspendThread increments the PS2-visible + // ThreadInfo::suspendCount and THEN calls suspend_self() as two separate + // steps (they cannot be merged -- g_sched_mutex must never be nested + // under a ThreadInfo::m, see SleepThread's "Drop info->m before ANY + // scheduler operation" comment), so a concurrent host thread's + // ResumeThread -> clear_suspend() can run in between. At that moment + // fc->state is still Running (we have not reached this function yet), so + // the resume cannot enqueue us; clear_suspend() calls wake_locked() + // unconditionally, which records the cancellation in wake_pending + // instead. arm_park() preserves wake_pending (does not clear it) and + // block_current() consumes it, returning WokenInWindow without ever + // parking. Net effect: SuspendThread behaves as suspend-immediately- + // resumed, and fc->suspendCount is left exactly at the value + // clear_suspend() set (its store(0) already ran before we got here), so + // yield_point()'s suspendCount>0 gate will not spuriously re-block us + // later on a count nobody is left to clear. + // + // If no such race occurs, arm_park()/block_current() park normally and + // resume via the matching clear_suspend() wake. + arm_park(); + block_current(); +} + +void ps2sched::suspend_other(int tid) +{ + std::lock_guard lk(g_sched_mutex); + FiberContext* fc = fiber_for(tid); + if (!fc) return; + fc->suspendCount.fetch_add(1, std::memory_order_relaxed); + if (fc->state == FiberContext::State::Ready) + { + remove_locked(fc); // pull it out of the queue now + fc->state = FiberContext::State::Blocked; + } + // If Blocked already: stays blocked (gate keeps it off-queue on wakeups). + // Cannot be Running: only one fiber runs at a time (N=1 invariant). +} + +void ps2sched::clear_suspend(int tid) +{ + bool notify = false; + { + std::lock_guard lk(g_sched_mutex); + FiberContext* fc = fiber_for(tid); + if (!fc) return; + fc->suspendCount.store(0, std::memory_order_relaxed); + // Always route through wake_locked (not gated on state==Blocked). + // See suspend_self() for the full race this closes: a clear_suspend() + // that fires before the target has transitioned to Blocked (still + // Running, mid its own ThreadInfo::suspendCount++ -> suspend_self() + // window) must still be able to record wake_pending so + // suspend_self()'s block_current() call observes it instead of + // parking with no one left to wake it. + wake_locked(fc); // race-safe enqueue / wake_pending / no-op + notify = true; + } + if (notify) g_sched_cv.notify_all(); +} + +void ps2sched::rotate_ready_queue(int priority) +{ + std::lock_guard lk(g_sched_mutex); + FiberContext** pp = &g_run_queue; + while (*pp && (*pp)->priority != priority) + pp = &(*pp)->next; + if (!*pp) return; + FiberContext* victim = *pp; // first node at this priority + *pp = victim->next; + victim->next = nullptr; + FiberContext** ins = pp; // re-insert after the last node at this priority + while (*ins && (*ins)->priority == priority) + ins = &(*ins)->next; + victim->next = *ins; + *ins = victim; +} + +void ps2sched::async_guest_begin() +{ + if (on_guest_execution_slot()) + { + std::fprintf(stderr, + "FATAL [ps2sched]: async_guest_begin from guest executor thread\n"); + std::terminate(); + } + // Publish the waiter BEFORE taking the lock so the executor's canRun gate + // (g_host_token_waiters == 0) observes it no later than its next predicate + // evaluation, then park until the token is free. + g_host_token_waiters.fetch_add(1, std::memory_order_relaxed); + std::unique_lock lk(g_sched_mutex); + g_sched_cv.wait(lk, [] + { + return g_running_fiber == nullptr && !g_guest_token_held_by_host; + }); + g_guest_token_held_by_host = true; + tls_holds_guest_token = true; + // g_currentThreadId stays -1 on this host worker thread. + g_host_token_waiters.fetch_sub(1, std::memory_order_relaxed); + // Record the grant (under g_sched_mutex) so a deferring executor knows a + // worker got its turn and may resume fibers again (see canRun). + ++g_host_token_grants; +} + +void ps2sched::async_guest_end() +{ + { + std::lock_guard lk(g_sched_mutex); + // Only the worker that acquired the token may release it. A non-owner + // reaching here means a blocking-syscall retry path called end() without + // owning the token — a bug. Refuse to clear a token we do not own. + SCHED_REQUIRE(g_guest_token_held_by_host, + "async_guest_end with token not held"); + SCHED_REQUIRE(tls_holds_guest_token, + "async_guest_end called by non-owner"); + g_guest_token_held_by_host = false; + tls_holds_guest_token = false; + } + g_sched_cv.notify_all(); // wake the executor +} + +bool ps2sched::yield_point() +{ + if ((++tls_backedge_counter & 127u) != 0u) return false; // cheap fast path + + FiberContext* fc = tls_current_fiber; + if (!fc) return false; // running under a host worker (AsyncGuestScope): no preempt + + // 1. Terminate request -> unwind THIS fiber's own stack. + if (fc->terminateRequested.load(std::memory_order_relaxed)) + { + throw ThreadExitException(); // caught in fiber_trampoline + } + + // 2. Suspend request -> block self. + if (fc->suspendCount.load(std::memory_order_relaxed) > 0) + { + block_current(); + return false; + } + + // 3. Higher-priority fiber ready -> cooperative yield (enqueue Ready first). + bool yield = false; + { + std::lock_guard lk(g_sched_mutex); + if (g_run_queue && g_run_queue->priority < fc->priority) + { + enqueue_locked(fc); + yield = true; + } + } + if (yield) ps2fiber_yield(); + + // 4. A host worker is parked in async_guest_begin() waiting for the guest + // token -> cooperatively yield so it can run. The executor-side half of + // this handoff is the g_host_token_waiters deferral in canRun; this is + // the fiber-side half: a fiber that never blocks would otherwise keep + // the executor inside ps2fiber_resume forever, and the executor gate + // alone can only act BETWEEN resumes. The executor's post-resume code + // re-enqueues us (state == Running), so no self-enqueue is needed. + if (fc && g_host_token_waiters.load(std::memory_order_relaxed) > 0) + { + ps2fiber_yield(); + // Re-check terminate after resuming: scheduler_shutdown() may have + // fired while the worker held the token. + if (fc->terminateRequested.load(std::memory_order_relaxed)) + { + throw ThreadExitException(); // caught in fiber_trampoline + } + return true; + } + return false; +} + +int ps2sched::host_token_waiters() +{ + return g_host_token_waiters.load(std::memory_order_relaxed); +} + +// True on any guest execution context: the executor thread (all backends), +// or a fiber's own OS thread on the pthread backend (set in fiber_trampoline). +// Shares on_guest_execution_slot() with the async_guest_begin() abort guard, +// so the two can never drift apart. Callers use this to decide whether they +// already own the guest execution slot or must borrow the token / avoid +// joining workers that wait on guest context. +bool ps2sched::is_guest_thread() +{ + return on_guest_execution_slot(); +} + +// Declared in ps2_fiber.h. Lets the fiber backend assert it only switches +// contexts on the registered executor thread without exposing the thread id. +bool ps2fiber_on_executor_thread() +{ + return tls_is_executor_thread; +} diff --git a/ps2xRuntime/src/lib/ps2_scheduler_internal.h b/ps2xRuntime/src/lib/ps2_scheduler_internal.h new file mode 100644 index 000000000..f3ff0c41f --- /dev/null +++ b/ps2xRuntime/src/lib/ps2_scheduler_internal.h @@ -0,0 +1,216 @@ +#ifndef PS2_SCHEDULER_INTERNAL_H +#define PS2_SCHEDULER_INTERNAL_H + +// --------------------------------------------------------------------------- +// ps2_scheduler_internal.h — private implementation types for the +// N=1 fiber cooperative scheduler. +// +// NEVER include this from a public header. Include from ps2_scheduler.cpp +// and Kernel/Syscall files ONLY. +// --------------------------------------------------------------------------- + +#include "ps2_scheduler.h" // public API (extern g_currentThreadId) +#include "ps2_fiber.h" // PS2Fiber* +#include // R5900Context, PS2Runtime +#include "ps2_dispatch_history.h" +#include "ps2_syscall_override_state.h" + +#include +#include +#include +#include +#include +#include +#include + +// --------------------------------------------------------------------------- +// Per-fiber state +// --------------------------------------------------------------------------- + +struct FiberContext +{ + int tid = 0; + int priority = 1; // lower number = higher priority (PS2 EE convention) + + // Monotonic counter assigned once at construction. Token-validated wakeups + // use {tid, generation} to reject stale wakeups after tid recycling. + uint32_t generation = 0; + + // Per-fiber saved CPU state. dispatchLoop() reads/writes this in-place. + R5900Context cpu; + + // Per-fiber dispatch-trace ring (diagnostics only). Fresh at construction, + // destroyed at fiber teardown — no cross-fiber sharing under the N=1 + // executor. + DispatchHistory dispatchHistory; + + // Per-fiber active-override stack (see ps2_syscall_override_state.h). Fresh + // per fiber, destroyed at teardown: the reentrancy guard in + // dispatchSyscallOverride is scoped to one fiber's call chain, never shared + // across fibers on the N=1 executor. + SyscallOverrideStack syscallOverrides; + + // Fiber handle — owns the stack; freed by guest_executor_main on Finished. + PS2Fiber* fiber = nullptr; + + // Scheduling state machine + enum class State + { + Fresh, // created, never resumed + Ready, // in g_run_queue + Running, // currently executing on g_guest_thread + Blocked, // parked (sema/event/sleep/suspend/vsync) + Exiting, // in fiber_trampoline exit hook; may yield, do NOT free/re-enqueue-as-Finished + Finished // fn returned / ThreadExitException caught + }; + State state = State::Fresh; + + // Intrusive singly-linked list pointer (used by the run queue). + FiberContext* next = nullptr; + + // Guards against duplicate run-queue insertion. + bool in_run_queue = false; + + // Set by a waker that fires mid-park; consumed by block_current / executor + // post-resume. See wake_locked. + bool wake_pending = false; + + // Set by request_terminate() (under g_sched_mutex). Read by yield_point on + // the fiber's own execution (relaxed — executor is the only reader while the + // fiber runs; writers wake it). + std::atomic terminateRequested{false}; + + // > 0 means the fiber must stay off the run queue (suspended). Set by + // suspend_self/suspend_other; cleared by clear_suspend. All wakeup paths gate. + std::atomic suspendCount{0}; + + // Join priority-floor: active while waiting in join_fiber. See join_fiber. + bool joinFloorActive = false; + int joinSavedPriority = 0; + + // Trampoline reads rt/rdram from here instead of smuggling them through guest registers. + PS2Runtime* rt = nullptr; + uint8_t* rdram = nullptr; + + // Safety net for create_fiber early-exit; normal paths null `fiber` first. + // ps2fiber_free is a no-op on nullptr. + ~FiberContext() + { + if (fiber != nullptr) + { + ps2fiber_free(fiber); + fiber = nullptr; + } + } + + // FiberContext owns a raw fiber; it must not be copied or moved (the map stores + // it via unique_ptr and never copies it). + FiberContext() + { + static std::atomic s_nextGeneration{1}; + generation = s_nextGeneration.fetch_add(1, std::memory_order_relaxed); + } + FiberContext(const FiberContext&) = delete; + FiberContext& operator=(const FiberContext&) = delete; +}; + +// --------------------------------------------------------------------------- +// Globals (defined in ps2_scheduler.cpp; extern-declared here) +// --------------------------------------------------------------------------- + +// The single global scheduler mutex. All queue operations, state transitions, +// and condition-variable waits are guarded by this. +extern std::mutex g_sched_mutex; +extern std::condition_variable g_sched_cv; + +// Intrusive singly-linked list; head is the highest-priority ready fiber. +// Access only under g_sched_mutex. +extern FiberContext* g_run_queue; + +// The fiber currently executing guest code on g_guest_thread (or nullptr). +// Written under g_sched_mutex before/after ps2fiber_resume(). +extern FiberContext* g_running_fiber; + +// True while a host (non-fiber) thread holds the "guest token" for running +// handler code (INTC/DMAC/alarm/RPC). Serializes with g_running_fiber. +extern bool g_guest_token_held_by_host; + +// Set to true during scheduler_shutdown to stop the executor loop. +extern std::atomic g_stop; + +// All fiber contexts, keyed by tid. +extern std::unordered_map> g_fiber_map; + +// The single guest executor thread. +extern std::thread g_guest_thread; + +// --------------------------------------------------------------------------- +// Thread-locals (defined in ps2_scheduler.cpp) +// --------------------------------------------------------------------------- + +// Pointer to the FiberContext the executor is currently running. +// NULL when not on g_guest_thread or between fibers. +extern thread_local FiberContext* tls_current_fiber; + +// Back-edge counter sampled by yield_point(). +extern thread_local uint32_t tls_backedge_counter; + +// --------------------------------------------------------------------------- +// Intrusive priority-sorted queue helpers (callers must hold g_sched_mutex) +// --------------------------------------------------------------------------- + +// Insert fc into the run queue in priority order (stable FIFO within same priority). +inline void enqueue_locked(FiberContext* fc) +{ + if (fc->in_run_queue) return; + fc->in_run_queue = true; + fc->state = FiberContext::State::Ready; + FiberContext** pp = &g_run_queue; + // Lower priority number = higher priority. Stable: insert after existing same-prio nodes. + while (*pp && (*pp)->priority <= fc->priority) + pp = &(*pp)->next; + fc->next = *pp; + *pp = fc; +} + +// Remove fc from the run queue. +inline void remove_locked(FiberContext* fc) +{ + if (!fc->in_run_queue) return; + FiberContext** pp = &g_run_queue; + while (*pp && *pp != fc) + pp = &(*pp)->next; + if (*pp) + { + *pp = fc->next; + fc->next = nullptr; + fc->in_run_queue = false; + } +} + +// Pop and return the head of the run queue (highest priority), or nullptr. +inline FiberContext* pop_head_locked() +{ + FiberContext* fc = g_run_queue; + if (!fc) return nullptr; + g_run_queue = fc->next; + fc->next = nullptr; + fc->in_run_queue = false; + return fc; +} + +// Return the FiberContext for tid, or nullptr if not found. +// Caller should hold g_sched_mutex or ensure no concurrent modifications. +inline FiberContext* fiber_for(int tid) +{ + auto it = g_fiber_map.find(tid); + return (it != g_fiber_map.end()) ? it->second.get() : nullptr; +} + +// --------------------------------------------------------------------------- +// Fiber exit hook — called by fiber_trampoline after dispatchLoop returns. +// Set once by Thread.cpp during static initialization. +// --------------------------------------------------------------------------- +extern void (*g_fiber_exit_hook)(int tid, uint8_t* rdram, R5900Context* ctx, PS2Runtime* rt); + +#endif // PS2_SCHEDULER_INTERNAL_H diff --git a/ps2xTest/CMakeLists.txt b/ps2xTest/CMakeLists.txt index 5ab39217e..b6da61051 100644 --- a/ps2xTest/CMakeLists.txt +++ b/ps2xTest/CMakeLists.txt @@ -28,6 +28,7 @@ add_library(ps2_test_lib STATIC src/ps2_sif_dma_tests.cpp src/ps2_recompiler_tests.cpp src/ps2_runtime_expansion_tests.cpp + src/ps2_scheduler_workload_regression_tests.cpp ) target_include_directories(ps2_test_lib PRIVATE @@ -35,6 +36,7 @@ target_include_directories(ps2_test_lib PRIVATE ${CMAKE_SOURCE_DIR}/ps2xRecomp/include ${CMAKE_SOURCE_DIR}/ps2xAnalyzer/include ${CMAKE_SOURCE_DIR}/ps2xRuntime/include + ${CMAKE_SOURCE_DIR}/ps2xRuntime/src/lib ) target_link_libraries(ps2_test_lib PRIVATE diff --git a/ps2xTest/include/SchedTestSupport.h b/ps2xTest/include/SchedTestSupport.h new file mode 100644 index 000000000..a5f8acbc3 --- /dev/null +++ b/ps2xTest/include/SchedTestSupport.h @@ -0,0 +1,75 @@ +// --------------------------------------------------------------------------- +// Shared scaffolding for scheduler-shaped tests (ps2_runtime_expansion_tests.cpp +// and ps2_scheduler_workload_regression_tests.cpp): the RAII fixture that owns the +// runtime + guest RAM for a single test, and the register-access / wait-poll +// helpers every scheduler test needs regardless of which file it lives in. +// --------------------------------------------------------------------------- +#pragma once + +#include "ps2_runtime.h" +#include "ps2_scheduler.h" +#include "ps2_syscalls.h" +#include "runtime/ps2_memory.h" + +#include +#include +#include +#include + +namespace ps2x_test +{ + // Binds a MIPS-ABI argument register to a 32-bit value. + inline void setRegU32(R5900Context &ctx, int reg, uint32_t value) + { + ctx.r[reg] = _mm_set_epi64x(0, static_cast(value)); + } + + // Reads back a return-value register as a signed 32-bit result. + inline int32_t getRegS32(const R5900Context &ctx, int reg) + { + return static_cast(::getRegU32(&ctx, reg)); + } + + // Polls `pred` at 1ms intervals until it is true or `timeout` elapses, + // always giving `pred` one final check at the deadline. + template + bool waitUntil(Predicate pred, std::chrono::milliseconds timeout) + { + const auto deadline = std::chrono::steady_clock::now() + timeout; + while (std::chrono::steady_clock::now() < deadline) + { + if (pred()) + { + return true; + } + std::this_thread::sleep_for(std::chrono::milliseconds(1)); + } + return pred(); + } + + // Owns the runtime + guest RAM a scheduler test dispatches fibers against. + // Construction clears residual thread/sema/handler state from a previous + // test (notifyRuntimeStop) and brings up a fresh scheduler epoch + // (scheduler_init, which also heals any g_activeThreads drift left by a + // prior epoch's shutdown races); destruction tears the scheduler down and + // requests the runtime stop, in that order, so a test body only needs to + // add its own test-specific drains (signaling semas it created, joining + // threads it spawned) before falling out of scope. + struct SchedFixture + { + PS2Runtime runtime; + std::vector rdram = std::vector(PS2_RAM_SIZE, 0u); + + SchedFixture() + { + ps2_syscalls::notifyRuntimeStop(); + ps2sched::scheduler_init(); + } + + ~SchedFixture() + { + ps2sched::scheduler_shutdown(); + runtime.requestStop(); + } + }; +} diff --git a/ps2xTest/src/main.cpp b/ps2xTest/src/main.cpp index 0ed5e9b76..ec6434bde 100644 --- a/ps2xTest/src/main.cpp +++ b/ps2xTest/src/main.cpp @@ -15,7 +15,37 @@ void register_ps2_sif_rpc_tests(); void register_ps2_sif_dma_tests(); void register_ps2_recompiler_tests(); void register_ps2_runtime_expansion_tests(); +void register_scheduler_tests(); +void register_scheduler_protocol_tests(); +void register_scheduler_race_tests(); +void register_scheduler_stress_tests(); +void register_scheduler_vsync_priority_tests(); +void register_scheduler_lifecycle_tests(); +void register_scheduler_borrowed_worker_tests(); +void register_scheduler_window_tests(); +void register_scheduler_sleep_resume_tests(); +void register_scheduler_shutdown_clean_tests(); +void register_scheduler_borrowed_guard_tests(); +void register_scheduler_sema_delete_tests(); +void register_scheduler_tid_reuse_tests(); +void register_scheduler_evf_mode_tests(); +void register_scheduler_shutdown_fiber_tests(); +void register_scheduler_join_host_tests(); +void register_scheduler_fiber_alloc_tests(); +void register_scheduler_reinit_tests(); +void register_scheduler_join_priority_tests(); +void register_scheduler_park_window_tests(); +void register_scheduler_fiber_ptr_tests(); void reset_ps2_test_function_table(); +void register_scheduler_token_handoff_tests(); +void register_scheduler_rpc_loop_park_tests(); +void register_scheduler_guest_context_stop_tests(); +void register_scheduler_recovery_isolation_tests(); +void register_runtime_async_stack_pool_tests(); +void register_scheduler_dmac_guest_dispatch_tests(); +void register_scheduler_stack_isolation_tests(); +void register_scheduler_override_isolation_tests(); +void register_scheduler_join_starvation_tests(); int main() { @@ -34,6 +64,36 @@ int main() register_ps2_sif_dma_tests(); register_ps2_recompiler_tests(); register_ps2_runtime_expansion_tests(); + register_scheduler_tests(); + register_scheduler_protocol_tests(); + register_scheduler_race_tests(); + register_scheduler_stress_tests(); + register_scheduler_vsync_priority_tests(); + register_scheduler_lifecycle_tests(); + register_scheduler_borrowed_worker_tests(); + register_scheduler_window_tests(); + register_scheduler_sleep_resume_tests(); + register_scheduler_shutdown_clean_tests(); + register_scheduler_borrowed_guard_tests(); + register_scheduler_sema_delete_tests(); + register_scheduler_tid_reuse_tests(); + register_scheduler_evf_mode_tests(); + register_scheduler_shutdown_fiber_tests(); + register_scheduler_join_host_tests(); + register_scheduler_fiber_alloc_tests(); + register_scheduler_reinit_tests(); + register_scheduler_join_priority_tests(); + register_scheduler_park_window_tests(); + register_scheduler_fiber_ptr_tests(); + register_scheduler_token_handoff_tests(); + register_scheduler_rpc_loop_park_tests(); + register_scheduler_guest_context_stop_tests(); + register_scheduler_recovery_isolation_tests(); + register_runtime_async_stack_pool_tests(); + register_scheduler_dmac_guest_dispatch_tests(); + register_scheduler_stack_isolation_tests(); + register_scheduler_override_isolation_tests(); + register_scheduler_join_starvation_tests(); int res = MiniTest::Run(); std::cout.flush(); std::cerr.flush(); diff --git a/ps2xTest/src/ps2_gs_tests.cpp b/ps2xTest/src/ps2_gs_tests.cpp index d35cf7b10..771e374e3 100644 --- a/ps2xTest/src/ps2_gs_tests.cpp +++ b/ps2xTest/src/ps2_gs_tests.cpp @@ -3275,15 +3275,69 @@ void register_ps2_gs_tests() setRegU32(resetCtx, 5, 1u); setRegU32(resetCtx, 6, 2u); setRegU32(resetCtx, 7, 1u); + // sceGsResetGraph(mode=0) is what captures the field base (see + // GS.cpp resetGsSyncVState / g_gs_sync_v_base_tick). Bracket it + // with tick samples so we know whether the vsync worker (a + // free-running wall-clock timer, independent of this thread) ticked + // over while this call was doing its own work. On a loaded/virtualized + // CI runner that call can itself take long enough for a tick to land + // inside it, which shifts the base and makes any hardcoded absolute + // field expectation unanswerable -- only the relative parity checks + // below remain valid in that case. + const uint64_t tickPreSetup = ps2_syscalls::GetCurrentVSyncTick(); ps2_stubs::sceGsResetGraph(rdram.data(), &resetCtx, &runtime); + const uint64_t tickPostSetup = ps2_syscalls::GetCurrentVSyncTick(); + const bool baseUnambiguous = (tickPreSetup == tickPostSetup); R5900Context sync0{}; ps2_stubs::sceGsSyncV(rdram.data(), &sync0, &runtime); - t.Equals(static_cast(getRegU32Test(sync0, 2)), 0, "first interlaced sceGsSyncV should report even field"); + // Sample the vsync tick this call consumed as close to the call as + // possible. The vsync worker (ps2xRuntime/.../Interrupt.cpp, + // interruptWorkerMain) free-runs on a wall-clock timer + // (kVblankPeriod, ~16.6ms) independent of this thread, and + // WaitForNextVSyncTick's contract is only "returns a tick strictly + // after the one you started on" -- not "exactly one tick later". + // On a loaded/virtualized CI runner (observed on windows-msvc) THIS + // test thread -- not the worker -- can be descheduled for longer + // than one vsync period between the two sceGsSyncV calls below, in + // which case the second call legitimately consumes tick N+2 (or + // later) instead of N+1, and a hardcoded "must be odd" expectation + // is simply wrong. Deriving the expected field from the observed + // tick keeps this assertion correct under that jitter while still + // catching real regressions in the field-parity logic. + const uint64_t tick0 = ps2_syscalls::GetCurrentVSyncTick(); + const int32_t field0 = static_cast(getRegU32Test(sync0, 2)); + if (baseUnambiguous) + { + t.Equals(field0, 0, "first interlaced sceGsSyncV should report even field"); + } R5900Context sync1{}; ps2_stubs::sceGsSyncV(rdram.data(), &sync1, &runtime); - t.Equals(static_cast(getRegU32Test(sync1, 2)), 1, "second interlaced sceGsSyncV should report odd field"); + const uint64_t tick1 = ps2_syscalls::GetCurrentVSyncTick(); + const int32_t field1 = static_cast(getRegU32Test(sync1, 2)); + + // Liveness: the second call must have actually waited for a new + // tick, not replayed the one the first call already consumed. + t.IsTrue(tick1 > tick0, + "second sceGsSyncV should consume a strictly later vsync tick than the first"); + + // getGsSyncVFieldForTick's field is (tick - base - 1) & 1, which is + // linear in the tick number, so the expected field for tick1 is + // field0 XOR'd with the parity of however many ticks actually + // elapsed. This holds exactly whether one tick elapsed (the common, + // fully-loaded-CI-free case) or several (a stalled runner). + const int32_t expectedField1 = field0 ^ static_cast((tick1 - tick0) & 1u); + t.Equals(field1, expectedField1, + "second interlaced sceGsSyncV field should match the parity of the tick it actually consumed"); + + // Keep full strength under normal timing: when the base was + // unambiguous (so field0 == 0 is known-good) and exactly one tick + // elapsed between the two syncs, the field must flip to odd. + if (baseUnambiguous && tick1 - tick0 == 1u) + { + t.Equals(field1, 1, "second interlaced sceGsSyncV should report odd field when exactly one tick elapsed"); + } R5900Context resetProgCtx{}; setRegU32(resetProgCtx, 4, 0u); diff --git a/ps2xTest/src/ps2_runtime_expansion_tests.cpp b/ps2xTest/src/ps2_runtime_expansion_tests.cpp index 20fb73d19..d446e6c4e 100644 --- a/ps2xTest/src/ps2_runtime_expansion_tests.cpp +++ b/ps2xTest/src/ps2_runtime_expansion_tests.cpp @@ -1,4 +1,5 @@ #include "MiniTest.h" +#include "SchedTestSupport.h" #include "ps2recomp/code_generator.h" #include "ps2recomp/instructions.h" #include "ps2recomp/r5900_decoder.h" @@ -15,18 +16,26 @@ #include "Stubs/Audio.h" #include "Stubs/GS.h" #include "Stubs/VU.h" +#include "Kernel/Syscalls/Interrupt.h" // interrupt_state::g_vsync_waitList, WaitForNextVSyncTick, EnsureVSyncWorkerRunning, stopInterruptWorker #include #include #include +#include #include #include +#include +#include #include #include +#include +#include #include +#include "Kernel/Syscalls/Helpers/State.h" // g_threads / g_thread_map_mutex using namespace ps2recomp; using namespace ps2_syscalls; +using namespace ps2x_test; namespace { @@ -37,18 +46,6 @@ namespace constexpr uint32_t EXCEPTION_VECTOR_GENERAL = 0x80000080u; constexpr uint32_t EXCEPTION_VECTOR_BOOT = 0xBFC00200u; - constexpr int KE_OK = 0; - - void setRegU32(R5900Context &ctx, int reg, uint32_t value) - { - ctx.r[reg] = _mm_set_epi64x(0, static_cast(value)); - } - - int32_t getRegS32(const R5900Context &ctx, int reg) - { - return static_cast(::getRegU32(&ctx, reg)); - } - uint32_t makeVifCmd(uint8_t opcode, uint8_t num, uint16_t imm) { return (static_cast(opcode) << 24) | @@ -99,26 +96,17 @@ namespace return generated.find(needle) != std::string::npos; } - template - bool waitUntil(Predicate pred, std::chrono::milliseconds timeout) - { - const auto deadline = std::chrono::steady_clock::now() + timeout; - while (std::chrono::steady_clock::now() < deadline) - { - if (pred()) - { - return true; - } - std::this_thread::sleep_for(std::chrono::milliseconds(1)); - } - return pred(); - } - uint32_t frameOffsetBytes(uint32_t x, uint32_t y, uint32_t fbw) { return GSPSMCT32::addrPSMCT32(0u, (fbw != 0u) ? fbw : 1u, x, y); } + // Bumped on every genuine entry into testRuntimeWorkerLoop(). g_activeThreads + // is incremented eagerly by StartThread (before the fiber ever gets a CPU + // time slice) and force-zeroed by notifyRuntimeStop(), so it cannot be used + // to prove the worker body actually ran -- this counter is the real signal. + std::atomic gRuntimeWorkerLoopRuns{0}; + void testRuntimeWorkerLoop(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) { if (!ctx || !runtime) @@ -129,6 +117,7 @@ namespace // Keep touching guest memory so teardown races are easier to catch. (void)Ps2FastRead64(rdram, static_cast(0x01FFFFF8u + (ctx->insn_count & 0x7u))); ++ctx->insn_count; + gRuntimeWorkerLoopRuns.fetch_add(1u, std::memory_order_release); if (runtime->isStopRequested()) { @@ -139,11 +128,16 @@ namespace std::this_thread::sleep_for(std::chrono::milliseconds(1)); } + // Counter shared between the INTC handler (vsync worker thread), the busy + // dispatch loop (worker std::thread), and the test thread. It must be + // atomic: it is a write/read pair shared across threads. + std::atomic gAsyncCounter{0u}; + + // Used by "guest execution is serialized per runtime" below to prove the + // N=1 fiber scheduler never runs guest code from more than one fiber at a + // time. std::atomic gSerializedGuestActive{0}; std::atomic gSerializedGuestMaxActive{0}; - std::atomic gPreemptionPolicyEntryCount{0}; - std::atomic gPreemptionPolicyAllowFirstProbe{false}; - std::atomic gPreemptionPolicyPeerRan{false}; void testSerializedGuestStep(uint8_t *, R5900Context *ctx, PS2Runtime *) { @@ -166,39 +160,6 @@ namespace } } - void testPreemptionPolicyStep(uint8_t *, R5900Context *ctx, PS2Runtime *runtime) - { - if (!ctx || !runtime) - { - return; - } - - const int32_t entryIndex = gPreemptionPolicyEntryCount.fetch_add(1, std::memory_order_acq_rel) + 1; - if (entryIndex == 1) - { - while (!gPreemptionPolicyAllowFirstProbe.load(std::memory_order_acquire)) - { - std::this_thread::yield(); - } - - bool shouldPreempt = false; - for (int attempt = 0; attempt < 256 && - !shouldPreempt; - ++attempt) - { - shouldPreempt = runtime->shouldPreemptGuestExecution(); - } - setRegU32(*ctx, 2, shouldPreempt ? 1u : 0u); - } - else - { - gPreemptionPolicyPeerRan.store(true, std::memory_order_release); - setRegU32(*ctx, 2, 2u); - } - - ctx->pc = 0u; - } - void testResumeOwnerFallbackHandler(uint8_t *, R5900Context *ctx, PS2Runtime *) { if (ctx) @@ -245,15 +206,10 @@ namespace return; } - uint32_t counter = 0u; - do + while (gAsyncCounter.load(std::memory_order_acquire) == 0u) { - std::memcpy(&counter, rdram + kAsyncCounterAddr, sizeof(counter)); - if (counter == 0u) - { - std::this_thread::sleep_for(std::chrono::milliseconds(1)); - } - } while (counter == 0u); + std::this_thread::sleep_for(std::chrono::milliseconds(1)); + } ctx->pc = 0u; } @@ -262,8 +218,7 @@ namespace { if (rdram) { - const uint32_t counter = 1u; - std::memcpy(rdram + kAsyncCounterAddr, &counter, sizeof(counter)); + gAsyncCounter.store(1u, std::memory_order_release); } if (ctx) @@ -320,6 +275,162 @@ namespace } +// --------------------------------------------------------------------------- +// Scheduler test helpers and per-test state +// --------------------------------------------------------------------------- + +namespace +{ + // ----------------------------------------------------------------------- + // Shared state for scheduler step functions (indexed by test slot 0..2) + // ----------------------------------------------------------------------- + constexpr uint32_t kSchedTestSemaIdAddr = 0x3000u; // rdram offset: holds sid (int32) + constexpr uint32_t kSchedDoneSemaIdAddr = 0x3004u; // rdram offset: holds done_sid (int32) + constexpr uint32_t kSchedLogBase = 0x3010u; // rdram offset: int32[4] log slots + constexpr uint32_t kSchedResultBase = 0x3020u; // rdram offset: int32[4] result slots + std::atomic gSchedSeqCounter{0}; + + // Worker step function: WaitSema(sid from rdram), record result and sequence, signal done_sid. + // In the fiber model this function IS the fiber body — it runs on a pool thread and + // blocks cooperatively via WaitSema (which calls block_current() internally). + void schedWorkerWaitAndSignal(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + if (!rdram || !ctx) + { + if (ctx) ctx->pc = 0u; + return; + } + + // Read sema id from rdram + int32_t sid = 0; + std::memcpy(&sid, rdram + kSchedTestSemaIdAddr, sizeof(sid)); + int32_t doneSid = 0; + std::memcpy(&doneSid, rdram + kSchedDoneSemaIdAddr, sizeof(doneSid)); + + // WaitSema(sid) — blocks cooperatively if count==0 + R5900Context wCtx{}; + setRegU32(wCtx, 4, static_cast(sid)); + ps2_syscalls::WaitSema(rdram, &wCtx, runtime); + const int32_t waitResult = getRegS32(wCtx, 2); + + // Record: sequence index → result. Only one fiber runs at a time (N=1 + // cooperative scheduler), so a plain relaxed load is enough to pick the + // slot; the payload writes below must land *before* the release bump so + // that a host thread which later acquire-loads the incremented counter + // (see the DeleteSema test's waitUntil) is guaranteed to see them. + const int32_t seq = gSchedSeqCounter.load(std::memory_order_relaxed); + if (seq >= 0 && seq < 4) + { + std::memcpy(rdram + kSchedResultBase + static_cast(seq * 4), &waitResult, sizeof(waitResult)); + const int32_t tid = g_currentThreadId; + std::memcpy(rdram + kSchedLogBase + static_cast(seq * 4), &tid, sizeof(tid)); + } + gSchedSeqCounter.fetch_add(1, std::memory_order_release); + + // Signal done_sid to wake the main test fiber + if (doneSid > 0) + { + R5900Context sCtx{}; + setRegU32(sCtx, 4, static_cast(doneSid)); + ps2_syscalls::SignalSema(rdram, &sCtx, runtime); + } + + ctx->pc = 0u; + } + + // Worker step function: just WaitSema(sid) indefinitely (for TerminateThread test). + void schedWorkerBlockForever(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + if (!rdram || !ctx) + { + if (ctx) ctx->pc = 0u; + return; + } + + int32_t sid = 0; + std::memcpy(&sid, rdram + kSchedTestSemaIdAddr, sizeof(sid)); + + R5900Context wCtx{}; + setRegU32(wCtx, 4, static_cast(sid)); + ps2_syscalls::WaitSema(rdram, &wCtx, runtime); + + ctx->pc = 0u; + } + + // Helper: create a semaphore via CreateSema (EE layout: count, max_count, init_count). + // Returns the sema id or -1 on failure. + int32_t createSchedSema(uint8_t *rdram, PS2Runtime *runtime, int initCount, int maxCount) + { + constexpr uint32_t kSemaParamAddr = 0x2F00u; + const uint32_t params[6] = { + 0u, // count (unused by EE layout) + static_cast(maxCount), // max_count + static_cast(initCount),// init_count + 0u, // wait_threads + 0u, // attr + 0u, // option + }; + std::memcpy(rdram + kSemaParamAddr, params, sizeof(params)); + + R5900Context cCtx{}; + setRegU32(cCtx, 4, kSemaParamAddr); + ps2_syscalls::CreateSema(rdram, &cCtx, runtime); + return getRegS32(cCtx, 2); + } + + // Helper: delete a semaphore. + void deleteSchedSema(uint8_t *rdram, PS2Runtime *runtime, int32_t sid) + { + if (sid <= 0) + { + return; + } + R5900Context dCtx{}; + setRegU32(dCtx, 4, static_cast(sid)); + ps2_syscalls::DeleteSema(rdram, &dCtx, runtime); + } + + // Helper: create+start a worker thread and return its tid (or -1 on error). + // In the fiber model, StartThread enqueues a fiber; the pool threads run it. + int32_t startSchedWorker(uint8_t *rdram, PS2Runtime *runtime, + uint32_t entryAddr, int priority, + uint32_t stackAddr, uint32_t stackSize, + uint32_t arg = 0u) + { + constexpr uint32_t kThreadParamAddr = 0x2E00u; + const uint32_t threadParam[7] = { + 0u, // attr + entryAddr, // entry + stackAddr, // stack + stackSize, // stack size + 0u, // gp (0 = caller's gp) + static_cast(priority), + 0u, // option + }; + std::memcpy(rdram + kThreadParamAddr, threadParam, sizeof(threadParam)); + + R5900Context createCtx{}; + setRegU32(createCtx, 4, kThreadParamAddr); + ps2_syscalls::CreateThread(rdram, &createCtx, runtime); + const int32_t tid = getRegS32(createCtx, 2); + if (tid <= 0) + { + return -1; + } + + R5900Context startCtx{}; + setRegU32(startCtx, 4, static_cast(tid)); + setRegU32(startCtx, 5, arg); + ps2_syscalls::StartThread(rdram, &startCtx, runtime); + if (getRegS32(startCtx, 2) != 0) + { + return -1; + } + return tid; + } + +} // anonymous namespace + void register_ps2_runtime_expansion_tests() { MiniTest::Case("PS2RuntimeExpansion", [](TestCase &tc) @@ -356,8 +467,12 @@ void register_ps2_runtime_expansion_tests() tc.Run("guest execution is serialized per runtime", [](TestCase &t) { - PS2Runtime runtime; - std::vector rdram(PS2_RAM_SIZE, 0u); + // Prove the N=1 cooperative fiber scheduler never executes guest + // code from more than one fiber at a time: start several fibers via + // CreateThread/StartThread and assert max concurrency stays at 1. + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; gSerializedGuestActive.store(0, std::memory_order_release); gSerializedGuestMaxActive.store(0, std::memory_order_release); @@ -368,131 +483,29 @@ void register_ps2_runtime_expansion_tests() 0x150000u, }; constexpr size_t kEntryCount = sizeof(kEntries) / sizeof(kEntries[0]); - R5900Context contexts[kEntryCount]{}; - std::vector workers; - workers.reserve(kEntryCount); for (size_t i = 0; i < kEntryCount; ++i) { runtime.registerFunction(kEntries[i], &testSerializedGuestStep); - contexts[i].pc = kEntries[i]; } for (size_t i = 0; i < kEntryCount; ++i) { - workers.emplace_back([&, i]() - { - runtime.dispatchLoop(rdram.data(), &contexts[i]); - }); + const int32_t tid = startSchedWorker(rdram.data(), &runtime, kEntries[i], + 10, 0x00300000u + static_cast(i) * 0x2000u, 0x2000u); + t.IsTrue(tid > 0, "serialized-guest worker should start"); } - for (std::thread &worker : workers) + const bool allDone = waitUntil([&]() { - if (worker.joinable()) - { - worker.join(); - } - } + return g_activeThreads.load(std::memory_order_relaxed) == 0; + }, std::chrono::milliseconds(2000)); + t.IsTrue(allDone, "all serialized-guest workers should finish"); t.Equals(gSerializedGuestActive.load(std::memory_order_acquire), 0, "serialized guest dispatch should leave no active workers"); t.Equals(gSerializedGuestMaxActive.load(std::memory_order_acquire), 1, - "dispatchLoop should not execute guest code concurrently on one runtime"); - }); - - tc.Run("wake handoff lets a contending guest thread acquire before returning", [](TestCase &t) - { - PS2Runtime runtime; - std::atomic peerRan{false}; - std::thread peer; - bool peerWaiting = false; - bool peerRanWhileMainHeld = false; - bool peerRanAfterHandoff = false; - - { - PS2Runtime::GuestExecutionScope mainScope(&runtime); - peer = std::thread([&]() - { - PS2Runtime::GuestExecutionScope peerScope(&runtime); - peerRan.store(true, std::memory_order_release); - }); - - peerWaiting = waitUntil([&]() - { - return runtime.guestExecutionWaiterCountForTesting() > 0u; - }, std::chrono::milliseconds(100)); - - peerRanWhileMainHeld = peerRan.load(std::memory_order_acquire); - runtime.yieldGuestExecutionAfterWake(); - peerRanAfterHandoff = peerRan.load(std::memory_order_acquire); - } - - if (peer.joinable()) - { - peer.join(); - } - - t.IsTrue(peerWaiting, "peer guest thread should contend while the waker owns guest execution"); - t.IsFalse(peerRanWhileMainHeld, "peer guest thread should not run before the waker yields execution"); - t.IsTrue(peerRanAfterHandoff, "wake handoff should let the peer acquire guest execution before returning"); - }); - - tc.Run("guest preemption policy requests a dispatcher handoff when another guest thread contends", [](TestCase &t) - { - PS2Runtime runtime; - std::vector rdram(PS2_RAM_SIZE, 0u); - constexpr uint32_t kFirstEntry = 0x190000u; - constexpr uint32_t kSecondEntry = 0x1A0000u; - - gPreemptionPolicyEntryCount.store(0, std::memory_order_release); - gPreemptionPolicyAllowFirstProbe.store(false, std::memory_order_release); - gPreemptionPolicyPeerRan.store(false, std::memory_order_release); - - runtime.registerFunction(kFirstEntry, &testPreemptionPolicyStep); - runtime.registerFunction(kSecondEntry, &testPreemptionPolicyStep); - - R5900Context firstCtx{}; - R5900Context secondCtx{}; - firstCtx.pc = kFirstEntry; - secondCtx.pc = kSecondEntry; - - std::thread firstWorker([&]() - { - runtime.dispatchLoop(rdram.data(), &firstCtx); - }); - - const bool firstEntered = waitUntil([&]() - { - return gPreemptionPolicyEntryCount.load(std::memory_order_acquire) >= 1; - }, std::chrono::milliseconds(100)); - - std::thread secondWorker([&]() - { - runtime.dispatchLoop(rdram.data(), &secondCtx); - }); - - const bool secondContending = waitUntil([&]() - { - return runtime.guestExecutionWaiterCountForTesting() > 0u; - }, std::chrono::milliseconds(100)); - - gPreemptionPolicyAllowFirstProbe.store(true, std::memory_order_release); - - if (firstWorker.joinable()) - { - firstWorker.join(); - } - if (secondWorker.joinable()) - { - secondWorker.join(); - } - - t.IsTrue(firstEntered, "first guest worker should enter before probing for preemption"); - t.IsTrue(secondContending, "second guest worker should contend for guest execution before the first returns"); - t.IsTrue(gPreemptionPolicyPeerRan.load(std::memory_order_acquire), - "second guest worker should run after the first returns to the dispatcher"); - t.Equals(getRegU32(&firstCtx, 2), 1u, - "first guest worker should observe that the runtime requested preemption under contention"); + "the N=1 fiber scheduler must never execute guest code concurrently on one runtime"); }); tc.Run("lookupFunction rejects internal resume PCs without exact registration", [](TestCase &t) @@ -616,6 +629,7 @@ void register_ps2_runtime_expansion_tests() runtime.registerFunction(kBusyEntry, &testWaitForAsyncCounter); runtime.registerFunction(kIntcHandlerEntry, &testSignalAsyncCounter); + gAsyncCounter.store(0u, std::memory_order_relaxed); R5900Context addCtx{}; setRegU32(addCtx, 4, 2u); @@ -652,8 +666,11 @@ void register_ps2_runtime_expansion_tests() if (!finished) { - const uint32_t counter = 1u; - std::memcpy(rdram.data() + kAsyncCounterAddr, &counter, sizeof(counter)); + // Do NOT self-heal with the expected value. Write a distinct sentinel so + // the counter==1 assertion below fails loudly when the handler never fired. + // We still release the spinning worker (it loops until counter != 0) so the + // binary does not hang on a slow/flaky machine. + gAsyncCounter.store(999u, std::memory_order_release); } if (worker.joinable()) @@ -662,10 +679,8 @@ void register_ps2_runtime_expansion_tests() } runtime.requestStop(); - notifyRuntimeStop(); - uint32_t counter = 0u; - std::memcpy(&counter, rdram.data() + kAsyncCounterAddr, sizeof(counter)); + const uint32_t counter = gAsyncCounter.load(std::memory_order_acquire); t.IsFalse(workerThrew.load(std::memory_order_acquire), "busy dispatch worker should not throw while VBlank handlers fire"); @@ -683,9 +698,13 @@ void register_ps2_runtime_expansion_tests() constexpr uint32_t kCallbackEntry = 0x180000u; constexpr uint32_t kCallerGp = 0x0036A7F0u; constexpr uint32_t kCallerSp = 0x00123450u; - constexpr uint32_t kAsyncStackFloor = 0x01F00000u; + // Kernel-area callback stack pool: see kAsyncCallbackStackFloor / + // kAsyncCallbackStackTop in ps2_runtime.h. Callback stacks must + // never sit at top-of-RAM (the guest's own main stack). + constexpr uint32_t kAsyncStackFloor = 0x00080000u; + constexpr uint32_t kAsyncStackTop = 0x00100000u; - runtime.configureGuestHeap(kAsyncStackFloor, kAsyncStackFloor); + runtime.configureGuestHeap(0x01F00000u, 0x01F00000u); runtime.registerFunction(kCallbackEntry, &testRecordAsyncCallbackStack); ps2_stubs::resetGsSyncVCallbackState(); gAsyncCallbackObservedSp.store(0u, std::memory_order_release); @@ -706,11 +725,11 @@ void register_ps2_runtime_expansion_tests() t.IsTrue(observedSp != 0u, "callback should execute"); t.Equals(observedGp, kCallerGp, "callback should preserve the registered GP"); t.IsTrue(observedSp != kCallerSp, "callback should not reuse the registering thread stack"); - t.IsTrue(observedSp >= kAsyncStackFloor, - "callback should switch to the reserved async stack pool"); + t.IsTrue(observedSp >= kAsyncStackFloor && observedSp < kAsyncStackTop, + "callback should switch to the reserved async stack pool " + "(kernel area, below the ELF base, never top-of-RAM)"); runtime.requestStop(); - notifyRuntimeStop(); }); tc.Run("MPEG init and callback stubs return success instead of TODO errors", [](TestCase &t) @@ -1566,7 +1585,15 @@ void register_ps2_runtime_expansion_tests() tc.Run("notifyRuntimeStop joins guest worker threads before teardown", [](TestCase &t) { + // StartThread only enqueues a fiber; nothing executes its body unless + // the fiber scheduler's executor thread is running, so this test must + // bracket itself with scheduler_init()/scheduler_shutdown() like the + // Scheduler* suites do. Without this, testRuntimeWorkerLoop() below + // never runs and both assertions below pass vacuously (g_activeThreads + // is incremented eagerly by StartThread itself and force-zeroed by + // notifyRuntimeStop(), regardless of whether the worker ever executed). notifyRuntimeStop(); + ps2sched::scheduler_init(); PS2Runtime runtime; std::vector rdram(PS2_RAM_SIZE, 0u); @@ -1599,17 +1626,37 @@ void register_ps2_runtime_expansion_tests() const bool started = waitUntil([&]() { - return g_activeThreads.load(std::memory_order_relaxed) > 0; + return g_activeThreads.load(std::memory_order_acquire) > 0; }, std::chrono::milliseconds(500)); t.IsTrue(started, "worker thread should become active"); + // g_activeThreads > 0 only proves StartThread's eager bookkeeping ran, + // not that the executor has actually dispatched the fiber. Wait for a + // genuine entry into testRuntimeWorkerLoop() before tearing down, so + // "joins guest worker threads before teardown" exercises a worker that + // is actually running (not one that was merely enqueued and never + // given a time slice before requestStop() force-zeroes the counter + // this test would otherwise be trivially satisfied by). + const bool actuallyRan = waitUntil([&]() + { + return gRuntimeWorkerLoopRuns.load(std::memory_order_acquire) > 0u; + }, std::chrono::milliseconds(2000)); + t.IsTrue(actuallyRan, "worker thread body should actually execute before teardown"); + runtime.requestStop(); + // 5000ms: draining requires the worker to actually be scheduled again + // after requestStop() (it re-checks isStopRequested() once per ~1ms + // sleep iteration), so this is bounded by scheduler round-trip latency, + // not by anything under test. Generous headroom avoids false failures + // when the host machine is under heavy load (e.g. running this whole + // suite back-to-back many times in a stress loop). const bool drained = waitUntil([&]() { - return g_activeThreads.load(std::memory_order_relaxed) == 0; - }, std::chrono::milliseconds(2000)); + return g_activeThreads.load(std::memory_order_acquire) == 0; + }, std::chrono::milliseconds(5000)); t.IsTrue(drained, "requestStop should drain all guest worker threads"); + ps2sched::scheduler_shutdown(); notifyRuntimeStop(); }); @@ -1713,7 +1760,6 @@ void register_ps2_runtime_expansion_tests() t.IsTrue(finalCount >= 0 && finalCount <= 1, "semaphore count should remain within [0, max_count]"); runtime.requestStop(); - notifyRuntimeStop(); }); tc.Run("WaitEventFlag AND-mode is stable under concurrent setters", [](TestCase &t) @@ -1827,7 +1873,6 @@ void register_ps2_runtime_expansion_tests() setRegU32(deleteCtx, 4, static_cast(eid)); DeleteEventFlag(rdram.data(), &deleteCtx, &runtime); runtime.requestStop(); - notifyRuntimeStop(); }); tc.Run("sceVu0ApplyMatrix uses libvux matrix math with the imported EE ABI", [](TestCase &t) @@ -1979,3 +2024,6262 @@ void register_ps2_runtime_expansion_tests() }); }); } + +// --------------------------------------------------------------------------- +// Scheduler tests +// +// End-to-end blackbox tests: they run real guest fibers on the executor thread +// and verify outcomes through public syscall calls and atomic counters. +// scheduler_init() and scheduler_shutdown() bracket each test so the scheduler is +// freshly started and cleanly stopped each time. +// --------------------------------------------------------------------------- + +void register_scheduler_tests() +{ + MiniTest::Case("Scheduler", [](TestCase &tc) + { + // A guest fiber blocking in WaitSema is woken by SignalSema from a + // host thread, demonstrating the full cooperative block/wake cycle. + // The worker fiber blocks on sid (count=0); the host signals sid and then + // waits on done_sid (signalled by the worker after WaitSema returns). + tc.Run("WaitSema blocks guest fiber and SignalSema from host wakes it", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00300000u, &schedWorkerWaitAndSignal); + + // Semaphore the worker will block on (initial count=0). + const int32_t sid = createSchedSema(rdram.data(), &runtime, 0, 1); + t.IsTrue(sid > 0, "WaitSema test: work sema should be created"); + + // Done semaphore: worker signals this after WaitSema returns (maxCount=1). + const int32_t doneSid = createSchedSema(rdram.data(), &runtime, 0, 1); + t.IsTrue(doneSid > 0, "WaitSema test: done sema should be created"); + + if (sid <= 0 || doneSid <= 0) + { + deleteSchedSema(rdram.data(), &runtime, sid); + deleteSchedSema(rdram.data(), &runtime, doneSid); + return; + } + + std::memcpy(rdram.data() + kSchedTestSemaIdAddr, &sid, sizeof(sid)); + std::memcpy(rdram.data() + kSchedDoneSemaIdAddr, &doneSid, sizeof(doneSid)); + gSchedSeqCounter.store(0, std::memory_order_relaxed); + + // Start a worker fiber at priority 10. Pool threads will pick it up and + // it will call WaitSema(sid, count=0) → cooperative block. + const int32_t workerTid = startSchedWorker( + rdram.data(), &runtime, + 0x00300000u, 10, + 0x00400000u, 0x800u); + t.IsTrue(workerTid > 0, "WaitSema test: worker StartThread should succeed"); + + if (workerTid <= 0) + { + deleteSchedSema(rdram.data(), &runtime, sid); + deleteSchedSema(rdram.data(), &runtime, doneSid); + return; + } + + // Wait until the worker blocks on sid (waitThreads == 1). + const bool workerBlocked = waitUntil([&]() + { + constexpr uint32_t kStatusAddr = 0x2C00u; + R5900Context refCtx{}; + setRegU32(refCtx, 4, static_cast(sid)); + setRegU32(refCtx, 5, kStatusAddr); + ps2_syscalls::ReferSemaStatus(rdram.data(), &refCtx, &runtime); + int32_t waitThreads = 0; + std::memcpy(&waitThreads, rdram.data() + kStatusAddr + 12u, sizeof(waitThreads)); + return waitThreads >= 1; + }, std::chrono::milliseconds(500)); + + t.IsTrue(workerBlocked, "WaitSema test: worker fiber should block on sid within 500ms"); + + // Signal sid from the host thread — the pool will pick up the worker fiber. + { + R5900Context sigCtx{}; + setRegU32(sigCtx, 4, static_cast(sid)); + ps2_syscalls::SignalSema(rdram.data(), &sigCtx, &runtime); + t.Equals(getRegS32(sigCtx, 2), sid, "WaitSema test: SignalSema(sid) should return sid"); + } + + // Wait until the worker signals done_sid (i.e., its WaitSema returned). + const bool workerDone = waitUntil([&]() + { + constexpr uint32_t kStatusAddr = 0x2C80u; + R5900Context refCtx{}; + setRegU32(refCtx, 4, static_cast(doneSid)); + setRegU32(refCtx, 5, kStatusAddr); + ps2_syscalls::ReferSemaStatus(rdram.data(), &refCtx, &runtime); + int32_t count = 0; + std::memcpy(&count, rdram.data() + kStatusAddr + 0u, sizeof(count)); // current count field (offset 0 in ee_sema_t) + return count >= 1; + }, std::chrono::milliseconds(1000)); + + t.IsTrue(workerDone, "WaitSema test: worker should signal done_sid after WaitSema(sid) returns"); + + // Verify the worker's WaitSema(sid) returned sid. + { + int32_t workerResult = -9999; + std::memcpy(&workerResult, rdram.data() + kSchedResultBase, sizeof(workerResult)); + t.Equals(workerResult, sid, "WaitSema(sid) in worker fiber should return sid after SignalSema"); + } + + // Wait for the worker fiber to finish. + const bool finished = waitUntil([&]() + { + return g_activeThreads.load(std::memory_order_acquire) <= 0; + }, std::chrono::milliseconds(1000)); + t.IsTrue(finished, "WaitSema test: worker fiber should finish within 1s"); + + deleteSchedSema(rdram.data(), &runtime, sid); + deleteSchedSema(rdram.data(), &runtime, doneSid); + }); + + // TerminateThread of a guest fiber blocked in WaitSema. + // Verifies that request_terminate wakes the blocked fiber, it propagates + // ThreadExitException, and g_activeThreads decrements. + tc.Run("TerminateThread wakes a WaitSema-blocked fiber and cleans up", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00310000u, &schedWorkerBlockForever); + + // Semaphore the worker will block on indefinitely (count=0). + const int32_t blockSid = createSchedSema(rdram.data(), &runtime, 0, 1); + t.IsTrue(blockSid > 0, "TerminateThread test: block sema should be created"); + + if (blockSid <= 0) + { + return; + } + + std::memcpy(rdram.data() + kSchedTestSemaIdAddr, &blockSid, sizeof(blockSid)); + + const int32_t workerTid = startSchedWorker( + rdram.data(), &runtime, + 0x00310000u, 10, + 0x00500000u, 0x800u); + t.IsTrue(workerTid > 0, "TerminateThread test: worker fiber should start"); + + if (workerTid <= 0) + { + deleteSchedSema(rdram.data(), &runtime, blockSid); + return; + } + + // Wait until the worker fiber is blocked on blockSid. + const bool workerBlocked = waitUntil([&]() + { + constexpr uint32_t kStatusAddr = 0x2C00u; + R5900Context refCtx{}; + setRegU32(refCtx, 4, static_cast(blockSid)); + setRegU32(refCtx, 5, kStatusAddr); + ps2_syscalls::ReferSemaStatus(rdram.data(), &refCtx, &runtime); + int32_t waitThreads = 0; + std::memcpy(&waitThreads, rdram.data() + kStatusAddr + 12u, sizeof(waitThreads)); + return waitThreads >= 1; + }, std::chrono::milliseconds(500)); + + t.IsTrue(workerBlocked, "TerminateThread test: worker should block on sema within 500ms"); + + const int activeBeforeTerminate = g_activeThreads.load(std::memory_order_acquire); + + // Terminate the worker fiber. + { + R5900Context termCtx{}; + setRegU32(termCtx, 4, static_cast(workerTid)); + ps2_syscalls::TerminateThread(rdram.data(), &termCtx, &runtime); + t.Equals(getRegS32(termCtx, 2), KE_OK, "TerminateThread should return KE_OK"); + } + + // g_activeThreads should decrement as the fiber unwinds. + const bool drained = waitUntil([&]() + { + return g_activeThreads.load(std::memory_order_acquire) < activeBeforeTerminate; + }, std::chrono::milliseconds(1000)); + t.IsTrue(drained, "TerminateThread: g_activeThreads should decrement after fiber exits"); + + deleteSchedSema(rdram.data(), &runtime, blockSid); + }); + + // DeleteSema while N guest fibers are blocked in WaitSema. + // Each waiter must receive KE_WAIT_DELETE; the sema id must be invalidated. + // The fibers signal done_sid after receiving KE_WAIT_DELETE so we can wait + // for all N completions from the host side. + tc.Run("DeleteSema wakes all blocked fibers with KE_WAIT_DELETE", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00320000u, &schedWorkerWaitAndSignal); + + // Semaphore workers block on (count=0). + const int32_t sid = createSchedSema(rdram.data(), &runtime, 0, 1); + t.IsTrue(sid > 0, "DeleteSema test: block sema should be created"); + + // Done semaphore: each worker signals it after WaitSema returns. + // maxCount=3 so all 3 signals can accumulate. + constexpr int kNumWorkers = 3; + const int32_t doneSid = createSchedSema(rdram.data(), &runtime, 0, kNumWorkers); + t.IsTrue(doneSid > 0, "DeleteSema test: done sema should be created"); + + if (sid <= 0 || doneSid <= 0) + { + deleteSchedSema(rdram.data(), &runtime, sid); + deleteSchedSema(rdram.data(), &runtime, doneSid); + return; + } + + std::memcpy(rdram.data() + kSchedTestSemaIdAddr, &sid, sizeof(sid)); + std::memcpy(rdram.data() + kSchedDoneSemaIdAddr, &doneSid, sizeof(doneSid)); + gSchedSeqCounter.store(0, std::memory_order_relaxed); + + constexpr uint32_t kWorkerStackBase = 0x00600000u; + constexpr uint32_t kWorkerStackSize = 0x800u; + + int32_t workerTids[kNumWorkers] = {-1, -1, -1}; + for (int i = 0; i < kNumWorkers; ++i) + { + const uint32_t stackAddr = + kWorkerStackBase + static_cast(i) * kWorkerStackSize * 2u; + workerTids[i] = startSchedWorker( + rdram.data(), &runtime, + 0x00320000u, 10, + stackAddr, kWorkerStackSize); + t.IsTrue(workerTids[i] > 0, + std::string("DeleteSema test: worker ") + std::to_string(i) + " should start"); + } + + // Wait until all N workers are blocked on sid (waitThreads == N). + const bool allBlocked = waitUntil([&]() + { + constexpr uint32_t kStatusAddr = 0x2D00u; + R5900Context refCtx{}; + setRegU32(refCtx, 4, static_cast(sid)); + setRegU32(refCtx, 5, kStatusAddr); + ps2_syscalls::ReferSemaStatus(rdram.data(), &refCtx, &runtime); + int32_t waiters = 0; + std::memcpy(&waiters, rdram.data() + kStatusAddr + 12u, sizeof(waiters)); + return waiters >= kNumWorkers; + }, std::chrono::milliseconds(1000)); + + t.IsTrue(allBlocked, "DeleteSema test: all workers should block on sid within 1s"); + + // DeleteSema: marks deleted, wakes all N blocked fibers via make_ready. + { + R5900Context delCtx{}; + setRegU32(delCtx, 4, static_cast(sid)); + ps2_syscalls::DeleteSema(rdram.data(), &delCtx, &runtime); + t.Equals(getRegS32(delCtx, 2), sid, "DeleteSema should return sid"); + } + + // Wait until all N workers have signalled done_sid (count == N). + const bool allDone = waitUntil([&]() + { + return gSchedSeqCounter.load(std::memory_order_acquire) >= kNumWorkers; + }, std::chrono::milliseconds(2000)); + + t.IsTrue(allDone, "DeleteSema test: all worker fibers should complete within 2s"); + + // Verify all workers received KE_WAIT_DELETE. + for (int i = 0; i < kNumWorkers; ++i) + { + int32_t result = 0; + std::memcpy(&result, + rdram.data() + kSchedResultBase + static_cast(i * 4), + sizeof(result)); + t.Equals(result, KE_WAIT_DELETE, + std::string("DeleteSema: worker ") + std::to_string(i) + + " WaitSema should return KE_WAIT_DELETE"); + } + + // PollSema on the deleted id should return KE_UNKNOWN_SEMID. + { + R5900Context laterCtx{}; + setRegU32(laterCtx, 4, static_cast(sid)); + ps2_syscalls::PollSema(rdram.data(), &laterCtx, &runtime); + t.Equals(getRegS32(laterCtx, 2), KE_UNKNOWN_SEMID, + "PollSema on a deleted sema id should return KE_UNKNOWN_SEMID"); + } + + // Wait for all fibers to finish and clean up. + const bool finished = waitUntil([&]() + { + return g_activeThreads.load(std::memory_order_acquire) <= 0; + }, std::chrono::milliseconds(2000)); + t.IsTrue(finished, "DeleteSema test: all fibers should finish within 2s"); + + deleteSchedSema(rdram.data(), &runtime, doneSid); + }); + }); +} + +// --------------------------------------------------------------------------- +// Scheduler protocol tests — 19-test suite +// --------------------------------------------------------------------------- + +namespace +{ + // ----------------------------------------------------------------------- + // RDRAM slot constants + // ----------------------------------------------------------------------- + static constexpr uint32_t kTParamAddr = 0x00004000u; + static constexpr uint32_t kSParamAddr = 0x00004020u; + static constexpr uint32_t kEParamAddr = 0x00004040u; + static constexpr uint32_t kReferScratch = 0x00004060u; + static constexpr uint32_t kSlotWorkSid = 0x00004100u; + static constexpr uint32_t kSlotDoneSid = 0x00004104u; + static constexpr uint32_t kSlotGoSid = 0x00004108u; + static constexpr uint32_t kSlotEid = 0x0000410Cu; + static constexpr uint32_t kSlotTidParam = 0x00004110u; + static constexpr uint32_t kSlotResBits = 0x00004114u; + static constexpr uint32_t kRunLog = 0x00004200u; + static constexpr uint32_t kResultBase = 0x00004260u; + static constexpr uint32_t kHostInGuestProbe = 0x0000428Cu; + + // These four mailboxes are polled across a host test thread <-> guest + // fiber thread boundary (see the step functions below and the T-suite + // tests). They must be atomics with acquire/release ordering rather than + // plain rdram-resident words: a plain memcpy poll establishes no + // happens-before edge, so ThreadSanitizer (correctly) flags the payload + // writes that accompany them (kRunLog/kResultBase) as racing. The + // release store here publishes any plain writes that happened-before it + // on the same thread; the matching acquire load on the polling thread + // makes those writes visible once the new value is observed. + // + // gSeq specifically is incremented by up to several different fiber + // threads within a single test (e.g. T1 has 3 fibers each completing and + // bumping it once). It must be incremented via fetch_add (an atomic + // read-modify-write), not a separate load()+store() pair: per the C++ + // memory model, a "release sequence" -- the chain that lets one acquire + // load synchronize-with every release store that contributed to it -- + // only extends across stores from *different* threads if those stores + // are RMW operations. A plain store() from a second thread breaks the + // release sequence started by the first thread's store(), so the host's + // final acquire-load would only be guaranteed to see the *last* fiber's + // writes, not the earlier ones -- an intermittent, hard-to-reproduce + // failure (observed as an occasional wrong-order T1 read) rather than + // anything ThreadSanitizer's happens-before model would catch, since no + // two fibers ever truly race on the counter's value (the N=1 scheduler + // serializes them) -- only its *visibility* to the host was at risk. + static std::atomic gSeq{0}; + static std::atomic gStartedFlag{0}; + static std::atomic gStopFlag{0}; + static std::atomic gProgressCtr{0}; + + // Thread status constants (mirrors State.h) + static constexpr int32_t kTHSRun = 0x01; + static constexpr int32_t kTHSWait = 0x04; + static constexpr int32_t kTHSSuspend = 0x08; + static constexpr int32_t kTHSWaitSuspend = 0x0c; + static constexpr int32_t kTSWSleep = 1; + static constexpr int32_t kTSWSema = 2; + + // ----------------------------------------------------------------------- + // Protocol-test helper: getSemaWaiters (reads waiters from ReferSemaStatus) + // ----------------------------------------------------------------------- + int32_t getSemaWaiters(uint8_t *rdram, PS2Runtime *runtime, int32_t sid) + { + constexpr uint32_t kStatusScratch = 0x2C00u; + R5900Context refCtx{}; + setRegU32(refCtx, 4, static_cast(sid)); + setRegU32(refCtx, 5, kStatusScratch); + ps2_syscalls::ReferSemaStatus(rdram, &refCtx, runtime); + int32_t waiters = 0; + std::memcpy(&waiters, rdram + kStatusScratch + 12u, sizeof(waiters)); + return waiters; + } + + // ----------------------------------------------------------------------- + // Helper: getThreadStatus — reads fiber status+waitType via ReferThreadStatus + // ----------------------------------------------------------------------- + static void getThreadStatus(std::vector &rdram, PS2Runtime &runtime, + int tid, int32_t &outStatus, int32_t &outWaitType) + { + R5900Context ctx{}; + setRegU32(ctx, 4, static_cast(tid)); + setRegU32(ctx, 5, kReferScratch); + ps2_syscalls::ReferThreadStatus(rdram.data(), &ctx, &runtime); + int32_t s = 0, w = 0; + std::memcpy(&s, rdram.data() + kReferScratch + 0x00, 4); + std::memcpy(&w, rdram.data() + kReferScratch + 0x24, 4); + outStatus = s; + outWaitType = w; + } + + // ----------------------------------------------------------------------- + // Helper: getEvfNumThreads — reads numThreads via ReferEventFlagStatus + // ----------------------------------------------------------------------- + static int32_t getEvfNumThreads(std::vector &rdram, PS2Runtime &runtime, int eid) + { + R5900Context ctx{}; + setRegU32(ctx, 4, static_cast(eid)); + setRegU32(ctx, 5, kReferScratch); + ps2_syscalls::ReferEventFlagStatus(rdram.data(), &ctx, &runtime); + // Ps2EventFlagInfo: attr+0, option+4, initBits+8, currBits+12, numThreads+16 + int32_t num = 0; + std::memcpy(&num, rdram.data() + kReferScratch + 16, 4); + return num; + } + + // ----------------------------------------------------------------------- + // Helper: getSemaCount — reads current count via ReferSemaStatus + // ----------------------------------------------------------------------- + static int32_t getSemaCount(std::vector &rdram, PS2Runtime *runtime, int sid) + { + R5900Context ctx{}; + setRegU32(ctx, 4, static_cast(sid)); + setRegU32(ctx, 5, kReferScratch); + ps2_syscalls::ReferSemaStatus(rdram.data(), &ctx, runtime); + // ee_sema_t: count+0 + int32_t count = 0; + std::memcpy(&count, rdram.data() + kReferScratch + 0, 4); + return count; + } + + // ----------------------------------------------------------------------- + // Helper: createSchedEvf — creates an event flag + // ----------------------------------------------------------------------- + static int createSchedEvf(std::vector &rdram, PS2Runtime *runtime, + uint32_t attr, uint32_t initBits) + { + uint32_t vals[3] = { attr, 0u, initBits }; + std::memcpy(rdram.data() + kEParamAddr, vals, 12); + R5900Context ctx{}; + setRegU32(ctx, 4, kEParamAddr); + ps2_syscalls::CreateEventFlag(rdram.data(), &ctx, runtime); + return getRegS32(ctx, 2); + } + + static void deleteSchedEvf(std::vector &rdram, PS2Runtime *runtime, int eid) + { + R5900Context ctx{}; + setRegU32(ctx, 4, static_cast(eid)); + ps2_syscalls::DeleteEventFlag(rdram.data(), &ctx, runtime); + } + + // ----------------------------------------------------------------------- + // Helper: rdram write/read helpers + // ----------------------------------------------------------------------- + static void rdramWrite32(std::vector &rdram, uint32_t addr, uint32_t val) + { + std::memcpy(rdram.data() + addr, &val, 4); + } + + static uint32_t rdramSeq(const std::vector & /*rdram*/) + { + return gSeq.load(std::memory_order_acquire); + } + + static void rdramSeqReset(std::vector & /*rdram*/) + { + gSeq.store(0u, std::memory_order_release); + } + + // ----------------------------------------------------------------------- + // Step functions for the 19 protocol tests + // All use the atomic gSeq counter (release/acquire) to publish per-step + // ----------------------------------------------------------------------- + + // stepWaitSemaRecordSignal: wait on workSid, record result+tid, signal doneSid + static void stepWaitSemaRecordSignal(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + int32_t workSid = 0, doneSid = 0; + std::memcpy(&workSid, rdram + kSlotWorkSid, 4); + std::memcpy(&doneSid, rdram + kSlotDoneSid, 4); + R5900Context sc{}; + setRegU32(sc, 4, static_cast(workSid)); + ps2_syscalls::WaitSema(rdram, &sc, runtime); + int32_t ret = getRegS32(sc, 2); + const uint32_t seq = gSeq.load(std::memory_order_relaxed); + std::memcpy(rdram + kResultBase + seq * 4, &ret, 4); + int32_t tid = g_currentThreadId; + std::memcpy(rdram + kRunLog + seq * 4, &tid, 4); + gSeq.fetch_add(1u, std::memory_order_release); // RMW: preserves release-sequence chaining (see gSeq/gAASeq declaration above) + if (doneSid > 0) + { + R5900Context sc2{}; + setRegU32(sc2, 4, static_cast(doneSid)); + ps2_syscalls::SignalSema(rdram, &sc2, runtime); + } + ctx->pc = 0u; + } + + // stepWaitGateLogOrder: wait on goSid, log tid in run order + static void stepWaitGateLogOrder(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + int32_t goSid = 0; + std::memcpy(&goSid, rdram + kSlotGoSid, 4); + R5900Context sc{}; + setRegU32(sc, 4, static_cast(goSid)); + ps2_syscalls::WaitSema(rdram, &sc, runtime); + const uint32_t seq = gSeq.load(std::memory_order_relaxed); + int32_t tid = g_currentThreadId; + std::memcpy(rdram + kRunLog + seq * 4, &tid, 4); + gSeq.fetch_add(1u, std::memory_order_release); // RMW: preserves release-sequence chaining (see gSeq/gAASeq declaration above) + ctx->pc = 0u; + } + + // stepSpinUntilStop: set kStartedFlag, spin yielding, log when stopped + static void stepSpinUntilStop(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + gStartedFlag.store(1u, std::memory_order_release); + while (gStopFlag.load(std::memory_order_acquire) == 0u) + { + runtime->shouldPreemptGuestExecution(); + } + const uint32_t seq = gSeq.load(std::memory_order_relaxed); + int32_t tid = g_currentThreadId; + std::memcpy(rdram + kRunLog + seq * 4, &tid, 4); + gSeq.fetch_add(1u, std::memory_order_release); // RMW: preserves release-sequence chaining (see gSeq/gAASeq declaration above) + ctx->pc = 0u; + } + + // stepProgressLoop: set kStartedFlag, increment kProgressCtr each iter, stop on kStopFlag + static void stepProgressLoop(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + (void)rdram; + gStartedFlag.store(1u, std::memory_order_release); + while (gStopFlag.load(std::memory_order_acquire) == 0u) + { + gProgressCtr.fetch_add(1u, std::memory_order_release); + runtime->shouldPreemptGuestExecution(); + } + ctx->pc = 0u; + } + + // stepSleepRecordSignal: SleepThread, then record + signal doneSid + static void stepSleepRecordSignal(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + int32_t doneSid = 0; + std::memcpy(&doneSid, rdram + kSlotDoneSid, 4); + R5900Context sc{}; + ps2_syscalls::SleepThread(rdram, &sc, runtime); + const uint32_t seq = gSeq.load(std::memory_order_relaxed); + int32_t tid = g_currentThreadId; + std::memcpy(rdram + kRunLog + seq * 4, &tid, 4); + gSeq.fetch_add(1u, std::memory_order_release); // RMW: preserves release-sequence chaining (see gSeq/gAASeq declaration above) + if (doneSid > 0) + { + R5900Context sc2{}; + setRegU32(sc2, 4, static_cast(doneSid)); + ps2_syscalls::SignalSema(rdram, &sc2, runtime); + } + ctx->pc = 0u; + } + + // T11: publishes its own current_fiber_token() so a foreign host thread can + // call enqueue_external_wakeup_validated() with a REAL (not stale) token. + static std::atomic gT11TokenLo{0}; + static std::atomic gT11TokenHi{0}; + + // stepSleepRecordSignalPublishToken: publish current_fiber_token(), then the + // same SleepThread/record/signal sequence as stepSleepRecordSignal. + static void stepSleepRecordSignalPublishToken(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + const uint64_t tok = ps2sched::current_fiber_token(); + gT11TokenLo.store(static_cast(tok & 0xFFFFFFFFu), std::memory_order_relaxed); + gT11TokenHi.store(static_cast(tok >> 32u), std::memory_order_release); + stepSleepRecordSignal(rdram, ctx, runtime); + } + + // stepWaitEvfAndRecord: WaitEventFlag(eid, 0x3, AND no-clear), record result+tid + static void stepWaitEvfAndRecord(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + int32_t eid = 0, doneSid = 0; + std::memcpy(&eid, rdram + kSlotEid, 4); + std::memcpy(&doneSid, rdram + kSlotDoneSid, 4); + R5900Context sc{}; + setRegU32(sc, 4, static_cast(eid)); + setRegU32(sc, 5, 0x3u); + setRegU32(sc, 6, 0x0u); + setRegU32(sc, 7, kSlotResBits); + ps2_syscalls::WaitEventFlag(rdram, &sc, runtime); + int32_t ret = getRegS32(sc, 2); + const uint32_t seq = gSeq.load(std::memory_order_relaxed); + std::memcpy(rdram + kResultBase + seq * 4, &ret, 4); + int32_t tid = g_currentThreadId; + std::memcpy(rdram + kRunLog + seq * 4, &tid, 4); + gSeq.fetch_add(1u, std::memory_order_release); // RMW: preserves release-sequence chaining (see gSeq/gAASeq declaration above) + if (doneSid > 0) + { + R5900Context sc2{}; + setRegU32(sc2, 4, static_cast(doneSid)); + ps2_syscalls::SignalSema(rdram, &sc2, runtime); + } + ctx->pc = 0u; + } + + // stepChangePrioOfTargetThenLog: read tidX from kSlotTidParam, bump to prio=5, yield 500x, log self + static void stepChangePrioOfTargetThenLog(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + int32_t targetTid = 0; + std::memcpy(&targetTid, rdram + kSlotTidParam, 4); + R5900Context sc{}; + setRegU32(sc, 4, static_cast(targetTid)); + setRegU32(sc, 5, 5u); + ps2_syscalls::ChangeThreadPriority(rdram, &sc, runtime); + for (int i = 0; i < 500; ++i) + { + runtime->shouldPreemptGuestExecution(); + } + const uint32_t seq = gSeq.load(std::memory_order_relaxed); + int32_t tid = g_currentThreadId; + std::memcpy(rdram + kRunLog + seq * 4, &tid, 4); + gSeq.fetch_add(1u, std::memory_order_release); // RMW: preserves release-sequence chaining (see gSeq/gAASeq declaration above) + ctx->pc = 0u; + } + + // stepSpinLogAtEntry: log tid at entry and exit. + // T4 uses this as fiber X: after Y bumps X's priority, the executor + // should preempt Y and run X. X just needs to log its tid at entry to + // prove the preemption happened; spinning with high priority would starve + // Y (since Y's lower-numbered priority < X means X always wins) and + // create a deadlock where neither fiber can complete. + static void stepSpinLogAtEntry(uint8_t *rdram, R5900Context *ctx, PS2Runtime * /*runtime*/) + { + const uint32_t seq = gSeq.load(std::memory_order_relaxed); + int32_t tid = g_currentThreadId; + std::memcpy(rdram + kRunLog + seq * 4, &tid, 4); + gSeq.fetch_add(1u, std::memory_order_release); // RMW: preserves release-sequence chaining (see gSeq/gAASeq declaration above) + gStartedFlag.store(1u, std::memory_order_release); + ctx->pc = 0u; + } + + // stepBlockForever: WaitSema on workSid (never wakes normally) + static void stepBlockForever(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + int32_t workSid = 0; + std::memcpy(&workSid, rdram + kSlotWorkSid, 4); + R5900Context sc{}; + setRegU32(sc, 4, static_cast(workSid)); + ps2_syscalls::WaitSema(rdram, &sc, runtime); + const uint32_t seq = gSeq.load(std::memory_order_relaxed); + int32_t ret = getRegS32(sc, 2); + std::memcpy(rdram + kResultBase + seq * 4, &ret, 4); + int32_t tid = g_currentThreadId; + std::memcpy(rdram + kRunLog + seq * 4, &tid, 4); + gSeq.fetch_add(1u, std::memory_order_release); // RMW: preserves release-sequence chaining (see gSeq/gAASeq declaration above) + ctx->pc = 0u; + } + + // stepHoldSilentExit: blocks on kSlotWorkSid then exits without logging + static void stepHoldSilentExit(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + int32_t holdSid = 0; + std::memcpy(&holdSid, rdram + kSlotWorkSid, 4); + R5900Context sc{}; + setRegU32(sc, 4, static_cast(holdSid)); + ps2_syscalls::WaitSema(rdram, &sc, runtime); + ctx->pc = 0u; + } + + // stepTerminateTarget: reads target tid, calls TerminateThread on it, logs self + static void stepTerminateTarget(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + int32_t targetTid = 0; + std::memcpy(&targetTid, rdram + kSlotTidParam, 4); + R5900Context sc{}; + setRegU32(sc, 4, static_cast(targetTid)); + ps2_syscalls::TerminateThread(rdram, &sc, runtime); + const uint32_t seq = gSeq.load(std::memory_order_relaxed); + int32_t tid = g_currentThreadId; + std::memcpy(rdram + kRunLog + seq * 4, &tid, 4); + gSeq.fetch_add(1u, std::memory_order_release); // RMW: preserves release-sequence chaining (see gSeq/gAASeq declaration above) + ctx->pc = 0u; + } + + // stepLogAndExit: log tid and exit immediately + static void stepLogAndExit(uint8_t *rdram, R5900Context *ctx, PS2Runtime * /*runtime*/) + { + const uint32_t seq = gSeq.load(std::memory_order_relaxed); + int32_t tid = g_currentThreadId; + std::memcpy(rdram + kRunLog + seq * 4, &tid, 4); + gSeq.fetch_add(1u, std::memory_order_release); // RMW: preserves release-sequence chaining (see gSeq/gAASeq declaration above) + ctx->pc = 0u; + } + +} // anonymous namespace + +// --------------------------------------------------------------------------- +// 19-test scheduler protocol suite +// --------------------------------------------------------------------------- +void register_scheduler_protocol_tests() +{ + MiniTest::Case("SchedulerProtocol", [](TestCase &tc) + { + // ------------------------------------------------------------------ + // Test 1: Priority ordering — 3 fibers at priority 5, 10, 20 + // ------------------------------------------------------------------ + tc.Run("fibers run in priority order 5 then 10 then 20", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00600000u, &stepWaitGateLogOrder); + + const int32_t goSid = createSchedSema(rdram.data(), &runtime, 0, 3); + t.IsTrue(goSid > 0, "T1: gate sema created"); + if (goSid <= 0) + { + return; + } + + { + int32_t v = goSid; + std::memcpy(rdram.data() + kSlotGoSid, &v, 4); + } + rdramSeqReset(rdram); + std::memset(rdram.data() + kRunLog, 0, 32u); + + const int32_t tidC = startSchedWorker(rdram.data(), &runtime, 0x00600000u, 20, 0x00500000u, 0x2000u); + const int32_t tidA = startSchedWorker(rdram.data(), &runtime, 0x00600000u, 5, 0x00502000u, 0x2000u); + const int32_t tidB = startSchedWorker(rdram.data(), &runtime, 0x00600000u, 10, 0x00504000u, 0x2000u); + t.IsTrue(tidA > 0, "T1: A started"); t.IsTrue(tidB > 0, "T1: B started"); t.IsTrue(tidC > 0, "T1: C started"); + + const bool allBlocked = waitUntil([&]() + { + return getSemaWaiters(rdram.data(), &runtime, goSid) >= 3; + }, std::chrono::milliseconds(1000)); + t.IsTrue(allBlocked, "T1: all 3 blocked on gate sema"); + + // Lock out the executor (same pattern as T3/T4 below) while issuing + // all 3 signals so all 3 fibers are enqueued Ready -- in priority + // order -- before the executor dispatches any of them. Without this, + // SignalSema's wake for the first fiber can reach the executor and + // let it run to completion before the 2nd/3rd SignalSema calls even + // happen (each call here still takes a real mutex and does real + // work), degrading the ordering guarantee from "picked by priority + // among 3 simultaneously-ready fibers" to "whoever the sema's FIFO + // waitList happened to pop first" -- a race that widens enormously + // under something like ThreadSanitizer's instrumentation slowdown + // (this exact test failed under TSan before this fix, on an + // otherwise pristine baseline run). + ps2sched::async_guest_begin(); + for (int i = 0; i < 3; ++i) + { + R5900Context sc{}; setRegU32(sc, 4, static_cast(goSid)); + ps2_syscalls::SignalSema(rdram.data(), &sc, &runtime); + } + ps2sched::async_guest_end(); + + const bool allDone = waitUntil([&](){ return rdramSeq(rdram) >= 3u; }, std::chrono::milliseconds(1000)); + t.IsTrue(allDone, "T1: all 3 completed"); + + int32_t log[3] = {}; + std::memcpy(log, rdram.data() + kRunLog, 12); + t.Equals(log[0], tidA, "T1: prio 5 runs first"); + t.Equals(log[1], tidB, "T1: prio 10 runs second"); + t.Equals(log[2], tidC, "T1: prio 20 runs last"); + + waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + deleteSchedSema(rdram.data(), &runtime, goSid); + }); + + // ------------------------------------------------------------------ + // Test 2: FIFO within equal priority + // ------------------------------------------------------------------ + tc.Run("equal-priority fibers run in FIFO creation order", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00610000u, &stepWaitGateLogOrder); + + const int32_t goSid = createSchedSema(rdram.data(), &runtime, 0, 2); + t.IsTrue(goSid > 0, "T2: gate sema created"); + if (goSid <= 0) + { + return; + } + + { + int32_t v = goSid; + std::memcpy(rdram.data() + kSlotGoSid, &v, 4); + } + rdramSeqReset(rdram); + std::memset(rdram.data() + kRunLog, 0, 16u); + + const int32_t tidA = startSchedWorker(rdram.data(), &runtime, 0x00610000u, 10, 0x00510000u, 0x2000u); + const int32_t tidB = startSchedWorker(rdram.data(), &runtime, 0x00610000u, 10, 0x00512000u, 0x2000u); + t.IsTrue(tidA > 0, "T2: A started"); t.IsTrue(tidB > 0, "T2: B started"); + + const bool allBlocked = waitUntil([&]() + { + return getSemaWaiters(rdram.data(), &runtime, goSid) >= 2; + }, std::chrono::milliseconds(1000)); + t.IsTrue(allBlocked, "T2: both blocked on gate sema"); + + // Lock out the executor while signaling both (see T1's comment above + // for why: otherwise the first wake can run to completion before the + // second SignalSema call even happens). + ps2sched::async_guest_begin(); + for (int i = 0; i < 2; ++i) + { + R5900Context sc{}; setRegU32(sc, 4, static_cast(goSid)); + ps2_syscalls::SignalSema(rdram.data(), &sc, &runtime); + } + ps2sched::async_guest_end(); + + const bool allDone = waitUntil([&](){ return rdramSeq(rdram) >= 2u; }, std::chrono::milliseconds(1000)); + t.IsTrue(allDone, "T2: both completed"); + + int32_t log[2] = {}; + std::memcpy(log, rdram.data() + kRunLog, 8); + t.Equals(log[0], tidA, "T2: A (created first) runs first"); + t.Equals(log[1], tidB, "T2: B (created second) runs second"); + + waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + deleteSchedSema(rdram.data(), &runtime, goSid); + }); + + // ------------------------------------------------------------------ + // Test 3: RotateThreadReadyQueue — race-free via async_guest_begin + // ------------------------------------------------------------------ + tc.Run("RotateThreadReadyQueue reorders equal-priority ready fibers [A,B,C]->[B,C,A]", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00620000u, &stepLogAndExit); + + rdramSeqReset(rdram); + std::memset(rdram.data() + kRunLog, 0, 32u); + + // Acquire guest token so executor cannot run fibers yet + ps2sched::async_guest_begin(); + + const int32_t tidA = startSchedWorker(rdram.data(), &runtime, 0x00620000u, 10, 0x00520000u, 0x2000u); + const int32_t tidB = startSchedWorker(rdram.data(), &runtime, 0x00620000u, 10, 0x00522000u, 0x2000u); + const int32_t tidC = startSchedWorker(rdram.data(), &runtime, 0x00620000u, 10, 0x00524000u, 0x2000u); + t.IsTrue(tidA > 0, "T3: A started"); t.IsTrue(tidB > 0, "T3: B started"); t.IsTrue(tidC > 0, "T3: C started"); + + // Rotate [A,B,C] -> [B,C,A] while executor is locked out + { + R5900Context sc{}; setRegU32(sc, 4, 10u); + ps2_syscalls::RotateThreadReadyQueue(rdram.data(), &sc, &runtime); + } + + // Release executor + ps2sched::async_guest_end(); + + const bool allDone = waitUntil([&](){ return rdramSeq(rdram) >= 3u; }, std::chrono::milliseconds(2000)); + t.IsTrue(allDone, "T3: all 3 fibers logged"); + + int32_t log[3] = {}; + std::memcpy(log, rdram.data() + kRunLog, 12); + t.Equals(log[0], tidB, "T3: B runs first after rotate"); + t.Equals(log[1], tidC, "T3: C runs second after rotate"); + t.Equals(log[2], tidA, "T3: A runs last (moved to tail)"); + + waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + }); + + // ------------------------------------------------------------------ + // Test 4: ChangeThreadPriority re-sorts Ready fiber + // ------------------------------------------------------------------ + tc.Run("ChangeThreadPriority raises X above Y causing X to preempt Y", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00630000u, &stepChangePrioOfTargetThenLog); + runtime.registerFunction(0x00638000u, &stepSpinLogAtEntry); + + rdramSeqReset(rdram); + std::memset(rdram.data() + kRunLog, 0, 32u); + gStartedFlag.store(0u, std::memory_order_release); + gStopFlag.store(0u, std::memory_order_release); + + // Lock out executor + ps2sched::async_guest_begin(); + + const int32_t tidX = startSchedWorker(rdram.data(), &runtime, 0x00638000u, 20, 0x00532000u, 0x2000u); + const int32_t tidY = startSchedWorker(rdram.data(), &runtime, 0x00630000u, 10, 0x00530000u, 0x2000u); + t.IsTrue(tidX > 0, "T4: X started"); t.IsTrue(tidY > 0, "T4: Y started"); + + { + int32_t v = tidX; + std::memcpy(rdram.data() + kSlotTidParam, &v, 4); + } + + // Release executor: Y runs first (prio 10), calls ChangeThreadPriority(X, 5), + // X bumped to prio=5 (higher than Y=10), Y yields 500x -> X preempts + ps2sched::async_guest_end(); + + // Wait for both to complete (Y logs after 500-yield loop; X logs at entry) + const bool allDone = waitUntil([&](){ return rdramSeq(rdram) >= 2u; }, std::chrono::milliseconds(2000)); + t.IsTrue(allDone, "T4: both fibers completed"); + + // Wait for both fibers to exit naturally. + waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + + int32_t log[2] = {}; + std::memcpy(log, rdram.data() + kRunLog, 8); + t.Equals(log[0], tidX, "T4: X (bumped to prio=5) logs first"); + t.Equals(log[1], tidY, "T4: Y logs second (after yielding to X)"); + + }); + + // ------------------------------------------------------------------ + // Test 5: SuspendThread / ResumeThread + // ------------------------------------------------------------------ + tc.Run("SuspendThread stops progress and ResumeThread restores it", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00640000u, &stepProgressLoop); + + gProgressCtr.store(0u, std::memory_order_release); + gStartedFlag.store(0u, std::memory_order_release); + gStopFlag.store(0u, std::memory_order_release); + + const int32_t tid = startSchedWorker(rdram.data(), &runtime, 0x00640000u, 10, 0x00540000u, 0x2000u); + t.IsTrue(tid > 0, "T5: progress fiber started"); + if (tid <= 0) + { + gStopFlag.store(1u, std::memory_order_release); + return; + } + + const bool started = waitUntil([&]() + { + return gStartedFlag.load(std::memory_order_acquire) != 0u; + }, std::chrono::milliseconds(500)); + t.IsTrue(started, "T5: fiber started"); + + { + R5900Context sc{}; setRegU32(sc, 4, static_cast(tid)); + ps2_syscalls::SuspendThread(rdram.data(), &sc, &runtime); + t.Equals(getRegS32(sc, 2), KE_OK, "T5: SuspendThread returned KE_OK"); + } + + // Wait for THS_SUSPEND status AND for the progress counter to stop + // advancing. The fiber may still be running for up to 127 iterations + // after SuspendThread sets info->status; the counter check ensures + // it has truly parked before we sample cBefore. + const bool suspended = waitUntil([&]() + { + int32_t status = 0, wt = 0; + getThreadStatus(rdram, runtime, tid, status, wt); + if (status != kTHSSuspend) return false; + // Also confirm the counter has stopped advancing. + const uint32_t c1 = gProgressCtr.load(std::memory_order_acquire); + std::this_thread::sleep_for(std::chrono::milliseconds(5)); + const uint32_t c2 = gProgressCtr.load(std::memory_order_acquire); + return c1 == c2; + }, std::chrono::milliseconds(500)); + t.IsTrue(suspended, "T5: fiber reached THS_SUSPEND"); + + // Verify progress halted + const uint32_t cBefore = gProgressCtr.load(std::memory_order_acquire); + std::this_thread::sleep_for(std::chrono::milliseconds(50)); + const uint32_t cAfter = gProgressCtr.load(std::memory_order_acquire); + t.Equals(cBefore, cAfter, "T5: counter not advancing while suspended"); + + { + R5900Context sc{}; setRegU32(sc, 4, static_cast(tid)); + ps2_syscalls::ResumeThread(rdram.data(), &sc, &runtime); + t.Equals(getRegS32(sc, 2), KE_OK, "T5: ResumeThread returned KE_OK"); + } + + const bool advanced = waitUntil([&]() + { + return gProgressCtr.load(std::memory_order_acquire) > cAfter; + }, std::chrono::milliseconds(500)); + t.IsTrue(advanced, "T5: progress resumes after ResumeThread"); + + gStopFlag.store(1u, std::memory_order_release); + waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + }); + + // ------------------------------------------------------------------ + // Test 6: SuspendThread while Blocked gates make_ready + // ------------------------------------------------------------------ + tc.Run("suspended fiber does not wake from SignalSema until ResumeThread", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00650000u, &stepWaitSemaRecordSignal); + + const int32_t workSid = createSchedSema(rdram.data(), &runtime, 0, 1); + const int32_t doneSid = createSchedSema(rdram.data(), &runtime, 0, 1); + t.IsTrue(workSid > 0, "T6: work sema created"); t.IsTrue(doneSid > 0, "T6: done sema created"); + if (workSid <= 0 || doneSid <= 0) + { + deleteSchedSema(rdram.data(), &runtime, workSid); + deleteSchedSema(rdram.data(), &runtime, doneSid); + return; + } + + { + int32_t w = workSid, d = doneSid; + std::memcpy(rdram.data() + kSlotWorkSid, &w, 4); + std::memcpy(rdram.data() + kSlotDoneSid, &d, 4); + } + rdramSeqReset(rdram); + { + int32_t bad = -9999; + std::memcpy(rdram.data() + kResultBase, &bad, 4); + } + + const int32_t tid = startSchedWorker(rdram.data(), &runtime, 0x00650000u, 10, 0x00550000u, 0x2000u); + t.IsTrue(tid > 0, "T6: fiber started"); + + const bool blocked = waitUntil([&]() + { + return getSemaWaiters(rdram.data(), &runtime, workSid) >= 1; + }, std::chrono::milliseconds(500)); + t.IsTrue(blocked, "T6: fiber blocked on workSid"); + + { + R5900Context sc{}; setRegU32(sc, 4, static_cast(tid)); + ps2_syscalls::SuspendThread(rdram.data(), &sc, &runtime); + t.Equals(getRegS32(sc, 2), KE_OK, "T6: SuspendThread returned KE_OK"); + } + + const bool isSuspended = waitUntil([&]() + { + int32_t status = 0, wt = 0; + getThreadStatus(rdram, runtime, tid, status, wt); + return status == kTHSWaitSuspend; + }, std::chrono::milliseconds(500)); + t.IsTrue(isSuspended, "T6: fiber reached THS_WAITSUSPEND"); + + { + R5900Context sc{}; setRegU32(sc, 4, static_cast(workSid)); + ps2_syscalls::SignalSema(rdram.data(), &sc, &runtime); + t.Equals(getRegS32(sc, 2), workSid, "T6: SignalSema returned workSid"); + } + + // Wait to confirm fiber did NOT consume it + std::this_thread::sleep_for(std::chrono::milliseconds(40)); + t.Equals(rdramSeq(rdram), 0u, "T6: fiber did not complete while suspended"); + t.Equals(getSemaCount(rdram, &runtime, workSid), 1, "T6: permit not consumed while suspended"); + + { + R5900Context sc{}; setRegU32(sc, 4, static_cast(tid)); + ps2_syscalls::ResumeThread(rdram.data(), &sc, &runtime); + t.Equals(getRegS32(sc, 2), KE_OK, "T6: ResumeThread returned KE_OK"); + } + + const bool woke = waitUntil([&](){ return rdramSeq(rdram) >= 1u; }, std::chrono::milliseconds(1000)); + t.IsTrue(woke, "T6: fiber woke after ResumeThread"); + { + int32_t ret = -9999; + std::memcpy(&ret, rdram.data() + kResultBase, 4); + t.Equals(ret, workSid, "T6: WaitSema returned workSid after resume"); + } + + waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + deleteSchedSema(rdram.data(), &runtime, workSid); + deleteSchedSema(rdram.data(), &runtime, doneSid); + }); + + // ------------------------------------------------------------------ + // Test 7: SignalSema/PollSema race — Mesa loop correctness + // ------------------------------------------------------------------ + tc.Run("SignalSema and PollSema race never over-consumes permits", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00660000u, &stepWaitSemaRecordSignal); + + const int32_t workSid = createSchedSema(rdram.data(), &runtime, 0, 1); + const int32_t doneSid = createSchedSema(rdram.data(), &runtime, 0, 1); + t.IsTrue(workSid > 0, "T7: work sema created"); t.IsTrue(doneSid > 0, "T7: done sema created"); + if (workSid <= 0 || doneSid <= 0) + { + deleteSchedSema(rdram.data(), &runtime, workSid); + deleteSchedSema(rdram.data(), &runtime, doneSid); + return; + } + + { + int32_t w = workSid, d = doneSid; + std::memcpy(rdram.data() + kSlotWorkSid, &w, 4); + std::memcpy(rdram.data() + kSlotDoneSid, &d, 4); + } + rdramSeqReset(rdram); + { + int32_t bad = -9999; + std::memcpy(rdram.data() + kResultBase, &bad, 4); + } + + const int32_t tid = startSchedWorker(rdram.data(), &runtime, 0x00660000u, 10, 0x00560000u, 0x2000u); + t.IsTrue(tid > 0, "T7: waiter fiber started"); + + const bool blocked = waitUntil([&]() + { + return getSemaWaiters(rdram.data(), &runtime, workSid) >= 1; + }, std::chrono::milliseconds(500)); + t.IsTrue(blocked, "T7: waiter blocked on workSid"); + + // Issue exactly ONE permit. Exactly one of {poller, waiter} must + // consume it; the other must get nothing. Counting consumers from + // observed results (not from the number of signals) is what lets this + // test actually catch a double-consume. + std::atomic pollOK{0}; + + std::thread poller([&]() + { + for (int i = 0; i < 200 && pollOK.load() == 0; ++i) + { + R5900Context sc{}; setRegU32(sc, 4, static_cast(workSid)); + ps2_syscalls::PollSema(rdram.data(), &sc, &runtime); + if (getRegS32(sc, 2) == workSid) + pollOK.store(1); + } + }); + + { + R5900Context sc{}; setRegU32(sc, 4, static_cast(workSid)); + ps2_syscalls::SignalSema(rdram.data(), &sc, &runtime); + } + + poller.join(); + + // If the poller stole the only permit, the waiter is still parked and + // this single signal will never wake it. Release it WITHOUT producing a + // permit so the test does not hang; it then records KE_RELEASE_WAIT + // (not workSid), keeping the consumer count honest. + // + // ReleaseWaitThread can legitimately return an error (KE_NOT_WAIT) if + // the waiter hasn't parked yet -- keep retrying until it reports + // KE_OK. WaitSema clears forceRelease exactly once, when it first + // transitions into THS_WAIT, and every retry checks-and-consumes + // forceRelease before re-parking rather than clearing it. So a + // single KE_OK from ReleaseWaitThread is sufficient proof the + // release landed -- do not call it again afterward. + if (pollOK.load() == 1) + { + const auto releaseDeadline = + std::chrono::steady_clock::now() + std::chrono::milliseconds(2000); + int32_t releaseRet = KE_NOT_WAIT; + while (releaseRet != KE_OK && + std::chrono::steady_clock::now() < releaseDeadline) + { + R5900Context sc{}; setRegU32(sc, 4, static_cast(tid)); + ps2_syscalls::ReleaseWaitThread(rdram.data(), &sc, &runtime); + releaseRet = getRegS32(sc, 2); + if (releaseRet != KE_OK) + std::this_thread::sleep_for(std::chrono::milliseconds(1)); + } + t.Equals(releaseRet, static_cast(KE_OK), "T7: ReleaseWaitThread eventually succeeds"); + } + + const bool waiterDone = waitUntil([&](){ return rdramSeq(rdram) >= 1u; }, std::chrono::milliseconds(2000)); + t.IsTrue(waiterDone, "T7: waiter fiber completed"); + + int32_t waiterRet = -9999; + std::memcpy(&waiterRet, rdram.data() + kResultBase, 4); + const int waiterOK = (waiterRet == workSid) ? 1 : 0; + + // The single permit must have been consumed by exactly one party. + const int consumers = pollOK.load() + waiterOK; + t.Equals(consumers, 1, "T7: exactly one consumer of the single permit (no double-consume)"); + t.IsTrue(getSemaCount(rdram, &runtime, workSid) >= 0, "T7: sema count never negative"); + t.Equals(getSemaCount(rdram, &runtime, workSid), 0, "T7: permit fully consumed, none left over"); + + waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + deleteSchedSema(rdram.data(), &runtime, workSid); + deleteSchedSema(rdram.data(), &runtime, doneSid); + }); + + // ------------------------------------------------------------------ + // Test 8: AsyncGuestScope blocks executor while held + // ------------------------------------------------------------------ + tc.Run("async_guest_begin blocks executor until async_guest_end", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00670000u, &stepProgressLoop); + + gStartedFlag.store(0u, std::memory_order_release); + gStopFlag.store(0u, std::memory_order_release); + gProgressCtr.store(0u, std::memory_order_release); + + ps2sched::async_guest_begin(); + + const int32_t tid = startSchedWorker(rdram.data(), &runtime, 0x00670000u, 10, 0x00570000u, 0x2000u); + t.IsTrue(tid > 0, "T8: fiber started"); + + // Fiber must NOT have started while host holds token + const uint32_t probe = gStartedFlag.load(std::memory_order_acquire); + t.Equals(probe, 0u, "T8: no fiber runs while host holds guest token"); + + ps2sched::async_guest_end(); + + const bool started = waitUntil([&]() + { + return gStartedFlag.load(std::memory_order_acquire) != 0u; + }, std::chrono::milliseconds(1000)); + t.IsTrue(started, "T8: fiber starts after async_guest_end"); + + gStopFlag.store(1u, std::memory_order_release); + waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + }); + + // ------------------------------------------------------------------ + // Test 9: AsyncGuestScope RAII releases token on exception + // ------------------------------------------------------------------ + tc.Run("AsyncGuestScope RAII releases token on exception path", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00680000u, &stepProgressLoop); + + gStartedFlag.store(0u, std::memory_order_release); + gStopFlag.store(0u, std::memory_order_release); + + struct TestGuestScope { + TestGuestScope() { ps2sched::async_guest_begin(); } + ~TestGuestScope() { ps2sched::async_guest_end(); } + }; + + try { + TestGuestScope g; + throw std::runtime_error("test exception"); + } catch (const std::exception &) { /* swallow */ } + + // Token should be released now + const int32_t tid = startSchedWorker(rdram.data(), &runtime, 0x00680000u, 10, 0x00580000u, 0x2000u); + t.IsTrue(tid > 0, "T9: fiber started"); + + const bool started = waitUntil([&]() + { + return gStartedFlag.load(std::memory_order_acquire) != 0u; + }, std::chrono::milliseconds(1000)); + t.IsTrue(started, "T9: fiber starts after RAII scope released token on exception"); + + gStopFlag.store(1u, std::memory_order_release); + waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + }); + + // ------------------------------------------------------------------ + // Test 10: WakeupThread from a foreign (host) thread wakes sleeping fiber. + // SleepThread only exits on an explicit WakeupThread permit (wakeupCount++); + // enqueue_external_wakeup_validated is a low-level scheduler primitive that + // does NOT set a permit, so it is NOT the right tool for waking SleepThread. + // This test verifies the correct path: a host thread calling WakeupThread + // (the ISR-style iWakeupThread equivalent) delivers the permit. + // ------------------------------------------------------------------ + tc.Run("WakeupThread from foreign host thread wakes sleeping fiber", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00690000u, &stepSleepRecordSignal); + + const int32_t doneSid = createSchedSema(rdram.data(), &runtime, 0, 1); + t.IsTrue(doneSid > 0, "T10: done sema created"); + if (doneSid <= 0) + { + return; + } + + { + int32_t d = doneSid; + std::memcpy(rdram.data() + kSlotDoneSid, &d, 4); + } + rdramSeqReset(rdram); + + const int32_t tid = startSchedWorker(rdram.data(), &runtime, 0x00690000u, 10, 0x00590000u, 0x2000u); + t.IsTrue(tid > 0, "T10: fiber started"); + + const bool sleeping = waitUntil([&]() + { + int32_t status = 0, wt = 0; + getThreadStatus(rdram, runtime, tid, status, wt); + return status == kTHSWait && wt == kTSWSleep; + }, std::chrono::milliseconds(500)); + t.IsTrue(sleeping, "T10: fiber is in SleepThread"); + + // Call WakeupThread from a foreign host thread (simulates iWakeupThread + // from an ISR). This sets wakeupCount++ and unblocks the fiber. + std::thread foreign([&]() + { + R5900Context sc{}; + setRegU32(sc, 4, static_cast(tid)); + ps2_syscalls::WakeupThread(rdram.data(), &sc, &runtime); + }); + foreign.join(); + + const bool woke = waitUntil([&](){ return rdramSeq(rdram) >= 1u; }, std::chrono::milliseconds(1000)); + t.IsTrue(woke, "T10: fiber woke from WakeupThread"); + { + int32_t log0 = 0; + std::memcpy(&log0, rdram.data() + kRunLog, 4); + t.Equals(log0, tid, "T10: correct fiber woke"); + } + + waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + deleteSchedSema(rdram.data(), &runtime, doneSid); + }); + + // ------------------------------------------------------------------ + // Test 11: Suspended fiber does NOT wake on a valid external wakeup + // ------------------------------------------------------------------ + tc.Run("suspended sleeping fiber ignores enqueue_external_wakeup_validated", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x006A0000u, &stepSleepRecordSignalPublishToken); + + const int32_t doneSid = createSchedSema(rdram.data(), &runtime, 0, 1); + t.IsTrue(doneSid > 0, "T11: done sema created"); + if (doneSid <= 0) + { + return; + } + + { + int32_t d = doneSid; + std::memcpy(rdram.data() + kSlotDoneSid, &d, 4); + } + rdramSeqReset(rdram); + gT11TokenLo.store(0u, std::memory_order_release); + gT11TokenHi.store(0u, std::memory_order_release); + + const int32_t tid = startSchedWorker(rdram.data(), &runtime, 0x006A0000u, 10, 0x005A0000u, 0x2000u); + t.IsTrue(tid > 0, "T11: fiber started"); + + const bool sleeping = waitUntil([&]() + { + int32_t status = 0, wt = 0; + getThreadStatus(rdram, runtime, tid, status, wt); + return status == kTHSWait && wt == kTSWSleep; + }, std::chrono::milliseconds(500)); + t.IsTrue(sleeping, "T11: fiber is sleeping"); + + const bool tokenPublished = waitUntil([&]() + { + return gT11TokenHi.load(std::memory_order_acquire) != 0u || + gT11TokenLo.load(std::memory_order_acquire) != 0u; + }, std::chrono::milliseconds(500)); + t.IsTrue(tokenPublished, "T11: fiber published its real current_fiber_token()"); + const uint64_t token = (static_cast(gT11TokenHi.load(std::memory_order_acquire)) << 32) | + static_cast(gT11TokenLo.load(std::memory_order_acquire)); + + { + R5900Context sc{}; setRegU32(sc, 4, static_cast(tid)); + ps2_syscalls::SuspendThread(rdram.data(), &sc, &runtime); + t.Equals(getRegS32(sc, 2), KE_OK, "T11: SuspendThread returned KE_OK"); + } + + const bool isSuspended = waitUntil([&]() + { + int32_t status = 0, wt = 0; + getThreadStatus(rdram, runtime, tid, status, wt); + return status == kTHSWaitSuspend; + }, std::chrono::milliseconds(500)); + t.IsTrue(isSuspended, "T11: fiber reached THS_WAITSUSPEND"); + + std::thread foreign([&](){ ps2sched::enqueue_external_wakeup_validated(tid, token); }); + foreign.join(); + + std::this_thread::sleep_for(std::chrono::milliseconds(40)); + t.Equals(rdramSeq(rdram), 0u, "T11: suspended fiber did not wake from external wakeup"); + { + int32_t status = 0, wt = 0; + getThreadStatus(rdram, runtime, tid, status, wt); + t.IsTrue(status == kTHSWaitSuspend || status == kTHSSuspend, + "T11: fiber remains suspended"); + } + + { + R5900Context sc{}; setRegU32(sc, 4, static_cast(tid)); + ps2_syscalls::ResumeThread(rdram.data(), &sc, &runtime); + t.Equals(getRegS32(sc, 2), KE_OK, "T11: ResumeThread returned KE_OK"); + } + + // After ResumeThread the fiber should RE-PARK inside SleepThread + // (no wakeupCount permit exists). Verify it did NOT exit. + std::this_thread::sleep_for(std::chrono::milliseconds(60)); + t.Equals(rdramSeq(rdram), 0u, + "T11: fiber did not exit SleepThread after ResumeThread (no spurious wake)"); + { + int32_t status = 0, wt = 0; + getThreadStatus(rdram, runtime, tid, status, wt); + t.IsTrue(status == kTHSWait && wt == kTSWSleep, + "T11: fiber re-parked in THS_SLEEP after ResumeThread"); + } + + // A genuine WakeupThread must now release the fiber. + { + R5900Context sc{}; setRegU32(sc, 4, static_cast(tid)); + ps2_syscalls::WakeupThread(rdram.data(), &sc, &runtime); + t.Equals(getRegS32(sc, 2), KE_OK, "T11: WakeupThread returned KE_OK"); + } + + const bool woke = waitUntil([&](){ return rdramSeq(rdram) >= 1u; }, std::chrono::milliseconds(1000)); + t.IsTrue(woke, "T11: fiber woke after WakeupThread"); + + waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + deleteSchedSema(rdram.data(), &runtime, doneSid); + }); + + // ------------------------------------------------------------------ + // Test 12: WaitSema basic round-trip + // ------------------------------------------------------------------ + tc.Run("WaitSema returns sid after SignalSema and consumes exactly one permit", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x006B0000u, &stepWaitSemaRecordSignal); + + const int32_t workSid = createSchedSema(rdram.data(), &runtime, 0, 1); + const int32_t doneSid = createSchedSema(rdram.data(), &runtime, 0, 1); + t.IsTrue(workSid > 0, "T12: work sema created"); t.IsTrue(doneSid > 0, "T12: done sema created"); + if (workSid <= 0 || doneSid <= 0) + { + deleteSchedSema(rdram.data(), &runtime, workSid); + deleteSchedSema(rdram.data(), &runtime, doneSid); + return; + } + + { + int32_t w = workSid, d = doneSid; + std::memcpy(rdram.data() + kSlotWorkSid, &w, 4); + std::memcpy(rdram.data() + kSlotDoneSid, &d, 4); + } + rdramSeqReset(rdram); + { + int32_t bad = -9999; + std::memcpy(rdram.data() + kResultBase, &bad, 4); + } + + const int32_t tid = startSchedWorker(rdram.data(), &runtime, 0x006B0000u, 10, 0x005B0000u, 0x2000u); + t.IsTrue(tid > 0, "T12: fiber started"); + + const bool blocked = waitUntil([&]() + { + return getSemaWaiters(rdram.data(), &runtime, workSid) >= 1; + }, std::chrono::milliseconds(500)); + t.IsTrue(blocked, "T12: fiber blocked on workSid"); + + { + R5900Context sc{}; setRegU32(sc, 4, static_cast(workSid)); + ps2_syscalls::SignalSema(rdram.data(), &sc, &runtime); + } + + const bool done = waitUntil([&](){ return rdramSeq(rdram) >= 1u; }, std::chrono::milliseconds(1000)); + t.IsTrue(done, "T12: fiber completed"); + + { + int32_t ret = -9999; + std::memcpy(&ret, rdram.data() + kResultBase, 4); + t.Equals(ret, workSid, "T12: WaitSema returned workSid after signal"); + } + t.Equals(getSemaCount(rdram, &runtime, workSid), 0, "T12: permit consumed"); + + waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + deleteSchedSema(rdram.data(), &runtime, workSid); + deleteSchedSema(rdram.data(), &runtime, doneSid); + }); + + // ------------------------------------------------------------------ + // Test 13: SleepThread / WakeupThread round-trip + // ------------------------------------------------------------------ + tc.Run("SleepThread blocks fiber and WakeupThread wakes it", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x006C0000u, &stepSleepRecordSignal); + + const int32_t doneSid = createSchedSema(rdram.data(), &runtime, 0, 1); + t.IsTrue(doneSid > 0, "T13: done sema created"); + if (doneSid <= 0) + { + return; + } + + { + int32_t d = doneSid; + std::memcpy(rdram.data() + kSlotDoneSid, &d, 4); + } + rdramSeqReset(rdram); + + const int32_t tid = startSchedWorker(rdram.data(), &runtime, 0x006C0000u, 10, 0x005C0000u, 0x2000u); + t.IsTrue(tid > 0, "T13: fiber started"); + + // 2000ms (not the more typical 500ms used for similar checks elsewhere + // in this file): this specifically waits for a *freshly created* fiber + // pool thread to be scheduled by the OS, enter SleepThread, and publish + // its status -- purely OS thread-startup latency, unrelated to what the + // test verifies below. WakeupThread's wakeupCount++ is safe to deliver + // at any point after StartThread returns (SleepThread's Mesa loop + // consumes a pending count immediately even if it arrives before the + // thread parks), so a generous bound here only protects against slow + // scheduling, not a race in the wakeup path itself. + const bool sleeping = waitUntil([&]() + { + int32_t status = 0, wt = 0; + getThreadStatus(rdram, runtime, tid, status, wt); + return status == kTHSWait && wt == kTSWSleep; + }, std::chrono::milliseconds(2000)); + t.IsTrue(sleeping, "T13: fiber is in SleepThread"); + + { + R5900Context sc{}; setRegU32(sc, 4, static_cast(tid)); + ps2_syscalls::WakeupThread(rdram.data(), &sc, &runtime); + t.Equals(getRegS32(sc, 2), KE_OK, "T13: WakeupThread returned KE_OK"); + } + + const bool woke = waitUntil([&](){ return rdramSeq(rdram) >= 1u; }, std::chrono::milliseconds(1000)); + t.IsTrue(woke, "T13: fiber woke after WakeupThread"); + { + int32_t log0 = 0; + std::memcpy(&log0, rdram.data() + kRunLog, 4); + t.Equals(log0, tid, "T13: correct fiber woke"); + } + + waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + deleteSchedSema(rdram.data(), &runtime, doneSid); + }); + + // ------------------------------------------------------------------ + // Test 14: TerminateThread(other) via killer fiber + // ------------------------------------------------------------------ + tc.Run("TerminateThread from killer fiber terminates blocked worker", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x006D0000u, &stepBlockForever); + runtime.registerFunction(0x006D8000u, &stepTerminateTarget); + + const int32_t workSid = createSchedSema(rdram.data(), &runtime, 0, 1); + t.IsTrue(workSid > 0, "T14: work sema created"); + if (workSid <= 0) + { + return; + } + + { + int32_t w = workSid; + std::memcpy(rdram.data() + kSlotWorkSid, &w, 4); + } + rdramSeqReset(rdram); + + const int32_t workerTid = startSchedWorker(rdram.data(), &runtime, 0x006D0000u, 10, 0x005D0000u, 0x2000u); + t.IsTrue(workerTid > 0, "T14: worker started"); + + const bool blocked = waitUntil([&]() + { + return getSemaWaiters(rdram.data(), &runtime, workSid) >= 1; + }, std::chrono::milliseconds(500)); + t.IsTrue(blocked, "T14: worker blocked on workSid"); + + { + int32_t v = workerTid; + std::memcpy(rdram.data() + kSlotTidParam, &v, 4); + } + + const int32_t killerTid = startSchedWorker(rdram.data(), &runtime, 0x006D8000u, 5, 0x005D2000u, 0x2000u); + t.IsTrue(killerTid > 0, "T14: killer started"); + + const bool allDone = waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(3000)); + t.IsTrue(allDone, "T14: both worker and killer finished"); + t.IsTrue(rdramSeq(rdram) >= 1u, "T14: killer logged after join_fiber returned"); + + deleteSchedSema(rdram.data(), &runtime, workSid); + }); + + // ------------------------------------------------------------------ + // Test 15: scheduler_shutdown with blocked fibers + // ------------------------------------------------------------------ + tc.Run("scheduler_shutdown terminates all blocked fibers within 5s", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x006E0000u, &stepBlockForever); + + const int32_t workSid = createSchedSema(rdram.data(), &runtime, 0, 3); + t.IsTrue(workSid > 0, "T15: work sema created"); + if (workSid <= 0) + { + return; + } + + { + int32_t w = workSid; + std::memcpy(rdram.data() + kSlotWorkSid, &w, 4); + } + + const int32_t tid1 = startSchedWorker(rdram.data(), &runtime, 0x006E0000u, 10, 0x005E0000u, 0x2000u); + const int32_t tid2 = startSchedWorker(rdram.data(), &runtime, 0x006E0000u, 10, 0x005E2000u, 0x2000u); + const int32_t tid3 = startSchedWorker(rdram.data(), &runtime, 0x006E0000u, 10, 0x005E4000u, 0x2000u); + t.IsTrue(tid1 > 0, "T15: fiber 1 started"); + t.IsTrue(tid2 > 0, "T15: fiber 2 started"); + t.IsTrue(tid3 > 0, "T15: fiber 3 started"); + + const bool allBlocked = waitUntil([&]() + { + return getSemaWaiters(rdram.data(), &runtime, workSid) >= 3; + }, std::chrono::milliseconds(1000)); + t.IsTrue(allBlocked, "T15: all 3 blocked on workSid"); + + const auto t0 = std::chrono::steady_clock::now(); + ps2sched::scheduler_shutdown(); + const auto elapsed = std::chrono::steady_clock::now() - t0; + + t.IsTrue(elapsed < std::chrono::seconds(5), "T15: scheduler_shutdown returned within 5s"); + t.Equals(g_activeThreads.load(), 0, "T15: all fibers terminated"); + + // Do NOT call scheduler_shutdown again + deleteSchedSema(rdram.data(), &runtime, workSid); + }); + + // ------------------------------------------------------------------ + // Test 16: WaitEventFlag AND-mode + // ------------------------------------------------------------------ + tc.Run("WaitEventFlag AND-mode re-blocks on partial bit set and completes on full set", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x006F0000u, &stepWaitEvfAndRecord); + + const int32_t eid = createSchedEvf(rdram, &runtime, 0u, 0u); + const int32_t doneSid = createSchedSema(rdram.data(), &runtime, 0, 1); + t.IsTrue(eid > 0, "T16: event flag created"); t.IsTrue(doneSid > 0, "T16: done sema created"); + if (eid <= 0 || doneSid <= 0) + { + if (eid > 0) deleteSchedEvf(rdram, &runtime, eid); + deleteSchedSema(rdram.data(), &runtime, doneSid); + return; + } + + { + int32_t e = eid, d = doneSid; + std::memcpy(rdram.data() + kSlotEid, &e, 4); + std::memcpy(rdram.data() + kSlotDoneSid, &d, 4); + } + rdramSeqReset(rdram); + { + int32_t bad = -9999; + std::memcpy(rdram.data() + kResultBase, &bad, 4); + } + rdramWrite32(rdram, kSlotResBits, 0u); + + const int32_t tid = startSchedWorker(rdram.data(), &runtime, 0x006F0000u, 10, 0x005F0000u, 0x2000u); + t.IsTrue(tid > 0, "T16: fiber started"); + + const bool waiting = waitUntil([&]() + { + return getEvfNumThreads(rdram, runtime, eid) >= 1; + }, std::chrono::milliseconds(500)); + t.IsTrue(waiting, "T16: fiber registered as EventFlag waiter"); + + // Partial set — AND not satisfied + { + R5900Context sc{}; setRegU32(sc, 4, static_cast(eid)); setRegU32(sc, 5, 0x1u); + ps2_syscalls::SetEventFlag(rdram.data(), &sc, &runtime); + } + + // Fiber may briefly dequeue and re-block; wait for it to re-block + std::this_thread::sleep_for(std::chrono::milliseconds(30)); + const bool reBlocked = waitUntil([&]() + { + return getEvfNumThreads(rdram, runtime, eid) >= 1; + }, std::chrono::milliseconds(500)); + t.IsTrue(reBlocked, "T16: fiber re-blocked after partial bit set"); + t.Equals(rdramSeq(rdram), 0u, "T16: fiber did not complete on partial bit"); + + // Complete set + { + R5900Context sc{}; setRegU32(sc, 4, static_cast(eid)); setRegU32(sc, 5, 0x2u); + ps2_syscalls::SetEventFlag(rdram.data(), &sc, &runtime); + } + + const bool completed = waitUntil([&](){ return rdramSeq(rdram) >= 1u; }, std::chrono::milliseconds(1000)); + t.IsTrue(completed, "T16: fiber completed after all AND bits set"); + + { + int32_t ret = -9999; + std::memcpy(&ret, rdram.data() + kResultBase, 4); + t.Equals(ret, KE_OK, "T16: WaitEventFlag returned KE_OK"); + } + { + uint32_t resBits = 0; + std::memcpy(&resBits, rdram.data() + kSlotResBits, 4); + t.IsTrue((resBits & 0x3u) == 0x3u, "T16: result bits include all waited bits"); + } + + waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + deleteSchedEvf(rdram, &runtime, eid); + deleteSchedSema(rdram.data(), &runtime, doneSid); + }); + + // ------------------------------------------------------------------ + // Test 17: DeleteEventFlag wakes all waiters with KE_WAIT_DELETE + // ------------------------------------------------------------------ + tc.Run("DeleteEventFlag wakes all waiters with KE_WAIT_DELETE", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00700000u, &stepWaitEvfAndRecord); + + // EA_MULTI = 0x2 allows multiple waiters + const int32_t eid = createSchedEvf(rdram, &runtime, 0x2u, 0u); + const int32_t doneSid = createSchedSema(rdram.data(), &runtime, 0, 3); + t.IsTrue(eid > 0, "T17: event flag created"); t.IsTrue(doneSid > 0, "T17: done sema created"); + if (eid <= 0 || doneSid <= 0) + { + if (eid > 0) deleteSchedEvf(rdram, &runtime, eid); + deleteSchedSema(rdram.data(), &runtime, doneSid); + return; + } + + { + int32_t e = eid, d = doneSid; + std::memcpy(rdram.data() + kSlotEid, &e, 4); + std::memcpy(rdram.data() + kSlotDoneSid, &d, 4); + } + rdramSeqReset(rdram); + for (int i = 0; i < 3; ++i) + { + int32_t bad = -9999; + std::memcpy(rdram.data() + kResultBase + static_cast(i * 4), &bad, 4); + } + + const int32_t tid1 = startSchedWorker(rdram.data(), &runtime, 0x00700000u, 10, 0x00480000u, 0x2000u); + const int32_t tid2 = startSchedWorker(rdram.data(), &runtime, 0x00700000u, 10, 0x00482000u, 0x2000u); + const int32_t tid3 = startSchedWorker(rdram.data(), &runtime, 0x00700000u, 10, 0x00484000u, 0x2000u); + t.IsTrue(tid1 > 0, "T17: fiber 1 started"); t.IsTrue(tid2 > 0, "T17: fiber 2 started"); t.IsTrue(tid3 > 0, "T17: fiber 3 started"); + + const bool allWaiting = waitUntil([&]() + { + return getEvfNumThreads(rdram, runtime, eid) >= 3; + }, std::chrono::milliseconds(1000)); + t.IsTrue(allWaiting, "T17: all 3 fibers waiting on event flag"); + + deleteSchedEvf(rdram, &runtime, eid); + + const bool allDone = waitUntil([&](){ return rdramSeq(rdram) >= 3u; }, std::chrono::milliseconds(2000)); + t.IsTrue(allDone, "T17: all 3 fibers completed after DeleteEventFlag"); + + for (int i = 0; i < 3; ++i) + { + int32_t ret = -9999; + std::memcpy(&ret, rdram.data() + kResultBase + static_cast(i * 4), 4); + t.Equals(ret, KE_WAIT_DELETE, + std::string("T17: fiber ") + std::to_string(i) + " received KE_WAIT_DELETE"); + } + + // eid is now invalid + { + R5900Context sc{}; setRegU32(sc, 4, static_cast(eid)); setRegU32(sc, 5, kReferScratch); + ps2_syscalls::ReferEventFlagStatus(rdram.data(), &sc, &runtime); + t.IsTrue(getRegS32(sc, 2) < 0, "T17: deleted evf returns error on Refer"); + } + + waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(2000)); + deleteSchedSema(rdram.data(), &runtime, doneSid); + }); + + // ------------------------------------------------------------------ + // Test 18: DeleteSema wakes all blocked waiters with KE_WAIT_DELETE + // ------------------------------------------------------------------ + tc.Run("DeleteSema wakes all blocked waiters with KE_WAIT_DELETE", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00710000u, &stepWaitSemaRecordSignal); + + const int32_t workSid = createSchedSema(rdram.data(), &runtime, 0, 1); + const int32_t doneSid = createSchedSema(rdram.data(), &runtime, 0, 3); + t.IsTrue(workSid > 0, "T18: work sema created"); t.IsTrue(doneSid > 0, "T18: done sema created"); + if (workSid <= 0 || doneSid <= 0) + { + deleteSchedSema(rdram.data(), &runtime, workSid); + deleteSchedSema(rdram.data(), &runtime, doneSid); + return; + } + + { + int32_t w = workSid, d = doneSid; + std::memcpy(rdram.data() + kSlotWorkSid, &w, 4); + std::memcpy(rdram.data() + kSlotDoneSid, &d, 4); + } + rdramSeqReset(rdram); + for (int i = 0; i < 3; ++i) + { + int32_t bad = -9999; + std::memcpy(rdram.data() + kResultBase + static_cast(i * 4), &bad, 4); + } + + const int32_t tid1 = startSchedWorker(rdram.data(), &runtime, 0x00710000u, 10, 0x004A0000u, 0x2000u); + const int32_t tid2 = startSchedWorker(rdram.data(), &runtime, 0x00710000u, 10, 0x004A2000u, 0x2000u); + const int32_t tid3 = startSchedWorker(rdram.data(), &runtime, 0x00710000u, 10, 0x004A4000u, 0x2000u); + t.IsTrue(tid1 > 0, "T18: fiber 1 started"); t.IsTrue(tid2 > 0, "T18: fiber 2 started"); t.IsTrue(tid3 > 0, "T18: fiber 3 started"); + + const bool allBlocked = waitUntil([&]() + { + return getSemaWaiters(rdram.data(), &runtime, workSid) >= 3; + }, std::chrono::milliseconds(1000)); + t.IsTrue(allBlocked, "T18: all 3 blocked on workSid"); + + { + R5900Context sc{}; setRegU32(sc, 4, static_cast(workSid)); + ps2_syscalls::DeleteSema(rdram.data(), &sc, &runtime); + t.Equals(getRegS32(sc, 2), workSid, "T18: DeleteSema returned workSid"); + } + + const bool allDone = waitUntil([&](){ return rdramSeq(rdram) >= 3u; }, std::chrono::milliseconds(2000)); + t.IsTrue(allDone, "T18: all 3 fibers completed after DeleteSema"); + + for (int i = 0; i < 3; ++i) + { + int32_t ret = -9999; + std::memcpy(&ret, rdram.data() + kResultBase + static_cast(i * 4), 4); + t.Equals(ret, KE_WAIT_DELETE, + std::string("T18: fiber ") + std::to_string(i) + " received KE_WAIT_DELETE"); + } + + // workSid is now invalid + { + R5900Context sc{}; setRegU32(sc, 4, static_cast(workSid)); + ps2_syscalls::PollSema(rdram.data(), &sc, &runtime); + t.IsTrue(getRegS32(sc, 2) < 0, "T18: deleted sema returns error on Poll"); + } + + waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(2000)); + deleteSchedSema(rdram.data(), &runtime, doneSid); + }); + + // ------------------------------------------------------------------ + // Test 19: Resource cleanup — g_activeThreads drains to 0 after each exit + // ------------------------------------------------------------------ + tc.Run("g_activeThreads drains to 0 after each fiber exits", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00720000u, &stepLogAndExit); + + rdramSeqReset(rdram); + std::memset(rdram.data() + kRunLog, 0, 32u); + + for (int i = 0; i < 4; ++i) + { + const int before = g_activeThreads.load(); + t.Equals(before, 0, std::string("T19: no leaked threads before fiber ") + std::to_string(i)); + + const uint32_t stackAddr = 0x004B0000u + static_cast(i) * 0x2000u; + const int32_t tid = startSchedWorker(rdram.data(), &runtime, 0x00720000u, 10, stackAddr, 0x2000u); + t.IsTrue(tid > 0, std::string("T19: fiber ") + std::to_string(i) + " started"); + + const bool drained = waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + t.IsTrue(drained, std::string("T19: fiber ") + std::to_string(i) + " exited and g_activeThreads returned to 0"); + } + + t.Equals(rdramSeq(rdram), 4u, "T19: all 4 fibers logged"); + + }); + + }); // MiniTest::Case +} + +// --------------------------------------------------------------------------- +// Scheduler race tests — park/wake, borrowed-worker, alarm shutdown, exit-hook re-entry, nested suspend. +// --------------------------------------------------------------------------- + +// Forward-declare alarm worker lifecycle functions from Sync.h (internal header). +// These are defined in the runtime TU; declared here to avoid including Sync.h. +namespace ps2_syscalls +{ + void ensureAlarmWorkerRunning(); + void stopAlarmWorker(); +} + +namespace +{ + // ----------------------------------------------------------------------- + // New RDRAM slot constants for the race suite (0x4300-0x43FF range) + // ----------------------------------------------------------------------- + static constexpr uint32_t kRSlotWorkSid = 0x00004300u; // int32 work sema id + static constexpr uint32_t kRSlotDoneSid = 0x00004304u; // int32 done sema id + static constexpr uint32_t kRSlotProgress = 0x00004308u; // uint32 progress counter + static constexpr uint32_t kRSlotStop = 0x0000430Cu; // uint32 stop flag + static constexpr uint32_t kRSlotStarted = 0x00004310u; // uint32 started flag + static constexpr uint32_t kRSlotTidParam = 0x00004314u; // int32 a tid passed to a fiber + static constexpr uint32_t kRSlotExitRan = 0x00004318u; // uint32 set by exit handler + static constexpr uint32_t kRSlotResult = 0x0000431Cu; // int32 a syscall return value + static constexpr uint32_t kRSlotAlarmFired = 0x00004320u; // uint32 set by alarm callback + + // R3: written by the alarm worker's callback thread, polled by the host + // test thread (see gSeq/... rationale above). + static std::atomic gRAlarmFired{0}; + static constexpr uint32_t kRSlotEntryReached = 0x00004324u; // uint32 fiber body reached + + // Host test thread <-> guest fiber thread mailboxes for the R-suite + // (same rationale as gSeq/gStartedFlag/... above: plain rdram polling + // across this boundary is a data race under TSan even though only one + // fiber runs at a time). + static std::atomic gRProgress{0}; + static std::atomic gREntryReached{0}; + static std::atomic gRExitRan{0}; + static std::atomic gRStarted{0}; + // R1's own current_fiber_token(), published once at start so the wake-storm + // thread can call enqueue_external_wakeup_validated() with a REAL token + // (the fiber's tid/generation never change across its WaitSema loop). + static std::atomic gRTokenLo{0}; + static std::atomic gRTokenHi{0}; + + // Entry / stack base constants + static constexpr uint32_t kREntryBase = 0x00730000u; + static constexpr uint32_t kRStackBase = 0x004C0000u; + static constexpr uint32_t kRStackSize = 0x2000u; + + // ----------------------------------------------------------------------- + // Step function: stepWaitSemaLoopRace + // Loops 500x: each iteration WaitSema(workSid) then increments kRSlotProgress. + // After the loop, signals doneSid once and exits. + // ----------------------------------------------------------------------- + static void stepWaitSemaLoopRace(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + int32_t workSid = 0, doneSid = 0; + std::memcpy(&workSid, rdram + kRSlotWorkSid, 4); + std::memcpy(&doneSid, rdram + kRSlotDoneSid, 4); + + const uint64_t tok = ps2sched::current_fiber_token(); + gRTokenLo.store(static_cast(tok & 0xFFFFFFFFu), std::memory_order_relaxed); + gRTokenHi.store(static_cast(tok >> 32u), std::memory_order_release); + + constexpr int kRounds = 500; + for (int i = 0; i < kRounds; ++i) + { + R5900Context sc{}; + setRegU32(sc, 4, static_cast(workSid)); + ps2_syscalls::WaitSema(rdram, &sc, runtime); + // On terminate, WaitSema unwinds via ThreadExitException — acceptable. + gRProgress.fetch_add(1u, std::memory_order_release); + } + + if (doneSid > 0) + { + R5900Context sc2{}; + setRegU32(sc2, 4, static_cast(doneSid)); + ps2_syscalls::SignalSema(rdram, &sc2, runtime); + } + ctx->pc = 0u; + } + + // ----------------------------------------------------------------------- + // Step function: stepMarkEntryAndExit (for R2) + // Sets kRSlotEntryReached=1 and exits immediately. + // ----------------------------------------------------------------------- + static void stepMarkEntryAndExit(uint8_t *rdram, R5900Context *ctx, PS2Runtime * /*runtime*/) + { + gREntryReached.store(1u, std::memory_order_release); + ctx->pc = 0u; + } + + // ----------------------------------------------------------------------- + // Step function: stepAlarmCallbackMark (for R3) + // Called as alarm callback; sets kRSlotAlarmFired=1. + // ----------------------------------------------------------------------- + static void stepAlarmCallbackMark(uint8_t *rdram, R5900Context *ctx, PS2Runtime * /*runtime*/) + { + gRAlarmFired.store(1u, std::memory_order_release); + ctx->pc = 0u; + } + + // ----------------------------------------------------------------------- + // Step functions for the exit-hook re-entrancy test. + // + // stepExitHandlerBlocking: the exit handler — runs as guest code on the + // same fiber during exit. Calls SleepThread to park while Exiting. + // After WakeupThread delivers a wakeup, marks kRSlotExitRan=1 and returns. + // ----------------------------------------------------------------------- + static void stepExitHandlerBlocking(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + R5900Context sc{}; + ps2_syscalls::SleepThread(rdram, &sc, runtime); + gRExitRan.store(1u, std::memory_order_release); + ctx->pc = 0u; + } + + // stepRegisterExitHandlerThenExit: the fiber body. + // Registers stepExitHandlerBlocking as an exit handler (at kREntryBase+0x3100 + // == 0x00733100), signals kRSlotStarted=1, then returns so the fiber begins + // exiting and on_fiber_exit runs the handler. + static void stepRegisterExitHandlerThenExit(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + // Handler entry must match what the test registered via registerFunction(). + static constexpr uint32_t kHandlerEntry = 0x00733100u; + R5900Context rc{}; + setRegU32(rc, 4, kHandlerEntry); // func + setRegU32(rc, 5, 0u); // arg + ps2_syscalls::RegisterExitHandler(rdram, &rc, runtime); + + gRStarted.store(1u, std::memory_order_release); + ctx->pc = 0u; + } + +} // anonymous namespace + +void register_scheduler_race_tests() +{ + MiniTest::Case("SchedulerRace", [](TestCase &tc) + { + // ------------------------------------------------------------------ + // R1: Concurrent wakeups while a fiber parks must never lose a wakeup. + // ------------------------------------------------------------------ + tc.Run("R1: concurrent wakeups while a fiber parks never lose a wakeup", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00730000u, &stepWaitSemaLoopRace); + + const int32_t workSid = createSchedSema(rdram.data(), &runtime, 0, 1); + const int32_t doneSid = createSchedSema(rdram.data(), &runtime, 0, 1); + t.IsTrue(workSid > 0 && doneSid > 0, "R1: semas created"); + if (workSid <= 0 || doneSid <= 0) + { + deleteSchedSema(rdram.data(), &runtime, workSid); + deleteSchedSema(rdram.data(), &runtime, doneSid); + return; + } + + std::memcpy(rdram.data() + kRSlotWorkSid, &workSid, 4); + std::memcpy(rdram.data() + kRSlotDoneSid, &doneSid, 4); + gRProgress.store(0u, std::memory_order_release); + gRTokenLo.store(0u, std::memory_order_release); + gRTokenHi.store(0u, std::memory_order_release); + + const int32_t tid = startSchedWorker(rdram.data(), &runtime, + 0x00730000u, 10, kRStackBase, kRStackSize); + t.IsTrue(tid > 0, "R1: worker started"); + + // Wait until the fiber is parked on workSid at least once. + const bool blocked = waitUntil([&]() + { + return getSemaWaiters(rdram.data(), &runtime, workSid) >= 1; + }, std::chrono::milliseconds(1000)); + t.IsTrue(blocked, "R1: worker parked on workSid"); + + const bool tokenPublished = waitUntil([&]() + { + return gRTokenHi.load(std::memory_order_acquire) != 0u || + gRTokenLo.load(std::memory_order_acquire) != 0u; + }, std::chrono::milliseconds(500)); + t.IsTrue(tokenPublished, "R1: worker published its real current_fiber_token()"); + const uint64_t token = (static_cast(gRTokenHi.load(std::memory_order_acquire)) << 32) | + static_cast(gRTokenLo.load(std::memory_order_acquire)); + + constexpr int kRounds = 500; + std::atomic stopWakers{false}; + + // (b) mid-park hammer: redundant external wakeups to maximise park-window race rate. + std::thread wakeStorm([&]() + { + while (!stopWakers.load(std::memory_order_acquire)) + { + ps2sched::enqueue_external_wakeup_validated(tid, token); // safe no-op unless genuinely Blocked + } + }); + + // (a) signaler: deliver exactly kRounds permits as fast as possible. + // WorkSid has max==1, so retry on KE_SEMA_OVF until the permit lands. + std::thread signaler([&]() + { + for (int i = 0; i < kRounds; ++i) + { + R5900Context sc{}; + setRegU32(sc, 4, static_cast(workSid)); + ps2_syscalls::SignalSema(rdram.data(), &sc, &runtime); + while (getRegS32(sc, 2) != workSid) + { + std::this_thread::yield(); + R5900Context sc2{}; + setRegU32(sc2, 4, static_cast(workSid)); + ps2_syscalls::SignalSema(rdram.data(), &sc2, &runtime); + sc = sc2; + } + } + }); + + // Pass condition: progress reaches kRounds (no lost wakeup). + const bool reached = waitUntil([&]() + { + return gRProgress.load(std::memory_order_acquire) >= static_cast(kRounds); + }, std::chrono::milliseconds(5000)); + + stopWakers.store(true, std::memory_order_release); + if (signaler.joinable()) signaler.join(); + if (wakeStorm.joinable()) wakeStorm.join(); + + t.IsTrue(reached, "R1: fiber completed all 500 park/wake rounds (no lost wakeup)"); + + // Drain: fiber signals doneSid and exits. + const bool finished = waitUntil([&]() + { + return g_activeThreads.load(std::memory_order_acquire) == 0; + }, std::chrono::milliseconds(2000)); + t.IsTrue(finished, "R1: worker fiber exits cleanly (g_activeThreads==0)"); + + deleteSchedSema(rdram.data(), &runtime, workSid); + deleteSchedSema(rdram.data(), &runtime, doneSid); + }); + + // ------------------------------------------------------------------ + // R2: blocking syscall from non-fiber AsyncGuestScope does not crash + // ------------------------------------------------------------------ + tc.Run("R2: blocking syscall from non-fiber AsyncGuestScope does not crash", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00731000u, &stepMarkEntryAndExit); + + const int32_t blockSid = createSchedSema(rdram.data(), &runtime, 0, 1); + t.IsTrue(blockSid > 0, "R2: sema created"); + + std::atomic waitRet{-9999}; + std::atomic waitThrew{false}; + std::atomic waitReturned{false}; + + // Run the non-fiber blocking WaitSema on a background host thread so the + // main thread can release it. block_current() is a no-op for non-fiber + // contexts, so WaitSema's Mesa re-check loop spins until count > 0. + std::thread guestBorrow([&]() + { + try + { + ps2sched::async_guest_begin(); + R5900Context sc{}; + setRegU32(sc, 4, static_cast(blockSid)); + ps2_syscalls::WaitSema(rdram.data(), &sc, &runtime); + waitRet.store(getRegS32(sc, 2), std::memory_order_release); + ps2sched::async_guest_end(); + } + catch (...) + { + waitThrew.store(true, std::memory_order_release); + ps2sched::async_guest_end(); + } + waitReturned.store(true, std::memory_order_release); + }); + + // Give the borrow thread a moment to enter WaitSema's spin loop, + // then feed it a permit so the Mesa re-check returns blockSid. + std::this_thread::sleep_for(std::chrono::milliseconds(20)); + { + R5900Context sc{}; + setRegU32(sc, 4, static_cast(blockSid)); + ps2_syscalls::SignalSema(rdram.data(), &sc, &runtime); + } + + const bool returned = waitUntil([&]() + { + return waitReturned.load(std::memory_order_acquire); + }, std::chrono::milliseconds(2000)); + + if (guestBorrow.joinable()) guestBorrow.join(); + + t.IsTrue(returned, "R2: non-fiber WaitSema returned (no hang, no crash)"); + t.IsFalse(waitThrew.load(std::memory_order_acquire), + "R2: non-fiber WaitSema did not throw"); + t.Equals(waitRet.load(std::memory_order_acquire), blockSid, + "R2: non-fiber WaitSema acquired the signalled permit and returned blockSid"); + + // Scheduler must still be healthy: a normal fiber runs to completion. + gREntryReached.store(0u, std::memory_order_release); + const int32_t tid = startSchedWorker(rdram.data(), &runtime, + 0x00731000u, 10, + kRStackBase + kRStackSize, kRStackSize); + t.IsTrue(tid > 0, "R2: post-test fiber started"); + const bool ran = waitUntil([&]() + { + return gREntryReached.load(std::memory_order_acquire) == 1u; + }, std::chrono::milliseconds(2000)); + t.IsTrue(ran, "R2: scheduler still healthy — fiber runs after the non-fiber block"); + + waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(2000)); + deleteSchedSema(rdram.data(), &runtime, blockSid); + }); + + // ------------------------------------------------------------------ + // R3: stopAlarmWorker joins promptly and no alarm fires after stop + // ------------------------------------------------------------------ + tc.Run("R3: stopAlarmWorker joins promptly and no alarm fires after stop", [](TestCase &t) + { + // SchedFixture's ctor calls notifyRuntimeStop(), which ensures any + // prior alarm worker is stopped+joined before this test starts. + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00732000u, &stepAlarmCallbackMark); + gRAlarmFired.store(0u, std::memory_order_release); + + // Queue an alarm far enough in the future that it cannot possibly + // become due before the immediately-following stopAlarmWorker() call + // sets the stop flag, no matter how much a tool like ThreadSanitizer + // slows down thread scheduling. This test's job is to verify that + // stopAlarmWorker() reliably prevents *any* pending alarm from firing + // once stop has been requested -- not to race a nearly-due alarm + // against the stop request (that race is a property of an + // unrealistically small tick count, not something the API promises + // to win). 60000 ticks * 64us/tick =~ 3.84s: SetAlarm() and the very + // next statement (stopAlarmWorker()) run back-to-back on this same + // thread with no intervening sleep, so stop is requested within a + // handful of microseconds -- many orders of magnitude below 3.84s + // even under heavy TSan instrumentation. + R5900Context sc{}; + setRegU32(sc, 4, 60000u); // ticks = 60000 (~3.84s); see rationale above + setRegU32(sc, 5, 0x00732000u); // handler entry + setRegU32(sc, 6, 0u); // arg + ps2_syscalls::SetAlarm(rdram.data(), &sc, &runtime); + const int32_t alarmId = getRegS32(sc, 2); + t.IsTrue(alarmId > 0, "R3: SetAlarm queued an alarm and started the worker"); + + // Immediately stop the worker — must return promptly (joins, does not spin). + const auto t0 = std::chrono::steady_clock::now(); + ps2_syscalls::stopAlarmWorker(); + const auto elapsed = std::chrono::steady_clock::now() - t0; + t.IsTrue(elapsed < std::chrono::seconds(1), + "R3: stopAlarmWorker returned within 1s (joined, did not hang)"); + + // After a clean join, the callback must not fire (worker exited before rdram was touched). + std::this_thread::sleep_for(std::chrono::milliseconds(50)); + const uint32_t fired = gRAlarmFired.load(std::memory_order_acquire); + t.Equals(fired, 0u, "R3: alarm callback did not fire after stopAlarmWorker()"); + }); + + // ------------------------------------------------------------------ + // R4: exit handler that blocks resumes and the fiber finishes cleanly + // ------------------------------------------------------------------ + tc.Run("R4: exit handler that blocks resumes and the fiber finishes cleanly", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + // Body entry: 0x00733000; handler entry: 0x00733100 + // (stepRegisterExitHandlerThenExit has kHandlerEntry=0x00733100 hardcoded) + runtime.registerFunction(0x00733000u, &stepRegisterExitHandlerThenExit); + runtime.registerFunction(0x00733100u, &stepExitHandlerBlocking); + + gRStarted.store(0u, std::memory_order_release); + gRExitRan.store(0u, std::memory_order_release); + + const int32_t tid = startSchedWorker(rdram.data(), &runtime, + 0x00733000u, 10, + kRStackBase + 2u * kRStackSize, kRStackSize); + t.IsTrue(tid > 0, "R4: worker started"); + + // Wait until the body ran and registered the blocking exit handler. + const bool started = waitUntil([&]() + { + return gRStarted.load(std::memory_order_acquire) == 1u; + }, std::chrono::milliseconds(2000)); + t.IsTrue(started, "R4: fiber body ran and registered the blocking exit handler"); + + // Give the exit handler time to reach SleepThread and park. + std::this_thread::sleep_for(std::chrono::milliseconds(50)); + t.IsTrue(g_activeThreads.load(std::memory_order_acquire) >= 1, + "R4: fiber still alive while its exit handler is blocked (not freed early)"); + + // Wake the sleeping exit handler from the host. + { + R5900Context sc{}; + setRegU32(sc, 4, static_cast(tid)); + ps2_syscalls::WakeupThread(rdram.data(), &sc, &runtime); + } + + // Handler should complete and the fiber should finish. + const bool handlerDone = waitUntil([&]() + { + return gRExitRan.load(std::memory_order_acquire) == 1u; + }, std::chrono::milliseconds(2000)); + t.IsTrue(handlerDone, "R4: blocking exit handler resumed and ran to completion"); + + const bool drained = waitUntil([&]() + { + return g_activeThreads.load(std::memory_order_acquire) == 0; + }, std::chrono::milliseconds(2000)); + t.IsTrue(drained, "R4: fiber freed cleanly after exit handler finished (no double-free/leak)"); + + }); + + // ------------------------------------------------------------------ + // R5: nested SuspendThread requires matching ResumeThread count to wake + // ------------------------------------------------------------------ + tc.Run("R5: nested SuspendThread requires matching ResumeThread count to wake", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + // Reuse stepProgressLoop (protocol-namespace step fn); fresh registration. + runtime.registerFunction(0x00734000u, &stepProgressLoop); + + // Reset the protocol-suite slots that stepProgressLoop uses. + { + gProgressCtr.store(0u, std::memory_order_release); + gStopFlag.store(0u, std::memory_order_release); + gStartedFlag.store(0u, std::memory_order_release); + } + + const int32_t tid = startSchedWorker(rdram.data(), &runtime, + 0x00734000u, 10, + kRStackBase + 3u * kRStackSize, kRStackSize); + t.IsTrue(tid > 0, "R5: worker started"); + + // Wait until the loop is making progress. + const bool running = waitUntil([&]() + { + return gProgressCtr.load(std::memory_order_acquire) > 0u; + }, std::chrono::milliseconds(2000)); + t.IsTrue(running, "R5: fiber is making progress before suspend"); + + auto progress = [&]() + { + return gProgressCtr.load(std::memory_order_acquire); + }; + auto suspend = [&]() + { + R5900Context sc{}; + setRegU32(sc, 4, static_cast(tid)); + ps2_syscalls::SuspendThread(rdram.data(), &sc, &runtime); + return getRegS32(sc, 2); + }; + auto resume = [&]() + { + R5900Context sc{}; + setRegU32(sc, 4, static_cast(tid)); + ps2_syscalls::ResumeThread(rdram.data(), &sc, &runtime); + return getRegS32(sc, 2); + }; + + // Suspend twice (nested). + t.Equals(suspend(), KE_OK, "R5: first SuspendThread OK"); + t.Equals(suspend(), KE_OK, "R5: second SuspendThread OK"); + + // Progress must halt. Sample, wait, sample again. + std::this_thread::sleep_for(std::chrono::milliseconds(50)); + const uint32_t afterSuspend = progress(); + std::this_thread::sleep_for(std::chrono::milliseconds(100)); + t.Equals(progress(), afterSuspend, "R5: progress halted while suspended (count=2)"); + + // Resume once → still suspended (PS2 count 2->1). + t.Equals(resume(), KE_OK, "R5: first ResumeThread OK"); + std::this_thread::sleep_for(std::chrono::milliseconds(100)); + t.Equals(progress(), afterSuspend, "R5: progress still halted after one resume (count=1)"); + + // Resume again → count reaches 0 → clear_suspend wakes the fiber. + t.Equals(resume(), KE_OK, "R5: second ResumeThread OK"); + const uint32_t c1 = progress(); + const bool resumed = waitUntil([&](){ return progress() > c1; }, + std::chrono::milliseconds(1000)); + t.IsTrue(resumed, "R5: progress resumes after matching second resume (count=0)"); + + // Stop the loop and drain. + { + gStopFlag.store(1u, std::memory_order_release); + } + waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(2000)); + + }); + + }); // MiniTest::Case("SchedulerRace") +} + +// --------------------------------------------------------------------------- +// Scheduler stress tests — R6 sleep/wake-storm and R7 borrowed-worker WaitSema. +// --------------------------------------------------------------------------- + +namespace +{ + // ----------------------------------------------------------------------- + // RDRAM slot constants for R6/R7 (0x4330–0x434F range) + // ----------------------------------------------------------------------- + static constexpr uint32_t kR6Counter = 0x00004330u; // R6: SleepThread iteration count (u32) + static constexpr uint32_t kR6Done = 0x00004334u; // R6: set to 1 when the loop completes (u32) + static constexpr uint32_t kR7Started = 0x00004338u; // R7: fiber sets to 1 when it begins running (u32) + static constexpr uint32_t kR7Signalled = 0x0000433Cu; // R7: fiber sets to 1 just after SignalSema (u32) + + // Host <-> fiber mailboxes (see gSeq/... rationale above). + static std::atomic gR6Counter{0}; + static std::atomic gR6Done{0}; + static std::atomic gR7Started{0}; + static std::atomic gR7Signalled{0}; + + // stepSleepLoopN: SleepThread N times; bump kR6Counter each return; set kR6Done at end. + static void stepSleepLoopN(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + constexpr uint32_t kN = 500u; + for (uint32_t i = 0; i < kN; ++i) + { + R5900Context sc{}; + ps2_syscalls::SleepThread(rdram, &sc, runtime); + gR6Counter.fetch_add(1u, std::memory_order_relaxed); + } + gR6Done.store(1u, std::memory_order_release); + ctx->pc = 0u; + } + + // stepSignalAfterDelay: mark started, yield-spin, then SignalSema(workSid) and exit. + // This fiber is the ONLY producer of the sema permit the borrowed host worker waits on. + static void stepSignalAfterDelay(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + gR7Started.store(1u, std::memory_order_release); + + for (int i = 0; i < 100; ++i) + { + runtime->shouldPreemptGuestExecution(); + } + + int32_t workSid = 0; + std::memcpy(&workSid, rdram + kSlotWorkSid, 4); + R5900Context sc{}; + setRegU32(sc, 4, static_cast(workSid)); + ps2_syscalls::SignalSema(rdram, &sc, runtime); + + gR7Signalled.store(1u, std::memory_order_release); + ctx->pc = 0u; + } + +} // anonymous namespace + +void register_scheduler_stress_tests() +{ + MiniTest::Case("SchedulerStress", [](TestCase &tc) + { + // ------------------------------------------------------------------ + // R6: SleepThread/WakeupThread no-count wake-storm + // ------------------------------------------------------------------ + tc.Run("SleepThread/WakeupThread wake-storm never drops a no-count wake", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00740000u, &stepSleepLoopN); + + // Reset the shared slots. + gR6Counter.store(0u, std::memory_order_release); + gR6Done.store(0u, std::memory_order_release); + + // Launch the sleeping fiber. It loops SleepThread 500 times. + const int32_t tid = startSchedWorker(rdram.data(), &runtime, + 0x00740000u, 10, + 0x004CA000u, 0x2000u); + t.IsTrue(tid > 0, "R6: sleep-loop fiber should start"); + if (tid <= 0) + { + return; + } + + // Tight wake-storm. We deliberately DO NOT wait for THS_WAIT before firing — + // doing so would close the arm_park/publish race window the test exists to probe. + // Fire many more wakeups than iterations: SleepThread re-sleeps immediately so + // duplicate/early wakeups are harmless (no-count: an extra wake just bumps + // wakeupCount and the next SleepThread consumes it). The point is that across + // 500 iterations, many WakeupThread calls land in the pre-park window; not one + // may be lost or the fiber hangs forever in SleepThread. + const auto deadline = std::chrono::steady_clock::now() + std::chrono::milliseconds(2500); + while (std::chrono::steady_clock::now() < deadline) + { + if (gR6Done.load(std::memory_order_acquire) != 0u) break; + + R5900Context wc{}; + setRegU32(wc, 4, static_cast(tid)); + ps2_syscalls::WakeupThread(rdram.data(), &wc, &runtime); + // No sleep, no THS_WAIT confirmation — keep the loop as tight as possible so + // wakeups race against the fiber's arm_park -> block_current transition. + } + + // The fiber must have completed all 500 iterations. + const bool finished = waitUntil([&]() + { + return gR6Done.load(std::memory_order_acquire) == 1u; + }, std::chrono::milliseconds(500)); + + const uint32_t counter = gR6Counter.load(std::memory_order_acquire); + + t.IsTrue(finished, "R6: sleep-loop fiber should complete all 500 SleepThread iterations (no wake lost)"); + t.Equals(counter, 500u, "R6: fiber should have returned from SleepThread exactly 500 times"); + + // Fiber should have exited and decremented g_activeThreads. + const bool drained = waitUntil([&]() + { + return g_activeThreads.load(std::memory_order_acquire) <= 0; + }, std::chrono::milliseconds(1000)); + t.IsTrue(drained, "R6: sleep-loop fiber should exit and drain g_activeThreads"); + + }); + + // ------------------------------------------------------------------ + // R7: borrowed-worker WaitSema, permit produced only by a fiber + // ------------------------------------------------------------------ + tc.Run("borrowed-worker WaitSema unblocks via fiber-only SignalSema", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00748000u, &stepSignalAfterDelay); + + // count=0, max=1: the borrowed worker must block; only the fiber can produce a permit. + const int32_t workSid = createSchedSema(rdram.data(), &runtime, 0, 1); + t.IsTrue(workSid > 0, "R7: work sema should be created"); + if (workSid <= 0) + { + return; + } + + rdramWrite32(rdram, kSlotWorkSid, static_cast(workSid)); + gR7Started.store(0u, std::memory_order_release); + gR7Signalled.store(0u, std::memory_order_release); + + // Start the signalling fiber FIRST so it is enqueued Ready before the borrowed + // worker grabs the token. The token-release loop handles either interleaving, + // but enqueuing first keeps the common path deterministic. + const int32_t fiberTid = startSchedWorker(rdram.data(), &runtime, + 0x00748000u, 10, + 0x004CC000u, 0x2000u); + t.IsTrue(fiberTid > 0, "R7: signalling fiber should start"); + if (fiberTid <= 0) + { + deleteSchedSema(rdram.data(), &runtime, workSid); + return; + } + + // Background host thread = a borrowed worker (IRQ-handler style). It acquires the + // guest token, calls WaitSema on the count=0 sema, and records the result. + std::atomic workerDone{false}; + std::atomic workerRet{-9999}; + std::atomic workerThrew{false}; + + std::thread borrowedWorker([&]() + { + try + { + g_currentThreadId = -1; // non-fiber host worker (matches IRQ/alarm workers) + ps2sched::async_guest_begin(); // acquire the guest token (AsyncGuestScope-equivalent) + R5900Context wc{}; + setRegU32(wc, 4, static_cast(workSid)); + ps2_syscalls::WaitSema(rdram.data(), &wc, &runtime); // count=0 -> borrowed-worker block/retry path + workerRet.store(getRegS32(wc, 2), std::memory_order_release); + ps2sched::async_guest_end(); // release the guest token + } + catch (...) + { + workerThrew.store(true, std::memory_order_release); + // Best-effort token release on the exception path so we never wedge the executor. + ps2sched::async_guest_end(); + } + workerDone.store(true, std::memory_order_release); + }); + + // If the borrowed-worker path is present: WaitSema drops the token, the fiber runs, + // signals workSid, the worker re-checks count, consumes the permit, returns workSid. + // If absent: the worker spins holding the token, the fiber never runs, + // and workerDone never becomes true -> this wait times out -> FAIL. + const bool finished = waitUntil([&]() + { + return workerDone.load(std::memory_order_acquire); + }, std::chrono::milliseconds(3000)); + + if (!finished) + { + // Deadlock escape hatch so we don't hang the whole test binary on failure: + // signal the sema from this (main) thread to release the borrowed worker, + // then join. The assertion below still records the failure. + R5900Context sc{}; + setRegU32(sc, 4, static_cast(workSid)); + ps2_syscalls::SignalSema(rdram.data(), &sc, &runtime); + } + + if (borrowedWorker.joinable()) + { + borrowedWorker.join(); + } + + t.IsTrue(finished, "R7: borrowed-worker WaitSema should complete (the token is released so a fiber can signal)"); + t.IsFalse(workerThrew.load(std::memory_order_acquire), "R7: borrowed worker should not throw"); + t.Equals(workerRet.load(std::memory_order_acquire), workSid, "R7: borrowed-worker WaitSema should return workSid"); + + // The permit must have come from the fiber, not from the deadlock escape hatch. + const uint32_t signalled = gR7Signalled.load(std::memory_order_acquire); + t.Equals(signalled, 1u, "R7: the fiber (not the main thread) should have produced the permit"); + + // Everything should drain. + const bool drained = waitUntil([&]() + { + return g_activeThreads.load(std::memory_order_acquire) <= 0; + }, std::chrono::milliseconds(1000)); + t.IsTrue(drained, "R7: signalling fiber should exit and drain g_activeThreads"); + + deleteSchedSema(rdram.data(), &runtime, workSid); + }); + + }); // MiniTest::Case("SchedulerStress") +} + +// --------------------------------------------------------------------------- +// Scheduler supplement tests — S1 (vsync wait-list cleanup) and S2 (update_priority queued fiber). +// --------------------------------------------------------------------------- + +namespace +{ + // ----------------------------------------------------------------------- + // New RDRAM slot constants for S1 (0x4350-0x4368 range) + // ----------------------------------------------------------------------- + static constexpr uint32_t kS1SlotEntered = 0x00004350u; // uint32: set to 1 by stepVsyncWaitForever before first WaitForNextVSyncTick + static constexpr uint32_t kS1SlotWorkSid = 0x00004354u; // int32: sema id fiber B blocks on (count 0) + static constexpr uint32_t kS1SlotBWoke = 0x00004358u; // uint32: set to 1 by fiber B if its WaitSema ever returns + + // Host <-> fiber mailboxes for S1 (see gSeq/... rationale above). + static std::atomic gS1Entered{0}; + static std::atomic gS1BWoke{0}; + + // stepVsyncWaitForever — fiber body for S1's target fiber A. + // Marks kS1SlotEntered=1, then loops calling WaitForNextVSyncTick so it stays + // parked in g_vsync_waitList until terminated. + static void stepVsyncWaitForever(uint8_t *rdram, R5900Context * /*ctx*/, PS2Runtime *runtime) + { + gS1Entered.store(1u, std::memory_order_release); + for (;;) + { + ps2_syscalls::WaitForNextVSyncTick(rdram, runtime); + // Belt-and-braces: observe terminate request promptly even if a tick + // fires and wakes us before TerminateThread delivers the request. + runtime->shouldPreemptGuestExecution(); + } + // unreachable; ThreadExitException unwinds out. + } + + // stepWaitSemaBait — fiber body for S1's fiber B. + // Reads the sema id from kS1SlotWorkSid, blocks on it (count 0 = never wakes + // normally). If it ever returns from WaitSema it sets kS1SlotBWoke=1. + static void stepWaitSemaBait(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + int32_t workSid = 0; + std::memcpy(&workSid, rdram + kS1SlotWorkSid, 4); + R5900Context sc{}; + setRegU32(sc, 4, static_cast(workSid)); + ps2_syscalls::WaitSema(rdram, &sc, runtime); // count 0 -> blocks + gS1BWoke.store(1u, std::memory_order_release); // reached only on spurious wakeup + ctx->pc = 0u; + } + +} // anonymous namespace + +void register_scheduler_vsync_priority_tests() +{ + MiniTest::Case("SchedulerVSyncAndPriority", [](TestCase &tc) + { + // ------------------------------------------------------------------ + // S1: vsync wait-list stale-tid cleanup + // ------------------------------------------------------------------ + tc.Run("vsync wait-list drops a terminated waiter's tid", [](TestCase &t) + { + namespace istate = ps2_syscalls::interrupt_state; + + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00750000u, &stepVsyncWaitForever); + runtime.registerFunction(0x00752000u, &stepWaitSemaBait); + + // Clear S1 slots. + gS1Entered.store(0u, std::memory_order_release); + gS1BWoke.store(0u, std::memory_order_release); + + // Sema that fiber B parks on (count 0, max 1). + const int32_t workSid = createSchedSema(rdram.data(), &runtime, 0, 1); + t.IsTrue(workSid > 0, "S1: bait sema created"); + if (workSid <= 0) + { + return; + } + { + int32_t v = workSid; + std::memcpy(rdram.data() + kS1SlotWorkSid, &v, 4); + } + + // Start fiber A (the vsync waiter). WaitForNextVSyncTick calls + // EnsureVSyncWorkerRunning internally. + const int32_t tidA = startSchedWorker( + rdram.data(), &runtime, 0x00750000u, 30, 0x004D0000u, 0x2000u); + t.IsTrue(tidA > 0, "S1: vsync-waiter fiber A started"); + if (tidA <= 0) + { + deleteSchedSema(rdram.data(), &runtime, workSid); + return; + } + + // Wait until A has set kS1SlotEntered AND its tid appears in g_vsync_waitList. + const bool aQueued = waitUntil([&]() + { + if (gS1Entered.load(std::memory_order_acquire) == 0u) return false; + std::lock_guard lk(istate::g_vsync_flag_mutex); + for (const auto &[tid, token] : istate::g_vsync_waitList) + if (tid == tidA) return true; + return false; + }, std::chrono::milliseconds(1000)); + t.IsTrue(aQueued, "S1: fiber A is queued in g_vsync_waitList"); + + // Sanity: confirm A's tid is currently in the list. + { + std::lock_guard lk(istate::g_vsync_flag_mutex); + bool found = false; + for (const auto &[tid, token] : istate::g_vsync_waitList) if (tid == tidA) found = true; + t.IsTrue(found, "S1: pre-terminate, A's tid is in the wait-list"); + } + + // Terminate fiber A. TerminateThread wakes A (request_terminate) and joins + // it (join_fiber). A unwinds out of WaitForNextVSyncTick; the wake path + // removes A's tid from g_vsync_waitList before signaling B. + { + R5900Context termCtx{}; + setRegU32(termCtx, 4, static_cast(tidA)); + ps2_syscalls::TerminateThread(rdram.data(), &termCtx, &runtime); + t.Equals(getRegS32(termCtx, 2), KE_OK, "S1: TerminateThread(A) returns KE_OK"); + } + + // Wait for A to finish (TerminateThread joined it, but g_activeThreads + // decrement may trail slightly). + const bool aGone = waitUntil([&]() + { + return g_activeThreads.load(std::memory_order_acquire) == 0; + }, std::chrono::milliseconds(1000)); + t.IsTrue(aGone, "S1: fiber A finished after TerminateThread"); + + // A's tid must no longer be in g_vsync_waitList. + { + std::lock_guard lk(istate::g_vsync_flag_mutex); + bool stale = false; + for (const auto &[tid, token] : istate::g_vsync_waitList) if (tid == tidA) stale = true; + t.IsFalse(stale, "S1: terminated fiber A's tid was removed from g_vsync_waitList"); + t.IsTrue(istate::g_vsync_waitList.empty(), + "S1: g_vsync_waitList is empty after the sole waiter is terminated"); + } + + // SECONDARY ASSERTION (recycle tripwire): start fiber B on the same tid pool. + // B blocks on a count-0 sema and must NOT be spuriously woken by a stale vsync + // wakeup targeting A's (possibly recycled) tid. + const int32_t tidB = startSchedWorker( + rdram.data(), &runtime, 0x00752000u, 30, 0x004D2000u, 0x2000u); + t.IsTrue(tidB > 0, "S1: bait fiber B started"); + + // Wait until B is actually blocked on the sema. + const bool bBlocked = waitUntil([&]() + { + return getSemaWaiters(rdram.data(), &runtime, workSid) >= 1; + }, std::chrono::milliseconds(1000)); + t.IsTrue(bBlocked, "S1: bait fiber B is blocked on the sema"); + + // Ensure the vsync worker is running and let it tick several times + // (~60 ms >= 2-3 real ticks at the 16.667 ms cadence). + ps2_syscalls::EnsureVSyncWorkerRunning(rdram.data(), &runtime); + std::this_thread::sleep_for(std::chrono::milliseconds(60)); + + // B must NOT have been spuriously woken by a stale vsync wakeup. + { + const uint32_t bWoke = gS1BWoke.load(std::memory_order_acquire); + t.Equals(bWoke, 0u, + "S1: bait fiber B was not spuriously woken by a stale vsync tid"); + } + + // Cleanup: signal the sema so B can unwind cleanly, then shut down. + { + R5900Context sigCtx{}; + setRegU32(sigCtx, 4, static_cast(workSid)); + ps2_syscalls::SignalSema(rdram.data(), &sigCtx, &runtime); + } + waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + + deleteSchedSema(rdram.data(), &runtime, workSid); + ps2sched::scheduler_shutdown(); // joins the vsync worker via stopInterruptWorker() + }); + + // ------------------------------------------------------------------ + // S2: update_priority re-sorts a queued (Ready) fiber + // ------------------------------------------------------------------ + tc.Run("ChangeThreadPriority re-sorts a queued (Ready) fiber to the front", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + // Reuse stepLogAndExit registered at a fresh entry address for S2. + runtime.registerFunction(0x00758000u, &stepLogAndExit); + + rdramSeqReset(rdram); + std::memset(rdram.data() + kRunLog, 0, 32u); + + // Freeze the executor so all three fibers sit Ready in the run queue and + // none can run. update_priority must take its wasQueued==true branch + // (remove_locked + enqueue_locked) for the queued A. + ps2sched::async_guest_begin(); + + // Start A(20), B(15), C(10). Queue order ascending by priority: [C(10), B(15), A(20)]. + const int32_t tidA = startSchedWorker(rdram.data(), &runtime, 0x00758000u, 20, 0x004D4000u, 0x2000u); + const int32_t tidB = startSchedWorker(rdram.data(), &runtime, 0x00758000u, 15, 0x004D6000u, 0x2000u); + const int32_t tidC = startSchedWorker(rdram.data(), &runtime, 0x00758000u, 10, 0x004D8000u, 0x2000u); + t.IsTrue(tidA > 0, "S2: A started"); + t.IsTrue(tidB > 0, "S2: B started"); + t.IsTrue(tidC > 0, "S2: C started"); + + // Change A's priority from 20 to 5 while A is queued Ready. + // Expected new order: [A(5), C(10), B(15)]. + { + R5900Context sc{}; + setRegU32(sc, 4, static_cast(tidA)); + setRegU32(sc, 5, 5u); + ps2_syscalls::ChangeThreadPriority(rdram.data(), &sc, &runtime); + t.Equals(getRegS32(sc, 2), KE_OK, "S2: ChangeThreadPriority(A,5) returns KE_OK"); + } + + // Release the executor; fibers now run in the re-sorted priority order. + ps2sched::async_guest_end(); + + const bool allDone = waitUntil([&](){ return rdramSeq(rdram) >= 3u; }, + std::chrono::milliseconds(2000)); + t.IsTrue(allDone, "S2: all 3 fibers completed"); + + int32_t log[3] = {}; + std::memcpy(log, rdram.data() + kRunLog, 12); + t.Equals(log[0], tidA, "S2: A (re-sorted to prio 5) runs first"); + t.Equals(log[1], tidC, "S2: C (prio 10) runs second"); + t.Equals(log[2], tidB, "S2: B (prio 15) runs last"); + + waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + }); + + }); // MiniTest::Case("SchedulerVSyncAndPriority") +} + +// --------------------------------------------------------------------------- +// Scheduler lifecycle tests — executor thread assertion, join-fiber priority +// stability, exit-hook completion, and multi-fiber shutdown ordering. +// --------------------------------------------------------------------------- + +// Defined in ps2_scheduler.cpp (declared in the private header ps2_fiber.h that +// the test target cannot include). Linked from ps2_runtime. +bool ps2fiber_on_executor_thread(); + +namespace +{ + // ---- V6 RDRAM slot constants (0x4370-0x43B0) ---- + // U1 + static constexpr uint32_t kU1SlotFiberOnExec = 0x00004370u; // int32: 1 if fiber saw on_executor==true + static constexpr uint32_t kU1SlotHostOnExec = 0x00004374u; // int32: host thread's on_executor result + + // Host <-> fiber mailboxes (see gSeq/... rationale above). + static std::atomic gU1FiberOnExec{-1}; + static std::atomic gU2Spinning{0}; + + // U2 + static constexpr uint32_t kU2SlotJoinerTid = 0x00004378u; // int32: A's tid (joiner) + static constexpr uint32_t kU2SlotTargetTid = 0x0000437Cu; // int32: B's tid (join target) + static constexpr uint32_t kU2SlotSpinning = 0x00004380u; // uint32: B sets 1 after first yield (window open) + static constexpr uint32_t kU2SlotJoinerPrio = 0x00004384u; // int32: A's current_priority after join + + // U3 + static constexpr uint32_t kU3SlotWorkSid = 0x00004388u; // int32: sema B blocks on (count 0) + static constexpr uint32_t kU3SlotExitRan = 0x0000438Cu; // uint32: set to 1 by B's exit handler + + // U4 (4 fibers: body-sentinel + hook-sentinel each) + static constexpr uint32_t kU4SlotWorkSid = 0x00004390u; // int32: sema all U4 fibers block on (count 0) + static constexpr uint32_t kU4BodyBase = 0x00004394u; // uint32[4]: body-reached sentinels + static constexpr uint32_t kU4HookBase = 0x000043A4u; // uint32[4]: exit-handler-ran sentinels + static constexpr int kU4FiberCount = 4; + + // ---- V6 step functions ---- + + // U1: runs on the executor thread; records whether ps2fiber_on_executor_thread() is true. + static void stepRecordOnExecutor(uint8_t *rdram, R5900Context *ctx, PS2Runtime * /*runtime*/) + { + const int32_t onExec = ps2fiber_on_executor_thread() ? 1 : 0; + gU1FiberOnExec.store(onExec, std::memory_order_release); + ctx->pc = 0u; + } + + // U2 target (fiber B): yields 20 times, setting kU2SlotSpinning=1 after the first yield, then exits. + static void stepU2TargetYieldLoop(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + // Signal the reprio thread FIRST, then loop many times so reprio has + // a generous window to fire ChangeThreadPriority(A, 10) while B is still + // running. 500 iterations gives reprio ~tens-of-microseconds window on + // modern hardware. The loop calls shouldPreemptGuestExecution() which + // triggers yield_point() every 128th call; since A's floored priority + // (61) is numerically higher than B (60), B never yields to A here. + gU2Spinning.store(1u, std::memory_order_release); // window is open + for (int i = 0; i < 500; ++i) + { + runtime->shouldPreemptGuestExecution(); // cooperative yield point + } + ctx->pc = 0u; + } + + // U2 joiner (fiber A): reads B's tid, calls TerminateThread(B), then reads its own current_priority. + static void stepU2JoinerTerminateThenLog(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + int32_t targetTid = 0; + std::memcpy(&targetTid, rdram + kU2SlotTargetTid, 4); + + R5900Context tc{}; + setRegU32(tc, 4, static_cast(targetTid)); + ps2_syscalls::TerminateThread(rdram, &tc, runtime); // request_terminate + join_fiber + + // Read OUR OWN current_priority (TH_SELF = tid 0). + R5900Context rc{}; + setRegU32(rc, 4, 0u); // TH_SELF + setRegU32(rc, 5, kReferScratch); // status struct dest + ps2_syscalls::ReferThreadStatus(rdram, &rc, runtime); + int32_t curPrio = 0; + std::memcpy(&curPrio, rdram + kReferScratch + 0x18, 4); // current_priority @0x18 + std::memcpy(rdram + kU2SlotJoinerPrio, &curPrio, 4); + + ctx->pc = 0u; + } + + // U3 exit handler: non-blocking; writes sentinel to kU3SlotExitRan. + static void stepU3ExitHandlerMark(uint8_t *rdram, R5900Context *ctx, PS2Runtime * /*runtime*/) + { + uint32_t one = 1u; + std::memcpy(rdram + kU3SlotExitRan, &one, 4); + ctx->pc = 0u; + } + + // U3 worker body: registers stepU3ExitHandlerMark as exit handler, then blocks on kU3SlotWorkSid. + static void stepU3RegisterHandlerThenBlock(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + static constexpr uint32_t kU3HandlerEntry = 0x00778100u; + R5900Context rc{}; + setRegU32(rc, 4, kU3HandlerEntry); // func + setRegU32(rc, 5, 0u); // arg + ps2_syscalls::RegisterExitHandler(rdram, &rc, runtime); + + int32_t workSid = 0; + std::memcpy(&workSid, rdram + kU3SlotWorkSid, 4); + R5900Context sc{}; + setRegU32(sc, 4, static_cast(workSid)); + ps2_syscalls::WaitSema(rdram, &sc, runtime); // blocks; never wakes normally + ctx->pc = 0u; + } + + // U4 exit handler: reads slot index from $a0 (handler arg), writes kU4HookBase+idx*4=1. + static void stepU4ExitHandlerMark(uint8_t *rdram, R5900Context *ctx, PS2Runtime * /*runtime*/) + { + const uint32_t idx = ::getRegU32(ctx, 4); // $a0 == handler.arg (slot index) + uint32_t one = 1u; + if (idx < static_cast(kU4FiberCount)) + std::memcpy(rdram + kU4HookBase + idx * 4u, &one, 4); + ctx->pc = 0u; + } + + // U4 worker body: reads slot index from $a0 (StartThread arg), writes body sentinel, + // registers exit handler forwarding the same index, then blocks on kU4SlotWorkSid. + static void stepU4RegisterHandlerThenBlock(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + static constexpr uint32_t kU4HandlerEntry = 0x00780100u; + const uint32_t idx = ::getRegU32(ctx, 4); // $a0 == StartThread arg == slot index + + uint32_t one = 1u; + if (idx < static_cast(kU4FiberCount)) + std::memcpy(rdram + kU4BodyBase + idx * 4u, &one, 4); + + R5900Context rc{}; + setRegU32(rc, 4, kU4HandlerEntry); // func + setRegU32(rc, 5, idx); // arg -> forwarded to handler's $a0 + ps2_syscalls::RegisterExitHandler(rdram, &rc, runtime); + + int32_t workSid = 0; + std::memcpy(&workSid, rdram + kU4SlotWorkSid, 4); + R5900Context sc{}; + setRegU32(sc, 4, static_cast(workSid)); + ps2_syscalls::WaitSema(rdram, &sc, runtime); // blocks; shutdown terminates it + ctx->pc = 0u; + } + +} // anonymous namespace + +void register_scheduler_lifecycle_tests() +{ + MiniTest::Case("SchedulerLifecycle", [](TestCase &tc) + { + // ------------------------------------------------------------------ + // U1: ps2fiber_on_executor_thread returns true on the executor, false elsewhere + // ------------------------------------------------------------------ + tc.Run("U1: ps2fiber_on_executor_thread is true on the executor, false elsewhere", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00760000u, &stepRecordOnExecutor); + + // Sentinels: -1 means "not written yet" so we can tell the fiber actually ran. + gU1FiberOnExec.store(-1, std::memory_order_release); + { int32_t init = -1; std::memcpy(rdram.data() + kU1SlotHostOnExec, &init, 4); } + + // Host-thread side: a plain std::thread is NOT the guest executor thread, + // so ps2fiber_on_executor_thread() must return false there. + std::thread hostProbe([&]() + { + const int32_t r = ps2fiber_on_executor_thread() ? 1 : 0; + std::memcpy(rdram.data() + kU1SlotHostOnExec, &r, 4); + }); + hostProbe.join(); + + // Fiber side: the body runs on the guest executor thread. + const int32_t tid = startSchedWorker(rdram.data(), &runtime, + 0x00760000u, 10, 0x004D8000u, 0x2000u); + t.IsTrue(tid > 0, "U1: probe fiber started"); + + const bool fiberWrote = waitUntil([&]() + { + return gU1FiberOnExec.load(std::memory_order_acquire) != -1; + }, std::chrono::milliseconds(2000)); + t.IsTrue(fiberWrote, "U1: probe fiber recorded its on-executor result"); + + waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + + const int32_t fiberSaw = gU1FiberOnExec.load(std::memory_order_acquire); + int32_t hostSaw = -1; + std::memcpy(&hostSaw, rdram.data() + kU1SlotHostOnExec, 4); + +#if defined(PS2X_FIBER_PTHREAD) + // Backend-specific expectation: ps2fiber_on_executor_thread() is + // documented as "true when called on the registered guest executor + // thread" (the thread running guest_executor_main). On the pthread + // backend each fiber body runs on its own OS thread — never the + // executor thread itself — so a fiber correctly sees false here. + // Only the ucontext/Win32-Fibers backends run fiber bodies ON the + // executor thread. + t.Equals(fiberSaw, 0, "U1: pthread-backend fiber runs on its own OS thread, on_executor==false"); +#else + t.Equals(fiberSaw, 1, "U1: fiber on the executor thread sees on_executor==true"); +#endif + t.Equals(hostSaw, 0, "U1: a non-executor host thread sees on_executor==false"); + + }); + + // ------------------------------------------------------------------ + // U2: join_fiber restores the concurrently-changed joiner priority + // ------------------------------------------------------------------ + tc.Run("U2: join_fiber restores the concurrently-changed joiner priority, not a stale snapshot", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00768000u, &stepU2JoinerTerminateThenLog); + runtime.registerFunction(0x00770000u, &stepU2TargetYieldLoop); + + gU2Spinning.store(0u, std::memory_order_release); + { int32_t neg = -1; std::memcpy(rdram.data() + kU2SlotJoinerPrio, &neg, 4); } + + // Lock the executor so no fibers run until we are ready. + ps2sched::async_guest_begin(); + + // Start target B (prio 60, lower priority = runs AFTER A). + // A (prio 50, higher priority) runs first and enters join_fiber(B) + // before B has a chance to run. join_fiber applies a priority floor + // (prio 61) so that B (prio 60) runs while A waits. + const int32_t tidB = startSchedWorker(rdram.data(), &runtime, 0x00770000u, 60, 0x004E0000u, 0x2000u); + t.IsTrue(tidB > 0, "U2: target B started"); + { int32_t v = tidB; std::memcpy(rdram.data() + kU2SlotTargetTid, &v, 4); } + + // Start joiner A (prio 50). A immediately calls TerminateThread(B) -> join_fiber(B). + const int32_t tidA = startSchedWorker(rdram.data(), &runtime, 0x00768000u, 50, 0x004DC000u, 0x2000u); + t.IsTrue(tidA > 0, "U2: joiner A started"); + { int32_t v = tidA; std::memcpy(rdram.data() + kU2SlotJoinerTid, &v, 4); } + + // Start reprio thread BEFORE releasing the executor so it is already + // spinning when B sets kU2SlotSpinning=1. Use a tight spin (no sleep) + // to avoid the 1-ms poll latency that would miss B's narrow window. + std::atomic reprioReady{false}; + std::thread reprio([&]() + { + reprioReady.store(true, std::memory_order_release); + const auto deadline = std::chrono::steady_clock::now() + std::chrono::milliseconds(4000); + uint32_t s = 0; + while (std::chrono::steady_clock::now() < deadline) + { + s = gU2Spinning.load(std::memory_order_acquire); + if (s == 1u) break; + } + if (s != 1u) return; + R5900Context sc{}; + setRegU32(sc, 4, static_cast(tidA)); + setRegU32(sc, 5, 10u); + ps2_syscalls::ChangeThreadPriority(rdram.data(), &sc, &runtime); + }); + + // Wait until reprio is spinning before releasing the executor. + while (!reprioReady.load(std::memory_order_acquire)) + std::this_thread::yield(); + + // Release executor. A (prio 50) runs first, enters join_fiber(B), yields. + // B (prio 60) runs, sets kU2SlotSpinning=1; reprio thread detects and fires. + ps2sched::async_guest_end(); + + // Wait for both fibers to drain (A finishes after the join returns and it logs prio). + const bool drained = waitUntil([&]() + { + return g_activeThreads.load(std::memory_order_acquire) == 0; + }, std::chrono::milliseconds(4000)); + reprio.join(); + t.IsTrue(drained, "U2: both fibers finished"); + + int32_t joinerPrio = -1; + std::memcpy(&joinerPrio, rdram.data() + kU2SlotJoinerPrio, 4); + + // A's priority after the join must be the concurrently-set value, not the + // stale original captured before the join began. + t.Equals(joinerPrio, 10, "U2: joiner priority is the concurrently-set value (10), not stale (50)"); + + }); + + // ------------------------------------------------------------------ + // U3: TerminateThread(other) runs the target fiber's exit handler to completion + // ------------------------------------------------------------------ + tc.Run("U3: TerminateThread(other) runs the target fiber's exit handler to completion", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00778000u, &stepU3RegisterHandlerThenBlock); + runtime.registerFunction(0x00778100u, &stepU3ExitHandlerMark); + + const int32_t workSid = createSchedSema(rdram.data(), &runtime, 0, 1); + t.IsTrue(workSid > 0, "U3: work sema created"); + if (workSid <= 0) + { + return; + } + { int32_t v = workSid; std::memcpy(rdram.data() + kU3SlotWorkSid, &v, 4); } + { uint32_t z = 0u; std::memcpy(rdram.data() + kU3SlotExitRan, &z, 4); } + + const int32_t workerTid = startSchedWorker(rdram.data(), &runtime, 0x00778000u, 10, 0x004E4000u, 0x2000u); + t.IsTrue(workerTid > 0, "U3: worker started"); + + const bool blocked = waitUntil([&]() + { + return getSemaWaiters(rdram.data(), &runtime, workSid) >= 1; + }, std::chrono::milliseconds(1000)); + t.IsTrue(blocked, "U3: worker blocked on workSid after registering its exit handler"); + + // Terminate from the host: request_terminate wakes the worker, it unwinds via + // ThreadExitException, fiber_trampoline runs the exit hook -> our handler. + { + R5900Context termCtx{}; + setRegU32(termCtx, 4, static_cast(workerTid)); + ps2_syscalls::TerminateThread(rdram.data(), &termCtx, &runtime); + t.Equals(getRegS32(termCtx, 2), KE_OK, "U3: TerminateThread returns KE_OK"); + } + + const bool drained = waitUntil([&]() + { + return g_activeThreads.load(std::memory_order_acquire) == 0; + }, std::chrono::milliseconds(3000)); + t.IsTrue(drained, "U3: g_activeThreads reached 0 after termination"); + + uint32_t exitRan = 0u; + std::memcpy(&exitRan, rdram.data() + kU3SlotExitRan, 4); + t.Equals(exitRan, 1u, "U3: terminated fiber's exit handler ran"); + + deleteSchedSema(rdram.data(), &runtime, workSid); + }); + + // ------------------------------------------------------------------ + // U4: scheduler_shutdown lets Exiting fibers complete their exit handlers + // ------------------------------------------------------------------ + tc.Run("U4: scheduler_shutdown lets Exiting fibers complete their exit handlers", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00780000u, &stepU4RegisterHandlerThenBlock); + runtime.registerFunction(0x00780100u, &stepU4ExitHandlerMark); + + const int32_t workSid = createSchedSema(rdram.data(), &runtime, 0, kU4FiberCount); + t.IsTrue(workSid > 0, "U4: work sema created"); + if (workSid <= 0) + { + return; + } + { int32_t v = workSid; std::memcpy(rdram.data() + kU4SlotWorkSid, &v, 4); } + std::memset(rdram.data() + kU4BodyBase, 0, kU4FiberCount * 4); + std::memset(rdram.data() + kU4HookBase, 0, kU4FiberCount * 4); + + const uint32_t stacks[kU4FiberCount] = { 0x004E8000u, 0x004EC000u, 0x004F0000u, 0x004F4000u }; + int32_t tids[kU4FiberCount] = {}; + for (int i = 0; i < kU4FiberCount; ++i) + { + // Each fiber's slot index is forwarded as its StartThread arg + // ($a1), so stepU4RegisterHandlerThenBlock knows which of the + // per-fiber body/hook sentinel slots to write. + tids[i] = startSchedWorker(rdram.data(), &runtime, 0x00780000u, 10, + stacks[i], 0x2000u, static_cast(i)); + t.IsTrue(tids[i] > 0, "U4: fiber started"); + } + + // Wait until all fibers reached their bodies AND are blocked on the sema. + const bool allBlocked = waitUntil([&]() + { + return getSemaWaiters(rdram.data(), &runtime, workSid) >= kU4FiberCount; + }, std::chrono::milliseconds(2000)); + t.IsTrue(allBlocked, "U4: all fibers blocked on workSid after registering handlers"); + + // All bodies must have run before we check the exit-hook sentinels. + for (int i = 0; i < kU4FiberCount; ++i) + { + uint32_t body = 0u; + std::memcpy(&body, rdram.data() + kU4BodyBase + i * 4u, 4); + t.Equals(body, 1u, "U4: fiber body ran (sentinel set) before shutdown"); + } + + const auto t0 = std::chrono::steady_clock::now(); + ps2sched::scheduler_shutdown(); + const auto elapsed = std::chrono::steady_clock::now() - t0; + + t.IsTrue(elapsed < std::chrono::seconds(5), "U4: scheduler_shutdown returned within 5s"); + t.Equals(g_activeThreads.load(), 0, "U4: all fibers terminated"); + + // Every fiber's exit handler must have run to completion. A body sentinel set + // without its hook sentinel means the Exiting fiber was abandoned mid-hook. + for (int i = 0; i < kU4FiberCount; ++i) + { + uint32_t hook = 0u; + std::memcpy(&hook, rdram.data() + kU4HookBase + i * 4u, 4); + t.Equals(hook, 1u, "U4: this fiber's exit handler completed during shutdown"); + } + + // Do NOT call scheduler_shutdown again (matches T15). + deleteSchedSema(rdram.data(), &runtime, workSid); + }); + + }); // MiniTest::Case("SchedulerLifecycle") +} + +// --------------------------------------------------------------------------- +// Borrowed-worker suite: borrowed-worker WaitSema, exit handler calling +// ExitThread, and Mesa-loop reblock during shutdown. +// --------------------------------------------------------------------------- + +// ExitThread is not declared elsewhere in this TU; forward-declare it (resolved +// at link time from the runtime, like the other ps2_syscalls:: entry points). +namespace ps2_syscalls +{ + void ExitThread(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime); +} + +namespace +{ + // ---- W-suite RDRAM slots (0x43C0–0x43DF) ---- + // W1 + static constexpr uint32_t kW1WorkSid = 0x000043C0u; // int32 : sema the host worker waits on (count 0) + static constexpr uint32_t kW1Signalled = 0x000043C4u; // uint32: fiber set 1 right after SignalSema + // W2 + static constexpr uint32_t kW2Sentinel = 0x000043C8u; // uint32: exit handler set 1 BEFORE calling ExitThread + static constexpr uint32_t kW2BodyRan = 0x000043CCu; // uint32: fiber body set 1 before returning + // W3 + static constexpr uint32_t kW3WorkSid = 0x000043D0u; // int32 : sema fiber A loops on (count stays 0) + static constexpr uint32_t kW3ExitFlag = 0x000043D4u; // uint32: reserved slot (see plan note) + + // W1 producer fiber: spin on a few yield points, SignalSema(workSid), + // mark signalled, exit. Mirrors stepSignalAfterDelay (R7). + static void stepW1SignalAfterYields(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + for (int i = 0; i < 5; ++i) + runtime->shouldPreemptGuestExecution(); // cooperative yield x5 + + int32_t workSid = 0; + std::memcpy(&workSid, rdram + kW1WorkSid, 4); + R5900Context sc{}; + setRegU32(sc, 4, static_cast(workSid)); + ps2_syscalls::SignalSema(rdram, &sc, runtime); + + uint32_t one = 1u; + std::memcpy(rdram + kW1Signalled, &one, 4); + ctx->pc = 0u; + } + + // W2 exit handler: write the sentinel FIRST (proves the handler ran), then call the + // real ExitThread syscall, which throws ThreadExitException from inside the exit hook. + // The trampoline wraps g_fiber_exit_hook in try/catch, so this must NOT crash the process. + static void stepW2ExitHandlerCallsExitThread(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + uint32_t one = 1u; + std::memcpy(rdram + kW2Sentinel, &one, 4); // sentinel BEFORE the throw + R5900Context ec{}; + ps2_syscalls::ExitThread(rdram, &ec, runtime); // throws ThreadExitException + // unreachable + ctx->pc = 0u; + } + + // W2 fiber body: register the exit handler, mark body ran, return normally so the + // trampoline drives on_fiber_exit -> our handler. + static void stepW2RegisterHandlerThenReturn(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + static constexpr uint32_t kW2HandlerEntry = 0x007A8100u; + R5900Context rc{}; + setRegU32(rc, 4, kW2HandlerEntry); // func + setRegU32(rc, 5, 0u); // arg + ps2_syscalls::RegisterExitHandler(rdram, &rc, runtime); + + uint32_t one = 1u; + std::memcpy(rdram + kW2BodyRan, &one, 4); + ctx->pc = 0u; // return -> begin exiting -> hook runs + } + + // W3 fiber A: Mesa loop on a count==0 sema. WaitSema blocks; on a normal wake it + // re-checks count, finds 0, re-blocks. During shutdown block_current returns + // immediately (g_stop + terminateRequested), WaitSema unwinds via ThreadExitException. + static void stepW3MesaLoopOnSema(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + int32_t workSid = 0; + std::memcpy(&workSid, rdram + kW3WorkSid, 4); + for (;;) + { + R5900Context sc{}; + setRegU32(sc, 4, static_cast(workSid)); + ps2_syscalls::WaitSema(rdram, &sc, runtime); // count==0 -> blocks + // If WaitSema ever returns with workSid we'd consume a permit and loop; the + // sema is never signalled, so the only way out is the terminate unwind + // (ThreadExitException) during shutdown, which skips this point entirely. + const int32_t r = getRegS32(sc, 2); + if (r != workSid) break; // KE_WAIT_DELETE or similar -> stop looping + } + ctx->pc = 0u; + } + + // W3 fiber B: keep the scheduler busy / hold a second active fiber so the sema is + // never signalled. Simply blocks on the same sema (also count 0), so shutdown must + // terminate it too. + static void stepW3HoldResource(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + int32_t workSid = 0; + std::memcpy(&workSid, rdram + kW3WorkSid, 4); + R5900Context sc{}; + setRegU32(sc, 4, static_cast(workSid)); + ps2_syscalls::WaitSema(rdram, &sc, runtime); // blocks; shutdown terminates it + ctx->pc = 0u; + } + +} // anonymous namespace + +void register_scheduler_borrowed_worker_tests() +{ + MiniTest::Case("SchedulerBorrowedWorker", [](TestCase &tc) + { + // ------------------------------------------------------------------ + // W1: borrowed host worker WaitSema — no deadlock, no g_threads[-1] entry + // ------------------------------------------------------------------ + tc.Run("W1: borrowed-worker WaitSema succeeds and never creates g_threads[-1]", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x007A0000u, &stepW1SignalAfterYields); + + // count=0, max=1: a borrowed worker MUST block; only the fiber produces a permit. + const int32_t workSid = createSchedSema(rdram.data(), &runtime, 0, 1); + t.IsTrue(workSid > 0, "W1: work sema created"); + if (workSid <= 0) + { + return; + } + rdramWrite32(rdram, kW1WorkSid, static_cast(workSid)); + rdramWrite32(rdram, kW1Signalled, 0u); + + // Producer fiber: enqueued Ready first so it can run as soon as the worker + // releases the guest token inside its backoff. + const int32_t fiberTid = startSchedWorker(rdram.data(), &runtime, + 0x007A0000u, 10, 0x004FC000u, 0x2000u); + t.IsTrue(fiberTid > 0, "W1: producer fiber started"); + if (fiberTid <= 0) + { + deleteSchedSema(rdram.data(), &runtime, workSid); + return; + } + + std::atomic workerDone{false}; + std::atomic workerRet{-9999}; + std::atomic workerThrew{false}; + + // Borrowed host worker (IRQ/alarm style): tid -1, holds the guest token, + // calls WaitSema on the count=0 sema. It backs off via NonFiberBackoff, + // the fiber signals, and the Mesa re-check returns workSid. + std::thread borrowedWorker([&]() + { + try + { + g_currentThreadId = -1; // non-fiber host worker + ps2sched::async_guest_begin(); + R5900Context wc{}; + setRegU32(wc, 4, static_cast(workSid)); + ps2_syscalls::WaitSema(rdram.data(), &wc, &runtime); + workerRet.store(getRegS32(wc, 2), std::memory_order_release); + ps2sched::async_guest_end(); + } + catch (...) + { + workerThrew.store(true, std::memory_order_release); + ps2sched::async_guest_end(); + } + workerDone.store(true, std::memory_order_release); + }); + + const bool finished = waitUntil([&]() + { + return workerDone.load(std::memory_order_acquire); + }, std::chrono::milliseconds(3000)); + + if (!finished) + { + // Escape hatch so a regression does not hang the whole binary; the + // assertions below still record the failure. + R5900Context sc{}; + setRegU32(sc, 4, static_cast(workSid)); + ps2_syscalls::SignalSema(rdram.data(), &sc, &runtime); + } + if (borrowedWorker.joinable()) borrowedWorker.join(); + + t.IsTrue(finished, "W1: borrowed-worker WaitSema completed (no deadlock)"); + t.IsFalse(workerThrew.load(std::memory_order_acquire), "W1: borrowed worker did not throw"); + t.Equals(workerRet.load(std::memory_order_acquire), workSid, "W1: WaitSema returned workSid"); + + // The permit must have come from the fiber, not the escape hatch. + uint32_t signalled = 0u; + std::memcpy(&signalled, rdram.data() + kW1Signalled, 4); + t.Equals(signalled, 1u, "W1: the producer fiber produced the permit"); + + // No ThreadInfo must be created for the borrowed worker (tid -1). + { + std::lock_guard lk(g_thread_map_mutex); + t.Equals(static_cast(g_threads.count(-1)), 0, + "W1: borrowed worker created no g_threads[-1] entry"); + } + + const bool drained = waitUntil([&]() + { + return g_activeThreads.load(std::memory_order_acquire) <= 0; + }, std::chrono::milliseconds(1000)); + t.IsTrue(drained, "W1: producer fiber drained g_activeThreads"); + + deleteSchedSema(rdram.data(), &runtime, workSid); + }); + + // ------------------------------------------------------------------ + // W2: exit handler calls ExitThread — hook completes, no crash + // ------------------------------------------------------------------ + tc.Run("W2: exit handler that calls ExitThread terminates the fiber cleanly", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x007A8000u, &stepW2RegisterHandlerThenReturn); + runtime.registerFunction(0x007A8100u, &stepW2ExitHandlerCallsExitThread); + + rdramWrite32(rdram, kW2Sentinel, 0u); + rdramWrite32(rdram, kW2BodyRan, 0u); + + const int32_t tid = startSchedWorker(rdram.data(), &runtime, + 0x007A8000u, 10, 0x00500000u, 0x2000u); + t.IsTrue(tid > 0, "W2: worker fiber started"); + if (tid <= 0) + { + return; + } + + // The fiber runs its body (registers the handler, returns), then the trampoline + // drives on_fiber_exit -> our handler -> ExitThread (throws). The exit hook + // is wrapped in try/catch, so a throwing handler must not crash the process. + const bool drained = waitUntil([&]() + { + return g_activeThreads.load(std::memory_order_acquire) == 0; + }, std::chrono::milliseconds(3000)); + t.IsTrue(drained, "W2: fiber cleaned up despite ExitThread thrown from its exit handler"); + + uint32_t bodyRan = 0u, sentinel = 0u; + std::memcpy(&bodyRan, rdram.data() + kW2BodyRan, 4); + std::memcpy(&sentinel, rdram.data() + kW2Sentinel, 4); + t.Equals(bodyRan, 1u, "W2: fiber body ran and registered the exit handler"); + t.Equals(sentinel, 1u, "W2: exit handler ran up to the ExitThread call (sentinel written)"); + + }); + + // ------------------------------------------------------------------ + // W3: fiber that reblocks in a Mesa loop after shutdown wake -> no hang + // ------------------------------------------------------------------ + tc.Run("W3: shutdown terminates a fiber that would reblock after being woken", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x007B0000u, &stepW3MesaLoopOnSema); // fiber A + runtime.registerFunction(0x007B0100u, &stepW3HoldResource); // fiber B + + // count=0, max=2: never signalled, so both fibers stay blocked until shutdown. + const int32_t workSid = createSchedSema(rdram.data(), &runtime, 0, 2); + t.IsTrue(workSid > 0, "W3: work sema created"); + if (workSid <= 0) + { + return; + } + rdramWrite32(rdram, kW3WorkSid, static_cast(workSid)); + + const int32_t tidA = startSchedWorker(rdram.data(), &runtime, + 0x007B0000u, 10, 0x00504000u, 0x2000u); + const int32_t tidB = startSchedWorker(rdram.data(), &runtime, + 0x007B0100u, 10, 0x00508000u, 0x2000u); + t.IsTrue(tidA > 0 && tidB > 0, "W3: both fibers started"); + if (tidA <= 0 || tidB <= 0) + { + deleteSchedSema(rdram.data(), &runtime, workSid); + return; + } + + // Wait until both fibers are parked on the sema (Mesa loop is armed). + const bool blocked = waitUntil([&]() + { + return getSemaWaiters(rdram.data(), &runtime, workSid) >= 2; + }, std::chrono::milliseconds(2000)); + t.IsTrue(blocked, "W3: both fibers blocked on the count-0 sema"); + + // Shutdown. Fiber A's WaitSema loop, if woken, would re-check count (still 0) + // and re-block. block_current() returns immediately under g_stop && + // terminateRequested, WaitSema observes terminated and throws, so the + // Mesa loop exits and shutdown does NOT hang. + const auto t0 = std::chrono::steady_clock::now(); + ps2sched::scheduler_shutdown(); + const auto elapsed = std::chrono::steady_clock::now() - t0; + + t.IsTrue(elapsed < std::chrono::seconds(2), + "W3: scheduler_shutdown returned within 2s"); + t.Equals(g_activeThreads.load(), 0, "W3: all fibers terminated"); + + // Do NOT call scheduler_shutdown again (matches T15/U4). + deleteSchedSema(rdram.data(), &runtime, workSid); + }); + + }); // MiniTest::Case("SchedulerBorrowedWorker") +} + +// --------------------------------------------------------------------------- +// Scheduler window tests — publish/arm window and notify_all coverage (X1, X2) +// --------------------------------------------------------------------------- + +namespace +{ + // ----------------------------------------------------------------------- + // RDRAM slot constants for the SchedulerWindow suite (0x43E0–0x43F8 range) + // ----------------------------------------------------------------------- + static constexpr uint32_t kX1WorkSid = 0x000043E0u; // sid the waiter blocks on + static constexpr uint32_t kX1WakeCount = 0x000043E4u; // # times the waiter fiber woke and re-checked + static constexpr uint32_t kX1StopFlag = 0x000043E8u; // host sets 1 to tell the waiter loop to exit + static constexpr uint32_t kX1Exited = 0x000043ECu; // waiter writes 1 just before ctx->pc = 0 + static constexpr uint32_t kX2WorkSid = 0x000043F0u; // X2 sid the executor fiber blocks on + static constexpr uint32_t kX2WakeCount = 0x000043F4u; // X2 # times the fiber woke + static constexpr uint32_t kX2Exited = 0x000043F8u; // X2 waiter exit sentinel + + // Host <-> fiber mailboxes (see gSeq/... rationale above). kX1WorkSid/ + // kX2WorkSid are written once before the fiber starts and never touched + // concurrently, so they stay plain rdram words. + static std::atomic gX1WakeCount{0}; + static std::atomic gX1StopFlag{0}; + static std::atomic gX1Exited{0}; + static std::atomic gX2WakeCount{0}; + static std::atomic gX2Exited{0}; + + // ----------------------------------------------------------------------- + // X1 waiter: Mesa loop. WaitSema on kX1WorkSid; on each successful return + // bump kX1WakeCount, then loop again until the host sets kX1StopFlag. + // Writes kX1Exited=1 on the way out. Runs ON the executor thread (real fiber). + // ----------------------------------------------------------------------- + static void stepX1WindowWaiter(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + int32_t sid = 0; + std::memcpy(&sid, rdram + kX1WorkSid, sizeof(sid)); + + for (;;) + { + if (gX1StopFlag.load(std::memory_order_acquire) != 0u) + { + break; + } + + R5900Context wc{}; + setRegU32(wc, 4, static_cast(sid)); + ps2_syscalls::WaitSema(rdram, &wc, runtime); + const int32_t ret = getRegS32(wc, 2); + + // KE_WAIT_DELETE means the sema was deleted under us (shutdown path) — stop. + if (ret == KE_WAIT_DELETE) + { + break; + } + if (ret == sid) + { + gX1WakeCount.fetch_add(1u, std::memory_order_release); + } + // Loop: WaitSema again. count is back to 0 (we consumed the permit), so + // we re-publish to the waitList and re-arm — re-entering the publish/arm window. + } + + gX1Exited.store(1u, std::memory_order_release); + ctx->pc = 0u; + } + + // ----------------------------------------------------------------------- + // X2 fiber: Mesa loop on kX2WorkSid, bumping kX2WakeCount each successful + // wake, up to 64 times. Writes kX2Exited=1 on the way out. + // ----------------------------------------------------------------------- + static void stepX2ExecutorWaiter(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + int32_t sid = 0; + std::memcpy(&sid, rdram + kX2WorkSid, sizeof(sid)); + + for (uint32_t i = 0; i < 64u; ++i) // bounded: wake up to 64 times + { + R5900Context wc{}; + setRegU32(wc, 4, static_cast(sid)); + ps2_syscalls::WaitSema(rdram, &wc, runtime); + const int32_t ret = getRegS32(wc, 2); + if (ret == KE_WAIT_DELETE) + { + break; + } + if (ret == sid) + { + gX2WakeCount.fetch_add(1u, std::memory_order_release); + } + } + + gX2Exited.store(1u, std::memory_order_release); + ctx->pc = 0u; + } + +} // anonymous namespace + +// --------------------------------------------------------------------------- +// SchedulerWindow suite registration +// --------------------------------------------------------------------------- +void register_scheduler_window_tests() +{ + MiniTest::Case("SchedulerWindow", [](TestCase &tc) + { + // ------------------------------------------------------------------ + // X1: SignalSema in the publish/arm window is never lost + // ------------------------------------------------------------------ + tc.Run("X1: SignalSema in the publish/arm window is never lost", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x007C0000u, &stepX1WindowWaiter); + + // count=0, maxCount=1: the waiter must block; each SignalSema does a real + // 0->1 transition with a waiter pop and wakeup while the fiber is mid-park. + const int32_t sid = createSchedSema(rdram.data(), &runtime, 0, 1); + t.IsTrue(sid > 0, "X1: work sema created"); + if (sid <= 0) + { + return; + } + + rdramWrite32(rdram, kX1WorkSid, static_cast(sid)); + gX1WakeCount.store(0u, std::memory_order_release); + gX1StopFlag.store(0u, std::memory_order_release); + gX1Exited.store(0u, std::memory_order_release); + + const int32_t tid = startSchedWorker(rdram.data(), &runtime, + 0x007C0000u, 10, 0x00508000u, 0x2000u); + t.IsTrue(tid > 0, "X1: waiter fiber started"); + if (tid <= 0) + { + deleteSchedSema(rdram.data(), &runtime, sid); + return; + } + + // NOTE: deliberately do NOT wait for getSemaWaiters(sid) >= 1. We start + // hammering immediately so signals land inside the publish/arm window. + std::atomic signalerThrew{false}; + std::atomic signalsSent{0u}; + + std::thread signaler([&]() + { + try + { + const auto deadline = + std::chrono::steady_clock::now() + std::chrono::milliseconds(150); + uint32_t iters = 0u; + while (std::chrono::steady_clock::now() < deadline && iters < 200000u) + { + R5900Context sc{}; + setRegU32(sc, 4, static_cast(sid)); + ps2_syscalls::SignalSema(rdram.data(), &sc, &runtime); + const int32_t sret = getRegS32(sc, 2); + signalsSent.fetch_add(1, std::memory_order_relaxed); + + // If the permit is still outstanding (waiter has not consumed it), + // drain it so the next SignalSema is a fresh 0->1 transition. + if (sret == sid) + { + R5900Context pc{}; + setRegU32(pc, 4, static_cast(sid)); + ps2_syscalls::PollSema(rdram.data(), &pc, &runtime); + } + ++iters; + } + } + catch (...) + { + signalerThrew.store(true, std::memory_order_release); + } + }); + + // PRIMARY ASSERTION: the waiter must wake at least once. Under the broken + // impl the first in-window signal is lost and the waiter parks forever, so + // kX1WakeCount stays 0 and this times out (caught here, not a hang). + const bool woke = waitUntil([&]() + { + return gX1WakeCount.load(std::memory_order_acquire) >= 1u; + }, std::chrono::milliseconds(3000)); + t.IsTrue(woke, "X1: waiter fiber woke at least once (no lost in-window wakeup)"); + + if (signaler.joinable()) + { + signaler.join(); + } + t.IsFalse(signalerThrew.load(std::memory_order_acquire), "X1: signaler did not throw"); + t.IsTrue(signalsSent.load(std::memory_order_relaxed) > 0u, "X1: signaler ran"); + + // Tell the waiter to exit its Mesa loop, then push one final permit so a + // waiter parked between iterations wakes, sees the stop flag, and returns. + gX1StopFlag.store(1u, std::memory_order_release); + { + R5900Context sc{}; + setRegU32(sc, 4, static_cast(sid)); + ps2_syscalls::SignalSema(rdram.data(), &sc, &runtime); + } + + const bool exited = waitUntil([&]() + { + return gX1Exited.load(std::memory_order_acquire) == 1u; + }, std::chrono::milliseconds(3000)); + t.IsTrue(exited, "X1: waiter fiber observed stop flag and exited"); + + const bool drained = waitUntil([&]() + { + return g_activeThreads.load(std::memory_order_acquire) <= 0; + }, std::chrono::milliseconds(3000)); + t.IsTrue(drained, "X1: waiter fiber drained g_activeThreads"); + + // Reported for visibility; not asserted as a hard number (cooperative + // scheduling means we cannot wake on every one of thousands of signals). + { + const uint32_t n = gX1WakeCount.load(std::memory_order_acquire); + std::cout << " [X1] wakes=" << n + << " signals=" << signalsSent.load(std::memory_order_relaxed) + << std::endl; + } + + deleteSchedSema(rdram.data(), &runtime, sid); + }); + + // ------------------------------------------------------------------ + // X2: notify_all wakes the executor under host-worker CV contention + // ------------------------------------------------------------------ + tc.Run("X2: notify_all wakes the executor under host-worker CV contention", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x007C8000u, &stepX2ExecutorWaiter); + + const int32_t sid = createSchedSema(rdram.data(), &runtime, 0, 1); + t.IsTrue(sid > 0, "X2: work sema created"); + if (sid <= 0) + { + return; + } + rdramWrite32(rdram, kX2WorkSid, static_cast(sid)); + gX2WakeCount.store(0u, std::memory_order_release); + gX2Exited.store(0u, std::memory_order_release); + + const int32_t tid = startSchedWorker(rdram.data(), &runtime, + 0x007C8000u, 10, 0x0050C000u, 0x2000u); + t.IsTrue(tid > 0, "X2: executor fiber started"); + if (tid <= 0) + { + deleteSchedSema(rdram.data(), &runtime, sid); + return; + } + + // Wait until the fiber is parked so the first signal targets a real waiter. + waitUntil([&]() { return getSemaWaiters(rdram.data(), &runtime, sid) >= 1; }, + std::chrono::milliseconds(1000)); + + std::atomic stopWorkers{false}; + std::atomic workerThrew{false}; + + // Host workers that contend on g_sched_cv via async_guest_begin/end. + constexpr int kNumWorkers = 3; + std::vector workers; + workers.reserve(kNumWorkers); + for (int w = 0; w < kNumWorkers; ++w) + { + workers.emplace_back([&]() + { + g_currentThreadId = -1; // borrowed host worker + try + { + while (!stopWorkers.load(std::memory_order_acquire)) + { + ps2sched::async_guest_begin(); + ps2sched::async_guest_end(); + } + } + catch (...) + { + workerThrew.store(true, std::memory_order_release); + } + }); + } + + // Signaler: drive make_ready -> notify on g_sched_cv while workers contend. + std::atomic signalerThrew{false}; + std::thread signaler([&]() + { + try + { + for (uint32_t i = 0; i < 64u; ++i) + { + // Each successful wake must make it across the CV to the executor. + // Wait (bounded) until the fiber's wake count catches up so we + // exercise one notify at a time against live CV contention. + R5900Context sc{}; + setRegU32(sc, 4, static_cast(sid)); + ps2_syscalls::SignalSema(rdram.data(), &sc, &runtime); + + const uint32_t target = i + 1u; + const bool advanced = waitUntil([&]() + { + return gX2WakeCount.load(std::memory_order_acquire) >= target; + }, std::chrono::milliseconds(200)); + if (!advanced) + { + break; // executor was stranded — leave count short; asserted below + } + } + } + catch (...) + { + signalerThrew.store(true, std::memory_order_release); + } + }); + + if (signaler.joinable()) + { + signaler.join(); + } + stopWorkers.store(true, std::memory_order_release); + for (auto &w : workers) + { + if (w.joinable()) + { + w.join(); + } + } + + t.IsFalse(workerThrew.load(std::memory_order_acquire), "X2: host workers did not throw"); + t.IsFalse(signalerThrew.load(std::memory_order_acquire), "X2: signaler did not throw"); + + // PRIMARY: every signal reached the executor and woke the fiber. Under a + // notify_one regression a notification can be consumed by a parked host + // worker instead of the executor, stranding the fiber; the wake count then + // stalls below 64 and this fails. (Probabilistic: depends on the OS handing + // a notify_one to a worker — see plan caveat.) + { + const uint32_t n = gX2WakeCount.load(std::memory_order_acquire); + t.Equals(n, 64u, "X2: all 64 signals reached the executor fiber (notify_all)"); + } + + const bool exited = waitUntil([&]() + { + return gX2Exited.load(std::memory_order_acquire) == 1u; + }, std::chrono::milliseconds(3000)); + t.IsTrue(exited, "X2: executor fiber completed its 64-wake loop and exited"); + + const bool drained = waitUntil([&]() + { + return g_activeThreads.load(std::memory_order_acquire) <= 0; + }, std::chrono::milliseconds(3000)); + t.IsTrue(drained, "X2: executor fiber drained g_activeThreads"); + + deleteSchedSema(rdram.data(), &runtime, sid); + }); + + }); // MiniTest::Case("SchedulerWindow") +} + +// --------------------------------------------------------------------------- +// Y-suite: four new scheduler correctness tests (Y1–Y4). +// RDRAM scratch block: 0x4400–0x44FF (does not overlap existing 0x4000–0x43FF). +// --------------------------------------------------------------------------- + +namespace +{ + // ----------------------------------------------------------------------- + // RDRAM slot constants for the Y-suite + // ----------------------------------------------------------------------- + + // Y1: SleepThread Mesa loop + static constexpr uint32_t kY1ExitedSleep = 0x00004400u; // uint32: 1 after SleepThread returns + static constexpr uint32_t kY1SleepRet = 0x00004404u; // int32 : SleepThread return value + + // Y4a: DeleteSema drain + static constexpr uint32_t kY4WorkSid = 0x00004408u; // int32 : sema id for Y4a waiters + static constexpr uint32_t kY4RetBase = 0x0000440Cu; // int32[4]: WaitSema return values per fiber + + // Y4b: stale-token rejection + static constexpr uint32_t kY4bTokenLo = 0x00004420u; // uint32: low 32 bits of fiber token + static constexpr uint32_t kY4bTokenHi = 0x00004424u; // uint32: high 32 bits of fiber token + + // Host <-> fiber mailboxes (see gSeq/... rationale above). + static std::atomic gY1ExitedSleep{0}; + static std::atomic gY1SleepRet{0}; + static std::atomic gY4bTokenLo{0}; + static std::atomic gY4bTokenHi{0}; + + // ----------------------------------------------------------------------- + // Y1 step function: SleepThread, then record exit sentinel and return value + // ----------------------------------------------------------------------- + static void stepY1SleepThenRecord(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + R5900Context sc{}; + ps2_syscalls::SleepThread(rdram, &sc, runtime); // blocks until a genuine WakeupThread + // Reached only when SleepThread truly returns (consumes a wakeupCount permit). + const int32_t ret = getRegS32(sc, 2); + gY1SleepRet.store(ret, std::memory_order_relaxed); + gY1ExitedSleep.store(1u, std::memory_order_release); + ctx->pc = 0u; + } + + // ----------------------------------------------------------------------- + // Y4a step function: WaitSema(sid from rdram), record return in an atomic slot + // ----------------------------------------------------------------------- + static std::atomic gY4SlotCounter{0}; + + static void stepY4WaitSemaRecordRet(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + int32_t sid = 0; + std::memcpy(&sid, rdram + kY4WorkSid, 4); + + R5900Context sc{}; + setRegU32(sc, 4, static_cast(sid)); + ps2_syscalls::WaitSema(rdram, &sc, runtime); // blocks until sema is deleted + int32_t ret = getRegS32(sc, 2); + + // Claim a result slot atomically. + const int slot = gY4SlotCounter.fetch_add(1, std::memory_order_relaxed); + if (slot >= 0 && slot < 4) + { + std::memcpy(rdram + kY4RetBase + static_cast(slot) * 4u, &ret, 4); + } + + ctx->pc = 0u; + } + + // ----------------------------------------------------------------------- + // Y4b step function: publish current_fiber_token() into rdram, then exit + // ----------------------------------------------------------------------- + static void stepY4bPublishTokenThenExit(uint8_t *rdram, R5900Context *ctx, PS2Runtime * /*runtime*/) + { + const uint64_t tok = ps2sched::current_fiber_token(); + const uint32_t lo = static_cast(tok & 0xFFFFFFFFu); + const uint32_t hi = static_cast(tok >> 32u); + gY4bTokenLo.store(lo, std::memory_order_relaxed); + gY4bTokenHi.store(hi, std::memory_order_release); + ctx->pc = 0u; + } + + // ----------------------------------------------------------------------- + // Z1: tid-reuse regression + // + // RDRAM slot: kZ1Done (0x00004450). Each fiber increments this on exit. + // ----------------------------------------------------------------------- + static constexpr uint32_t kZ1Done = 0x00004450u; // uint32: incremented by each completing fiber + + static void stepZ1QuickExit(uint8_t *rdram, R5900Context *ctx, PS2Runtime * /*runtime*/) + { + uint32_t n = 0u; + std::memcpy(&n, rdram + kZ1Done, 4); + ++n; + std::memcpy(rdram + kZ1Done, &n, 4); + ctx->pc = 0u; + } + +} // anonymous namespace + +// --------------------------------------------------------------------------- +// Y1 — SleepThread Mesa loop: ResumeThread must NOT count as a wakeup +// --------------------------------------------------------------------------- +void register_scheduler_sleep_resume_tests() +{ + MiniTest::Case("SchedulerSleepResume", [](TestCase &tc) + { + tc.Run("sleeping-and-suspended fiber re-parks after ResumeThread and only exits on WakeupThread", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x007D0000u, &stepY1SleepThenRecord); + + // Initialize sentinels so any premature write is detectable. + gY1ExitedSleep.store(0u, std::memory_order_release); + gY1SleepRet.store(-9999, std::memory_order_relaxed); + + const int32_t tid = startSchedWorker(rdram.data(), &runtime, + 0x007D0000u, 10, 0x00510000u, 0x2000u); + t.IsTrue(tid > 0, "Y1: sleeping fiber started"); + if (tid <= 0) + { + return; + } + + // Step 1: confirm the fiber is inside SleepThread (THS_WAIT / TSW_SLEEP). + const bool sleeping = waitUntil([&]() + { + int32_t status = 0, wt = 0; + getThreadStatus(rdram, runtime, tid, status, wt); + return status == kTHSWait && wt == kTSWSleep; + }, std::chrono::milliseconds(500)); + t.IsTrue(sleeping, "Y1: fiber entered SleepThread"); + if (!sleeping) + { + return; + } + + // Step 2: suspend the sleeping fiber; it must transition to THS_WAITSUSPEND. + { + R5900Context sc{}; + setRegU32(sc, 4, static_cast(tid)); + ps2_syscalls::SuspendThread(rdram.data(), &sc, &runtime); + t.Equals(getRegS32(sc, 2), KE_OK, "Y1: SuspendThread returned KE_OK"); + } + + const bool suspended = waitUntil([&]() + { + int32_t status = 0, wt = 0; + getThreadStatus(rdram, runtime, tid, status, wt); + return status == kTHSWaitSuspend; + }, std::chrono::milliseconds(500)); + t.IsTrue(suspended, "Y1: fiber is sleeping AND suspended (THS_WAITSUSPEND)"); + + // Step 3: ResumeThread clears the suspend gate but provides NO wakeupCount permit. + // The Mesa loop inside SleepThread must re-park. + { + R5900Context sc{}; + setRegU32(sc, 4, static_cast(tid)); + ps2_syscalls::ResumeThread(rdram.data(), &sc, &runtime); + t.Equals(getRegS32(sc, 2), KE_OK, "Y1: ResumeThread returned KE_OK"); + } + + // Step 4: wait generously, then assert the fiber did NOT exit SleepThread. + std::this_thread::sleep_for(std::chrono::milliseconds(60)); + + { + const uint32_t exited = gY1ExitedSleep.load(std::memory_order_acquire); + t.Equals(exited, 0u, + "Y1: fiber did not exit SleepThread after ResumeThread (no spurious wake)"); + } + + // Also confirm the fiber re-asserted THS_WAIT/TSW_SLEEP (re-parked in sleep). + { + int32_t status = 0, wt = 0; + getThreadStatus(rdram, runtime, tid, status, wt); + t.IsTrue(status == kTHSWait && wt == kTSWSleep, + "Y1: fiber re-parked in THS_SLEEP after ResumeThread"); + } + + // Step 5: a genuine WakeupThread must now release the fiber from SleepThread. + { + R5900Context sc{}; + setRegU32(sc, 4, static_cast(tid)); + ps2_syscalls::WakeupThread(rdram.data(), &sc, &runtime); + t.Equals(getRegS32(sc, 2), KE_OK, "Y1: WakeupThread returned KE_OK"); + } + + const bool woke = waitUntil([&]() + { + return gY1ExitedSleep.load(std::memory_order_acquire) == 1u; + }, std::chrono::milliseconds(1000)); + t.IsTrue(woke, "Y1: fiber exited SleepThread after WakeupThread"); + + { + const int32_t sleepRet = gY1SleepRet.load(std::memory_order_acquire); + t.Equals(sleepRet, KE_OK, "Y1: SleepThread returned KE_OK on genuine wakeup"); + } + + waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + }); + + }); // MiniTest::Case("SchedulerSleepResume") +} + +// --------------------------------------------------------------------------- +// Y2 — scheduler_shutdown promptly unwinds a running, back-edge-bearing fiber +// --------------------------------------------------------------------------- +void register_scheduler_shutdown_clean_tests() +{ + MiniTest::Case("SchedulerShutdownClean", [](TestCase &tc) + { + tc.Run("scheduler_shutdown terminates a non-blocking running fiber promptly", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x007D8000u, &stepProgressLoop); + + // Initialize: kStopFlag is intentionally left 0 so the only exit is the shutdown unwind. + gStartedFlag.store(0u, std::memory_order_release); + gProgressCtr.store(0u, std::memory_order_release); + gStopFlag.store(0u, std::memory_order_release); + + const int32_t tid = startSchedWorker(rdram.data(), &runtime, + 0x007D8000u, 10, 0x00514000u, 0x2000u); + t.IsTrue(tid > 0, "Y2: progress fiber started"); + if (tid <= 0) + { + return; + } + + // Step 1: confirm the fiber is actively looping (Running, not blocked). + const bool started = waitUntil([&]() + { + return gStartedFlag.load(std::memory_order_acquire) != 0u; + }, std::chrono::milliseconds(500)); + t.IsTrue(started, "Y2: fiber entered its loop"); + + const uint32_t ctr0 = gProgressCtr.load(std::memory_order_acquire); + const bool progressing = waitUntil([&]() + { + return gProgressCtr.load(std::memory_order_acquire) > ctr0; + }, std::chrono::milliseconds(500)); + t.IsTrue(progressing, "Y2: fiber is actively looping (progress counter advancing)"); + + // Step 2: shut down and measure elapsed time. + // kStopFlag remains 0; the only exit is the back-edge-triggered terminate throw. + const auto t0 = std::chrono::steady_clock::now(); + ps2sched::scheduler_shutdown(); + const auto elapsed = std::chrono::steady_clock::now() - t0; + + // Step 3: shutdown must be prompt and complete. + t.IsTrue(elapsed < std::chrono::milliseconds(500), + "Y2: scheduler_shutdown returned promptly (< 500 ms)"); + t.Equals(g_activeThreads.load(), 0, + "Y2: g_activeThreads is 0 after shutdown (fiber unwound)"); + + // Do NOT call scheduler_shutdown again (mirrors T15/W3/U4). + }); + + }); // MiniTest::Case("SchedulerShutdownClean") +} + +// --------------------------------------------------------------------------- +// Y3 — Borrowed-worker self-targeting syscalls return KE_ILLEGAL_THID, no g_threads[-1] +// --------------------------------------------------------------------------- +void register_scheduler_borrowed_guard_tests() +{ + MiniTest::Case("SchedulerBorrowedGuard", [](TestCase &tc) + { + tc.Run("self-targeting syscalls from a borrowed worker return KE_ILLEGAL_THID and do not corrupt g_threads", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + // Baseline: no -1 entry should exist at the start. + { + std::lock_guard lk(g_thread_map_mutex); + t.Equals(static_cast(g_threads.count(-1)), 0, + "Y3: no g_threads[-1] entry at baseline"); + } + + // Each case: a self-targeting syscall (a0=0 means TH_SELF) called from a + // borrowed host worker (g_currentThreadId == -1). + struct SyscallCase + { + const char *name; + void (*fn)(uint8_t *, R5900Context *, PS2Runtime *); + uint32_t a0; // $a0 + uint32_t a1; // $a1 + }; + + const SyscallCase cases[] = { + { "SuspendThread(0)", &ps2_syscalls::SuspendThread, 0u, 0u }, + { "ResumeThread(0)", &ps2_syscalls::ResumeThread, 0u, 0u }, + { "ChangeThreadPriority(0)", &ps2_syscalls::ChangeThreadPriority, 0u, 5u }, + { "CancelWakeupThread(0)", &ps2_syscalls::CancelWakeupThread, 0u, 0u }, + { "RotateThreadReadyQueue(0)", &ps2_syscalls::RotateThreadReadyQueue, 0u, 0u }, + { "ReferThreadStatus(0)", &ps2_syscalls::ReferThreadStatus, 0u, 0u }, + { "TerminateThread(0)", &ps2_syscalls::TerminateThread, 0u, 0u }, + }; + + for (const auto &c : cases) + { + std::atomic retVal{-1234}; + std::atomic threw{false}; + + std::thread worker([&]() + { + try + { + g_currentThreadId = -1; // borrowed host worker: no PS2 thread identity + ps2sched::async_guest_begin(); + R5900Context ctx{}; + setRegU32(ctx, 4, c.a0); // $a0 = 0 (TH_SELF / self-target) + setRegU32(ctx, 5, c.a1); // $a1 = extra arg (e.g. new priority) + c.fn(rdram.data(), &ctx, &runtime); + retVal.store(getRegS32(ctx, 2), std::memory_order_release); + ps2sched::async_guest_end(); + } + catch (...) + { + threw.store(true, std::memory_order_release); + ps2sched::async_guest_end(); + } + }); + worker.join(); + + t.IsFalse(threw.load(std::memory_order_acquire), + std::string(c.name) + ": borrowed-worker syscall must not throw"); + t.Equals(retVal.load(std::memory_order_acquire), KE_ILLEGAL_THID, + std::string(c.name) + ": must return KE_ILLEGAL_THID for self-target from borrowed worker"); + { + std::lock_guard lk(g_thread_map_mutex); + t.Equals(static_cast(g_threads.count(-1)), 0, + std::string(c.name) + ": no g_threads[-1] entry created"); + } + } + + }); + + }); // MiniTest::Case("SchedulerBorrowedGuard") +} + +// --------------------------------------------------------------------------- +// Y4 — Sema/event delete drains all waiters via generation-validated wakeups +// --------------------------------------------------------------------------- +void register_scheduler_sema_delete_tests() +{ + MiniTest::Case("SchedulerSemaDelete", [](TestCase &tc) + { + // ------------------------------------------------------------------ + // Y4a: N waiters on one sema all receive KE_WAIT_DELETE and exit cleanly + // ------------------------------------------------------------------ + tc.Run("Y4a: DeleteSema wakes all N waiters with KE_WAIT_DELETE via pair-based drain", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x007E0000u, &stepY4WaitSemaRecordRet); + + constexpr int kN = 4; + + // Create the sema with init=0 so all N fibers block. + const int32_t workSid = createSchedSema(rdram.data(), &runtime, 0, kN + 1); + t.IsTrue(workSid > 0, "Y4a: work sema created"); + if (workSid <= 0) + { + return; + } + + // Publish the sema id and reset result slots before starting fibers. + { + int32_t v = workSid; + std::memcpy(rdram.data() + kY4WorkSid, &v, 4); + } + for (int i = 0; i < kN; ++i) + { + int32_t bad = -9999; + std::memcpy(rdram.data() + kY4RetBase + static_cast(i) * 4u, &bad, 4); + } + gY4SlotCounter.store(0, std::memory_order_relaxed); + + // Start N fibers, each of which calls WaitSema(workSid). + constexpr uint32_t kStackBase = 0x00518000u; + constexpr uint32_t kStackSize = 0x2000u; + int32_t tids[kN] = {}; + for (int i = 0; i < kN; ++i) + { + const uint32_t stackAddr = kStackBase + static_cast(i) * kStackSize; + tids[i] = startSchedWorker(rdram.data(), &runtime, + 0x007E0000u, 10, stackAddr, kStackSize); + t.IsTrue(tids[i] > 0, "Y4a: fiber started"); + } + + // Wait until all N fibers are blocked on the sema. + const bool allBlocked = waitUntil([&]() + { + return getSemaWaiters(rdram.data(), &runtime, workSid) >= kN; + }, std::chrono::milliseconds(2000)); + t.IsTrue(allBlocked, "Y4a: all N fibers blocked on the sema"); + if (!allBlocked) + { + return; + } + + // DeleteSema: must drain the pair-based waitList and wake all N with KE_WAIT_DELETE. + { + R5900Context dc{}; + setRegU32(dc, 4, static_cast(workSid)); + ps2_syscalls::DeleteSema(rdram.data(), &dc, &runtime); + t.Equals(getRegS32(dc, 2), workSid, "Y4a: DeleteSema returned workSid"); + } + + // All N fibers must finish. + const bool drained = waitUntil([&]() + { + return g_activeThreads.load(std::memory_order_acquire) == 0; + }, std::chrono::milliseconds(2000)); + t.IsTrue(drained, "Y4a: all N fibers exited after DeleteSema"); + + // Every recorded return value must be KE_WAIT_DELETE. + for (int i = 0; i < kN; ++i) + { + int32_t ret = -9999; + std::memcpy(&ret, rdram.data() + kY4RetBase + static_cast(i) * 4u, 4); + t.Equals(ret, KE_WAIT_DELETE, + std::string("Y4a: slot ") + std::to_string(i) + " WaitSema returned KE_WAIT_DELETE"); + } + + }); + + // ------------------------------------------------------------------ + // Y4b: stale generation token is rejected by enqueue_external_wakeup_validated + // ------------------------------------------------------------------ + tc.Run("Y4b: enqueue_external_wakeup_validated with a wrong token does not wake the fiber", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x007E8000u, &stepY4bPublishTokenThenExit); + runtime.registerFunction(0x007F0000u, &stepY1SleepThenRecord); + + // Phase 1: start a short-lived fiber that publishes its token and exits. + gY4bTokenLo.store(0u, std::memory_order_release); + gY4bTokenHi.store(0u, std::memory_order_release); + + const int32_t tokenFiberTid = startSchedWorker(rdram.data(), &runtime, + 0x007E8000u, 10, 0x00528000u, 0x2000u); + t.IsTrue(tokenFiberTid > 0, "Y4b: token-publishing fiber started"); + if (tokenFiberTid <= 0) + { + return; + } + + // Wait for the token fiber to write its token and exit. + const bool tokenWritten = waitUntil([&]() + { + const uint32_t lo = gY4bTokenLo.load(std::memory_order_acquire); + const uint32_t hi = gY4bTokenHi.load(std::memory_order_acquire); + return lo != 0u || hi != 0u; + }, std::chrono::milliseconds(1000)); + t.IsTrue(tokenWritten, "Y4b: token fiber published its token"); + + waitUntil([&](){ return g_activeThreads.load() == 0; }, + std::chrono::milliseconds(1000)); + + const uint32_t tokenLo = gY4bTokenLo.load(std::memory_order_acquire); + const uint32_t tokenHi = gY4bTokenHi.load(std::memory_order_acquire); + const uint64_t capturedToken = + (static_cast(tokenHi) << 32u) | static_cast(tokenLo); + + // Phase 2: start a new sleeping fiber (reusing stepY1SleepThenRecord). + // Get its CURRENT token, then fabricate a WRONG token by flipping a generation bit. + gY1ExitedSleep.store(0u, std::memory_order_release); + gY1SleepRet.store(-9999, std::memory_order_relaxed); + + const int32_t sleeperTid = startSchedWorker(rdram.data(), &runtime, + 0x007F0000u, 10, 0x0052C000u, 0x2000u); + t.IsTrue(sleeperTid > 0, "Y4b: sleeper fiber started"); + if (sleeperTid <= 0) + { + return; + } + + // Wait until the sleeper is truly parked in SleepThread. + const bool sleeperSleeping = waitUntil([&]() + { + int32_t status = 0, wt = 0; + getThreadStatus(rdram, runtime, sleeperTid, status, wt); + return status == kTHSWait && wt == kTSWSleep; + }, std::chrono::milliseconds(500)); + t.IsTrue(sleeperSleeping, "Y4b: sleeper fiber is parked in SleepThread"); + if (!sleeperSleeping) + { + // Kick the sleeper out so we can clean up. + R5900Context sc{}; + setRegU32(sc, 4, static_cast(sleeperTid)); + ps2_syscalls::WakeupThread(rdram.data(), &sc, &runtime); + waitUntil([&](){ return g_activeThreads.load() == 0; }, + std::chrono::milliseconds(1000)); + return; + } + + // Fabricate a wrong token: flip one bit in the high word (generation part). + // This guarantees a generation mismatch regardless of the actual token layout. + const uint64_t wrongToken = capturedToken ^ (uint64_t(1u) << 32u); + + // Phase 3: fire the stale (wrong) token wake from a foreign host thread. + // The scheduler must detect the generation mismatch and drop it. + std::thread foreignWaker([&]() + { + ps2sched::enqueue_external_wakeup_validated(sleeperTid, wrongToken); + }); + foreignWaker.join(); + + // Phase 4: wait and assert the sleeper was NOT woken (stale token rejected). + std::this_thread::sleep_for(std::chrono::milliseconds(60)); + { + const uint32_t exited = gY1ExitedSleep.load(std::memory_order_acquire); + t.Equals(exited, 0u, + "Y4b: sleeper was not woken by the wrong-token wakeup (stale token rejected)"); + } + { + int32_t status = 0, wt = 0; + getThreadStatus(rdram, runtime, sleeperTid, status, wt); + t.IsTrue(status == kTHSWait && wt == kTSWSleep, + "Y4b: sleeper is still in THS_SLEEP after wrong-token wakeup"); + } + + // Sanity: a valid WakeupThread DOES reach the sleeper, proving it was genuinely + // reachable and the stale wake was specifically rejected (not an inert no-op). + { + R5900Context sc{}; + setRegU32(sc, 4, static_cast(sleeperTid)); + ps2_syscalls::WakeupThread(rdram.data(), &sc, &runtime); + t.Equals(getRegS32(sc, 2), KE_OK, "Y4b: WakeupThread returned KE_OK"); + } + + const bool woke = waitUntil([&]() + { + return gY1ExitedSleep.load(std::memory_order_acquire) == 1u; + }, std::chrono::milliseconds(1000)); + t.IsTrue(woke, "Y4b: sleeper woke normally after a valid WakeupThread"); + + waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + }); + + }); // MiniTest::Case("SchedulerSemaDelete") +} + +// --------------------------------------------------------------------------- +// Z1 — Borrowed-worker StartThread concurrent with executor Finished teardown +// +// Regression for the use-after-free where g_fiber_map.erase(tid) after +// re-acquiring g_sched_mutex could destroy a freshly-created FiberContext if +// a borrowed worker recycled the tid during the munmap window between the +// unlock and re-lock in the Finished branch. The fix erases the entry before +// dropping the lock so the tid is safe to reuse while the stack is being freed. +// --------------------------------------------------------------------------- +void register_scheduler_tid_reuse_tests() +{ + MiniTest::Case("SchedulerTidReuse", [](TestCase &tc) + { + // Z1: A borrowed host worker (g_currentThreadId == -1) creates and starts + // fibers in a tight loop while the executor is simultaneously tearing down + // just-exited fibers. Each fiber increments kZ1Done and exits; the worker + // deletes the dormant thread after each cycle so CreateThread can recycle + // the tid. If the executor ever erases a freshly-created FiberContext, + // the new fiber never runs and the final count falls short. + tc.Run("Z1: borrowed worker StartThread during Finished teardown does not destroy new fiber", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x007B0000u, &stepZ1QuickExit); + + constexpr int kCycles = 30; + std::atomic workerDone{false}; + std::atomic workerThrew{false}; + + // All fibers share one stack (0x00510000, 0x2000 bytes); each cycle waits + // for the previous fiber to exit before starting the next one. + std::thread borrowedWorker([&]() + { + try + { + g_currentThreadId = -1; // simulate IRQ/alarm borrowed host worker + + for (int i = 0; i < kCycles; ++i) + { + // Create and start the next fiber while holding the guest token. + ps2sched::async_guest_begin(); + const int32_t tid = startSchedWorker(rdram.data(), &runtime, + 0x007B0000u, 32, + 0x00510000u, 0x2000u); + ps2sched::async_guest_end(); + + if (tid <= 0) + { + break; // startSchedWorker failed — assertion below will catch it + } + + // Wait for the executor to run the fiber to completion (outside + // the guest scope so the executor can hold the token). + const bool exited = waitUntil( + [&](){ return g_activeThreads.load() == 0; }, + std::chrono::milliseconds(500)); + if (!exited) + { + break; // timed out — assertion below will catch it + } + + // Delete the dormant thread so CreateThread can reuse the tid + // on the next iteration, exercising the same-tid recycle path. + ps2sched::async_guest_begin(); + R5900Context dc{}; + setRegU32(dc, 4, static_cast(tid)); + ps2_syscalls::DeleteThread(rdram.data(), &dc, &runtime); + ps2sched::async_guest_end(); + } + } + catch (...) + { + workerThrew.store(true, std::memory_order_release); + } + workerDone.store(true, std::memory_order_release); + }); + + const bool done = waitUntil([&](){ return workerDone.load(); }, + std::chrono::milliseconds(20000)); + if (borrowedWorker.joinable()) + { + borrowedWorker.join(); + } + + t.IsTrue(done, "Z1: borrowed worker completed all cycles within 20s"); + t.IsFalse(workerThrew.load(), "Z1: borrowed worker did not throw"); + + uint32_t completions = 0u; + std::memcpy(&completions, rdram.data() + kZ1Done, 4); + t.Equals(static_cast(completions), kCycles, + "Z1: all fibers ran to completion (kZ1Done == kCycles)"); + + waitUntil([&](){ return g_activeThreads.load() == 0; }, + std::chrono::milliseconds(1000)); + }); + + }); // MiniTest::Case("SchedulerTidReuse") +} + +// --------------------------------------------------------------------------- +// AA tests — EventFlag mode tests (AA1, AA2, AA4, AA13) +// --------------------------------------------------------------------------- + +// Forward-declare PS2Fiber and the ps2fiber_alloc / ps2fiber_free / ps2fiber_current +// functions for AA7 and AA11. These are defined in the ps2_fiber backend (linked +// into ps2_runtime) but are not exposed through any public header; the test binary +// links against ps2_runtime so the symbols are available at link time. +// ps2fiber_on_executor_thread() is already forward-declared above, in the +// SchedulerLifecycle section, using the same pattern. +struct PS2Fiber; +PS2Fiber* ps2fiber_alloc(void (*fn)(void*), void* arg, size_t stack_bytes); +void ps2fiber_free(PS2Fiber* f); +PS2Fiber* ps2fiber_current(); + +namespace +{ + // ----------------------------------------------------------------------- + // AA RDRAM slot constants (beyond 0x00004460 to avoid collision) + // ----------------------------------------------------------------------- + static constexpr uint32_t kAASlotEid = 0x00004500u; // int32: event flag id + static constexpr uint32_t kAASlotResBits = 0x00004504u; // uint32: result bits from WaitEventFlag + static constexpr uint32_t kAASlotResult = 0x00004508u; // int32: WaitEventFlag return value + static constexpr uint32_t kAASeqAddr = 0x0000450Cu; // uint32: completion sequence counter + static constexpr uint32_t kAAStartedFlag = 0x00004510u; // uint32: set by fiber after arm_park + static constexpr uint32_t kAAWokenInWindow = 0x00004514u; // uint32: set by fiber if WokenInWindow + static constexpr uint32_t kAAFiberPtrSlot = 0x00004518u; // two uint32 words: lo/hi of PS2Fiber* ptr + + // gAASeq/gAAStarted/gAAWoken mirror kAASeqAddr/kAAStartedFlag/kAAWokenInWindow + // as atomics with acquire/release ordering (same rationale as gSeq/gStartedFlag/ + // gStopFlag/gProgressCtr above): these are polled across the host test thread / + // guest fiber thread boundary, so a plain rdram memcpy poll is a data race even + // though only one fiber ever runs at a time. gAASeq is incremented via fetch_add + // (not load()+store()) for the same release-sequence reason documented on gSeq's + // declaration above -- some AA tests (e.g. AA8) have more than one fiber bump it. + static std::atomic gAASeq{0}; + static std::atomic gAAStarted{0}; + static std::atomic gAAWoken{0}; + + // Helper: read currBits from ReferEventFlagStatus (offset 12 in Ps2EventFlagInfo). + static uint32_t getEvfCurrBits(std::vector &rdram, PS2Runtime &runtime, int eid) + { + R5900Context ctx{}; + setRegU32(ctx, 4, static_cast(eid)); + setRegU32(ctx, 5, kReferScratch); + ps2_syscalls::ReferEventFlagStatus(rdram.data(), &ctx, &runtime); + uint32_t bits = 0; + std::memcpy(&bits, rdram.data() + kReferScratch + 12, 4); + return bits; + } + + // ----------------------------------------------------------------------- + // AA1: WaitEventFlag OR-mode — unblocks on first matching bit + // ----------------------------------------------------------------------- + static void stepWaitEvfOrRecord(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + int32_t eid = 0; + std::memcpy(&eid, rdram + kAASlotEid, 4); + R5900Context sc{}; + setRegU32(sc, 4, static_cast(eid)); + setRegU32(sc, 5, 0x3u); // waitBits = 0x3 + setRegU32(sc, 6, 0x1u); // mode = WEF_OR + setRegU32(sc, 7, kAASlotResBits); + ps2_syscalls::WaitEventFlag(rdram, &sc, runtime); + int32_t ret = getRegS32(sc, 2); + std::memcpy(rdram + kAASlotResult, &ret, 4); + const uint32_t seq = gAASeq.load(std::memory_order_relaxed); + gAASeq.fetch_add(1u, std::memory_order_release); // RMW: preserves release-sequence chaining (see gSeq/gAASeq declaration above) + ctx->pc = 0u; + } + + // ----------------------------------------------------------------------- + // AA2: WaitEventFlag AND-mode with WEF_CLEAR_ALL — clears all bits on exit + // ----------------------------------------------------------------------- + static void stepWaitEvfClearAllRecord(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + int32_t eid = 0; + std::memcpy(&eid, rdram + kAASlotEid, 4); + R5900Context sc{}; + setRegU32(sc, 4, static_cast(eid)); + setRegU32(sc, 5, 0x3u); // waitBits = 0x3 (AND mode by default since WEF_OR not set) + setRegU32(sc, 6, 0x20u); // mode = WEF_CLEAR_ALL + setRegU32(sc, 7, kAASlotResBits); + ps2_syscalls::WaitEventFlag(rdram, &sc, runtime); + int32_t ret = getRegS32(sc, 2); + std::memcpy(rdram + kAASlotResult, &ret, 4); + const uint32_t seq = gAASeq.load(std::memory_order_relaxed); + gAASeq.fetch_add(1u, std::memory_order_release); // RMW: preserves release-sequence chaining (see gSeq/gAASeq declaration above) + ctx->pc = 0u; + } + + // ----------------------------------------------------------------------- + // AA4: WaitEventFlag on deleted EVF returns KE_UNKNOWN_EVFID immediately + // (step function for the waiters before deletion — reuses stepWaitEvfAndRecord) + // ----------------------------------------------------------------------- + // (stepWaitEvfAndRecord already handles KE_WAIT_DELETE; reuse for AA4 waiters) + + // ----------------------------------------------------------------------- + // AA5: WaitEventFlag block-forever fiber — used to test shutdown + // ----------------------------------------------------------------------- + static void stepWaitEvfBlockForever(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + int32_t eid = 0; + std::memcpy(&eid, rdram + kAASlotEid, 4); + R5900Context sc{}; + setRegU32(sc, 4, static_cast(eid)); + setRegU32(sc, 5, 0x1u); // waitBits = 0x1 + setRegU32(sc, 6, 0x0u); // mode = AND (no clear) + setRegU32(sc, 7, kAASlotResBits); + // This will block until SetEventFlag(bit1) or scheduler shutdown. + ps2_syscalls::WaitEventFlag(rdram, &sc, runtime); + // Record return value regardless of how we got here. + int32_t ret = getRegS32(sc, 2); + std::memcpy(rdram + kAASlotResult, &ret, 4); + const uint32_t seq = gAASeq.load(std::memory_order_relaxed); + gAASeq.fetch_add(1u, std::memory_order_release); // RMW: preserves release-sequence chaining (see gSeq/gAASeq declaration above) + ctx->pc = 0u; + } + + // ----------------------------------------------------------------------- + // AA6: step function for the long-running target fiber (non-fiber host joins it) + // Sets kAAStartedFlag, spins until kAAWokenInWindow (used as stop flag), exits. + // ----------------------------------------------------------------------- + static void stepSlowThenExit(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + gAAStarted.store(1u, std::memory_order_release); + // Spin until the stop flag is set (using kAAWokenInWindow as a dual-purpose flag) + for (;;) + { + if (gAAWoken.load(std::memory_order_acquire) != 0u) break; + runtime->shouldPreemptGuestExecution(); + } + const uint32_t seq = gAASeq.load(std::memory_order_relaxed); + gAASeq.fetch_add(1u, std::memory_order_release); // RMW: preserves release-sequence chaining (see gSeq/gAASeq declaration above) + ctx->pc = 0u; + } + + // ----------------------------------------------------------------------- + // AA9: step functions for join priority floor test + // ----------------------------------------------------------------------- + // Target fiber (prio 50): sets started flag, yields many times, exits. + static void stepAA9Target(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + // Signal that we are running so the joiner can start + gAAStarted.store(1u, std::memory_order_release); + // Spin briefly so the joiner can observe the floor in action + for (int i = 0; i < 200; ++i) + { + runtime->shouldPreemptGuestExecution(); + } + ctx->pc = 0u; + } + + // Joiner fiber (prio 10): reads target tid, calls TerminateThread(target), + // then reads its own current_priority and stores it. + static void stepAA9Joiner(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + int32_t targetTid = 0; + std::memcpy(&targetTid, rdram + kAASlotEid, 4); // reuse kAASlotEid for target tid + R5900Context tc{}; + setRegU32(tc, 4, static_cast(targetTid)); + ps2_syscalls::TerminateThread(rdram, &tc, runtime); // internally calls join_fiber + + // Read own priority after join completes (TH_SELF == 0) + R5900Context rc{}; + setRegU32(rc, 4, 0u); // TH_SELF + setRegU32(rc, 5, kReferScratch); + ps2_syscalls::ReferThreadStatus(rdram, &rc, runtime); + int32_t curPrio = 0; + std::memcpy(&curPrio, rdram + kReferScratch + 0x18, 4); // current_priority @0x18 + std::memcpy(rdram + kAASlotResult, &curPrio, 4); + + const uint32_t seq = gAASeq.load(std::memory_order_relaxed); + gAASeq.fetch_add(1u, std::memory_order_release); // RMW: preserves release-sequence chaining (see gSeq/gAASeq declaration above) + ctx->pc = 0u; + } + + // ----------------------------------------------------------------------- + // AA10: arm_park + block_current wakeup window test + // Deterministic handshake: after arm_park() the fiber publishes "armed" and + // spin-waits on "go"; the host waits for "armed", calls make_ready(tid), + // THEN sets "go". The wakeup is therefore guaranteed to land between + // arm_park() and block_current() — no timing window needed. + // ----------------------------------------------------------------------- + static std::atomic gAA10Armed{false}; + static std::atomic gAA10Go{false}; + + static void stepArmThenSignal(uint8_t *rdram, R5900Context *ctx, PS2Runtime * /*runtime*/) + { + // Arm the park — sets state=Blocked while still the running fiber. + ps2sched::arm_park(); + + // Publish "armed", then hold the fiber between arm_park() and + // block_current() until the host has injected the wakeup. The spin must + // NOT call any scheduler API: being the still-running fiber while a + // foreign thread calls make_ready() is exactly the window under test. + gAA10Armed.store(true, std::memory_order_release); + while (!gAA10Go.load(std::memory_order_acquire)) + { + std::this_thread::yield(); + } + + // make_ready() ran while we were still the running fiber, so wake_locked + // set wake_pending; block_current() consumes it and returns WokenInWindow. + const ps2sched::BlockResult br = ps2sched::block_current(); + if (br == ps2sched::BlockResult::WokenInWindow) + { + gAAWoken.store(1u, std::memory_order_release); + } + + const uint32_t seq = gAASeq.load(std::memory_order_relaxed); + gAASeq.fetch_add(1u, std::memory_order_release); // RMW: preserves release-sequence chaining (see gSeq/gAASeq declaration above) + ctx->pc = 0u; + } + + // ----------------------------------------------------------------------- + // AA11: ps2fiber_current() returns correct fiber pointer on entry and resume + // ----------------------------------------------------------------------- + // The fiber records ps2fiber_current() on first entry, parks via arm_park + + // block_current, then on resume records it again; both values go to + // kAAFiberPtrSlot (lo) and kAAWokenInWindow (hi), reusing those slots. + static void stepRecordFiberPtr(uint8_t *rdram, R5900Context *ctx, PS2Runtime * /*runtime*/) + { + // First entry: record fiber pointer (lower 32 bits for RDRAM compatibility) + PS2Fiber *f = ps2fiber_current(); + uintptr_t fp = reinterpret_cast(f); + uint32_t flo = static_cast(fp & 0xFFFFFFFFu); + std::memcpy(rdram + kAAFiberPtrSlot, &flo, 4); + + // Signal started so host knows the first value is recorded + gAAStarted.store(1u, std::memory_order_release); + + // Park once: arm + block so we can be woken from outside + ps2sched::arm_park(); + ps2sched::block_current(); + + // Second entry (after wake): record fiber pointer again (into kAAWokenInWindow slot) + PS2Fiber *f2 = ps2fiber_current(); + uintptr_t fp2 = reinterpret_cast(f2); + uint32_t flo2 = static_cast(fp2 & 0xFFFFFFFFu); + gAAWoken.store(flo2, std::memory_order_release); + + const uint32_t seq = gAASeq.load(std::memory_order_relaxed); + gAASeq.fetch_add(1u, std::memory_order_release); // RMW: preserves release-sequence chaining (see gSeq/gAASeq declaration above) + ctx->pc = 0u; + } + + // ----------------------------------------------------------------------- + // AA13: WEF_OR | WEF_CLEAR — only matched waited bits are cleared + // ----------------------------------------------------------------------- + static void stepWaitEvfOrClearRecord(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + int32_t eid = 0; + std::memcpy(&eid, rdram + kAASlotEid, 4); + R5900Context sc{}; + setRegU32(sc, 4, static_cast(eid)); + setRegU32(sc, 5, 0x3u); // waitBits = 0x3 + setRegU32(sc, 6, 0x11u); // mode = WEF_OR | WEF_CLEAR (0x1 | 0x10) + setRegU32(sc, 7, kAASlotResBits); + ps2_syscalls::WaitEventFlag(rdram, &sc, runtime); + int32_t ret = getRegS32(sc, 2); + std::memcpy(rdram + kAASlotResult, &ret, 4); + const uint32_t seq = gAASeq.load(std::memory_order_relaxed); + gAASeq.fetch_add(1u, std::memory_order_release); // RMW: preserves release-sequence chaining (see gSeq/gAASeq declaration above) + ctx->pc = 0u; + } + + // AA8: quick-exit fiber that increments the sequence counter + static void stepAA8LogAndExit(uint8_t *rdram, R5900Context *ctx, PS2Runtime * /*runtime*/) + { + const uint32_t seq = gAASeq.load(std::memory_order_relaxed); + gAASeq.fetch_add(1u, std::memory_order_release); // RMW: preserves release-sequence chaining (see gSeq/gAASeq declaration above) + ctx->pc = 0u; + } + +} // anonymous namespace + +// --------------------------------------------------------------------------- +// register_scheduler_evf_mode_tests — AA1, AA2, AA4, AA13 +// --------------------------------------------------------------------------- +void register_scheduler_evf_mode_tests() +{ + MiniTest::Case("SchedulerEvfMode", [](TestCase &tc) + { + // ------------------------------------------------------------------ + // AA1: WaitEventFlag OR-mode wakes on first matching bit + // ------------------------------------------------------------------ + tc.Run("AA1: WaitEventFlag OR-mode unblocks as soon as any waited bit is set", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00808000u, &stepWaitEvfOrRecord); + + // Create EVF with initBits=0, no special attr (single-waiter mode) + const int32_t eid = createSchedEvf(rdram, &runtime, 0u, 0u); + t.IsTrue(eid > 0, "AA1: event flag created"); + if (eid <= 0) + { + return; + } + + { + int32_t e = eid; + std::memcpy(rdram.data() + kAASlotEid, &e, 4); + } + uint32_t zero = 0u; + gAASeq.store(0u, std::memory_order_release); + std::memcpy(rdram.data() + kAASlotResBits, &zero, 4); + int32_t bad = -9999; + std::memcpy(rdram.data() + kAASlotResult, &bad, 4); + + const int32_t tid = startSchedWorker(rdram.data(), &runtime, + 0x00808000u, 10, + 0x00608000u, 0x2000u); + t.IsTrue(tid > 0, "AA1: fiber started"); + if (tid <= 0) + { + deleteSchedEvf(rdram, &runtime, eid); + return; + } + + // Wait until the fiber is blocking on the event flag (OR mode) + const bool waiting = waitUntil([&]() + { + return getEvfNumThreads(rdram, runtime, eid) >= 1; + }, std::chrono::milliseconds(500)); + t.IsTrue(waiting, "AA1: fiber registered as EventFlag waiter"); + + // Set ONLY bit 0x1 — OR-mode should be satisfied immediately + { + R5900Context sc{}; + setRegU32(sc, 4, static_cast(eid)); + setRegU32(sc, 5, 0x1u); + ps2_syscalls::SetEventFlag(rdram.data(), &sc, &runtime); + } + + // Fiber must complete without re-blocking + const bool completed = waitUntil([&]() + { + return gAASeq.load(std::memory_order_acquire) >= 1u; + }, std::chrono::milliseconds(1000)); + t.IsTrue(completed, "AA1: fiber completed after single-bit OR signal"); + + // Verify return value and result bits + { + int32_t ret = -9999; + std::memcpy(&ret, rdram.data() + kAASlotResult, 4); + t.Equals(ret, KE_OK, "AA1: WaitEventFlag returned KE_OK"); + } + { + uint32_t resBits = 0; + std::memcpy(&resBits, rdram.data() + kAASlotResBits, 4); + t.IsTrue((resBits & 0x1u) != 0u, "AA1: result bits include the set bit 0x1"); + } + + // Verify it did NOT re-block: sequence incremented exactly once + { + const uint32_t seq = gAASeq.load(std::memory_order_acquire); + t.Equals(seq, 1u, "AA1: sequence incremented exactly once (no re-block)"); + } + + waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + deleteSchedEvf(rdram, &runtime, eid); + }); + + // ------------------------------------------------------------------ + // AA2: WaitEventFlag AND-mode with WEF_CLEAR_ALL clears all bits on exit + // ------------------------------------------------------------------ + tc.Run("AA2: WEF_CLEAR_ALL clears all EVF bits on exit, not just the waited bits", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00808100u, &stepWaitEvfClearAllRecord); + + // Create EVF with initBits=0xF (extra bits beyond the wait mask of 0x3) + const int32_t eid = createSchedEvf(rdram, &runtime, 0u, 0xFu); + t.IsTrue(eid > 0, "AA2: event flag created with initBits=0xF"); + if (eid <= 0) + { + return; + } + + { + int32_t e = eid; + std::memcpy(rdram.data() + kAASlotEid, &e, 4); + } + uint32_t zero = 0u; + gAASeq.store(0u, std::memory_order_release); + std::memcpy(rdram.data() + kAASlotResBits, &zero, 4); + int32_t bad = -9999; + std::memcpy(rdram.data() + kAASlotResult, &bad, 4); + + // The initBits=0xF already has bits 0x3 set, so the fiber will satisfy + // the AND condition immediately on entry and WEF_CLEAR_ALL must clear all bits. + const int32_t tid = startSchedWorker(rdram.data(), &runtime, + 0x00808100u, 10, + 0x0060A000u, 0x2000u); + t.IsTrue(tid > 0, "AA2: fiber started"); + if (tid <= 0) + { + deleteSchedEvf(rdram, &runtime, eid); + return; + } + + const bool completed = waitUntil([&]() + { + return gAASeq.load(std::memory_order_acquire) >= 1u; + }, std::chrono::milliseconds(1000)); + t.IsTrue(completed, "AA2: fiber completed"); + + { + int32_t ret = -9999; + std::memcpy(&ret, rdram.data() + kAASlotResult, 4); + t.Equals(ret, KE_OK, "AA2: WaitEventFlag returned KE_OK"); + } + + // After WEF_CLEAR_ALL, all EVF bits must be zero (not 0xC = bits beyond mask) + { + const uint32_t currBits = getEvfCurrBits(rdram, runtime, eid); + t.Equals(currBits, 0u, "AA2: WEF_CLEAR_ALL cleared all bits to zero"); + } + + waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + deleteSchedEvf(rdram, &runtime, eid); + }); + + // ------------------------------------------------------------------ + // AA4: WaitEventFlag on deleted EVF id returns KE_UNKNOWN_EVFID immediately + // (extends T17: after all waiters drain, probe the deleted id) + // ------------------------------------------------------------------ + tc.Run("AA4: WaitEventFlag on deleted EVF id returns KE_UNKNOWN_EVFID without blocking", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00808300u, &stepWaitEvfAndRecord); + + // EA_MULTI=0x2 so multiple fibers can wait + const int32_t eid = createSchedEvf(rdram, &runtime, 0x2u, 0u); + const int32_t doneSid = createSchedSema(rdram.data(), &runtime, 0, 2); + t.IsTrue(eid > 0, "AA4: event flag created"); + t.IsTrue(doneSid > 0, "AA4: done sema created"); + if (eid <= 0 || doneSid <= 0) + { + if (eid > 0) deleteSchedEvf(rdram, &runtime, eid); + deleteSchedSema(rdram.data(), &runtime, doneSid); + return; + } + + { + int32_t e = eid, d = doneSid; + std::memcpy(rdram.data() + kSlotEid, &e, 4); + std::memcpy(rdram.data() + kSlotDoneSid, &d, 4); + } + rdramSeqReset(rdram); + for (int i = 0; i < 2; ++i) + { + int32_t bad = -9999; + std::memcpy(rdram.data() + kResultBase + static_cast(i * 4), &bad, 4); + } + + const int32_t tid1 = startSchedWorker(rdram.data(), &runtime, 0x00808300u, 10, 0x0060E000u, 0x2000u); + const int32_t tid2 = startSchedWorker(rdram.data(), &runtime, 0x00808300u, 10, 0x00610000u, 0x2000u); + t.IsTrue(tid1 > 0, "AA4: fiber 1 started"); + t.IsTrue(tid2 > 0, "AA4: fiber 2 started"); + + const bool allWaiting = waitUntil([&]() + { + return getEvfNumThreads(rdram, runtime, eid) >= 2; + }, std::chrono::milliseconds(1000)); + t.IsTrue(allWaiting, "AA4: both fibers waiting on event flag"); + + // Delete the EVF — both waiters receive KE_WAIT_DELETE + deleteSchedEvf(rdram, &runtime, eid); + + const bool allDone = waitUntil([&](){ return rdramSeq(rdram) >= 2u; }, std::chrono::milliseconds(2000)); + t.IsTrue(allDone, "AA4: both fibers completed after DeleteEventFlag"); + + for (int i = 0; i < 2; ++i) + { + int32_t ret = -9999; + std::memcpy(&ret, rdram.data() + kResultBase + static_cast(i * 4), 4); + t.Equals(ret, KE_WAIT_DELETE, + std::string("AA4: fiber ") + std::to_string(i) + " received KE_WAIT_DELETE"); + } + + waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(2000)); + + // Now probe the deleted EVF id: WaitEventFlag must return KE_UNKNOWN_EVFID immediately + { + R5900Context sc{}; + setRegU32(sc, 4, static_cast(eid)); + setRegU32(sc, 5, 0x3u); + setRegU32(sc, 6, 0x0u); + setRegU32(sc, 7, 0u); + ps2_syscalls::WaitEventFlag(rdram.data(), &sc, &runtime); + t.Equals(getRegS32(sc, 2), KE_UNKNOWN_EVFID, + "AA4: WaitEventFlag on deleted EVF id returns KE_UNKNOWN_EVFID"); + } + + deleteSchedSema(rdram.data(), &runtime, doneSid); + }); + + // ------------------------------------------------------------------ + // AA13: WEF_OR | WEF_CLEAR — only matched bit is cleared on exit + // ------------------------------------------------------------------ + tc.Run("AA13: WEF_OR|WEF_CLEAR clears only the matched bit, not all waited bits", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00840000u, &stepWaitEvfOrClearRecord); + + const int32_t eid = createSchedEvf(rdram, &runtime, 0u, 0u); + t.IsTrue(eid > 0, "AA13: event flag created"); + if (eid <= 0) + { + return; + } + + { + int32_t e = eid; + std::memcpy(rdram.data() + kAASlotEid, &e, 4); + } + uint32_t zero = 0u; + gAASeq.store(0u, std::memory_order_release); + std::memcpy(rdram.data() + kAASlotResBits, &zero, 4); + int32_t bad = -9999; + std::memcpy(rdram.data() + kAASlotResult, &bad, 4); + + const int32_t tid = startSchedWorker(rdram.data(), &runtime, + 0x00840000u, 10, + 0x00640000u, 0x2000u); + t.IsTrue(tid > 0, "AA13: fiber started"); + if (tid <= 0) + { + deleteSchedEvf(rdram, &runtime, eid); + return; + } + + const bool waiting = waitUntil([&]() + { + return getEvfNumThreads(rdram, runtime, eid) >= 1; + }, std::chrono::milliseconds(500)); + t.IsTrue(waiting, "AA13: fiber registered as EventFlag waiter"); + + // Set only bit 0x1 — OR-mode is satisfied, WEF_CLEAR should clear only bit 0x1 + { + R5900Context sc{}; + setRegU32(sc, 4, static_cast(eid)); + setRegU32(sc, 5, 0x1u); + ps2_syscalls::SetEventFlag(rdram.data(), &sc, &runtime); + } + + const bool completed = waitUntil([&]() + { + return gAASeq.load(std::memory_order_acquire) >= 1u; + }, std::chrono::milliseconds(1000)); + t.IsTrue(completed, "AA13: fiber completed"); + + { + int32_t ret = -9999; + std::memcpy(&ret, rdram.data() + kAASlotResult, 4); + t.Equals(ret, KE_OK, "AA13: WaitEventFlag returned KE_OK"); + } + + // WEF_CLEAR clears only ~waitBits (0x3) from currBits. Since only bit 0x1 + // was set, after WEF_CLEAR the EVF bits should be 0 (0x1 & ~0x3 == 0). + // If both bits were set simultaneously currBits would be 0x2 after clearing 0x1. + // In either case, bit 0x1 must be cleared. + { + const uint32_t currBits = getEvfCurrBits(rdram, runtime, eid); + t.IsTrue((currBits & 0x1u) == 0u, "AA13: WEF_CLEAR cleared the matched bit 0x1"); + } + + waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + deleteSchedEvf(rdram, &runtime, eid); + }); + + }); // MiniTest::Case("SchedulerEvfMode") +} + +// --------------------------------------------------------------------------- +// register_scheduler_shutdown_fiber_tests — AA5 +// --------------------------------------------------------------------------- +void register_scheduler_shutdown_fiber_tests() +{ + MiniTest::Case("SchedulerShutdownFiber", [](TestCase &tc) + { + // ------------------------------------------------------------------ + // AA5: block_current throws ThreadExitException on shutdown while fiber + // is inside WaitEventFlag Mesa loop + // ------------------------------------------------------------------ + tc.Run("AA5: scheduler_shutdown unblocks a WaitEventFlag-blocked fiber via ThreadExitException", [](TestCase &t) + { + notifyRuntimeStop(); + ps2sched::scheduler_init(); + PS2Runtime runtime; + std::vector rdram(PS2_RAM_SIZE, 0u); + + runtime.registerFunction(0x00810000u, &stepWaitEvfBlockForever); + + // EVF with initBits=0 so the fiber blocks immediately + const int32_t eid = createSchedEvf(rdram, &runtime, 0u, 0u); + t.IsTrue(eid > 0, "AA5: event flag created"); + if (eid <= 0) + { + return; + } + + { + int32_t e = eid; + std::memcpy(rdram.data() + kAASlotEid, &e, 4); + } + uint32_t zero = 0u; + gAASeq.store(0u, std::memory_order_release); + + const int32_t tid = startSchedWorker(rdram.data(), &runtime, + 0x00810000u, 10, + 0x00610000u, 0x2000u); + t.IsTrue(tid > 0, "AA5: fiber started"); + if (tid <= 0) + { + deleteSchedEvf(rdram, &runtime, eid); + return; + } + + // Wait until the fiber is in the Mesa loop (numThreads >= 1) + const bool waiting = waitUntil([&]() + { + return getEvfNumThreads(rdram, runtime, eid) >= 1; + }, std::chrono::milliseconds(500)); + t.IsTrue(waiting, "AA5: fiber is blocked inside WaitEventFlag Mesa loop"); + + // Call scheduler_shutdown(). This must set terminateRequested on the + // blocked fiber, wake it, and block_current() will throw ThreadExitException, + // which propagates through the WaitEventFlag Mesa loop to fiber_trampoline. + ps2sched::scheduler_shutdown(); + runtime.requestStop(); + + // scheduler_shutdown returns only after the executor thread has exited, + // which means all fibers have finished. + t.Equals(g_activeThreads.load(std::memory_order_acquire), 0, + "AA5: g_activeThreads == 0 after scheduler_shutdown"); + + // The fiber must no longer be on the EVF wait-list (EVF deleted by now + // via deleteSchedEvf or waiters removed by shutdown). Since we deleted the + // EVF after shutdown, just check g_activeThreads == 0 (fiber unwound cleanly). + deleteSchedEvf(rdram, &runtime, eid); + notifyRuntimeStop(); + }); + + }); // MiniTest::Case("SchedulerShutdownFiber") +} + +// --------------------------------------------------------------------------- +// register_scheduler_join_host_tests — AA6 +// --------------------------------------------------------------------------- +void register_scheduler_join_host_tests() +{ + MiniTest::Case("SchedulerJoinHost", [](TestCase &tc) + { + // ------------------------------------------------------------------ + // AA6: join_fiber from a non-fiber host thread blocks on g_sched_cv + // until the target fiber finishes + // ------------------------------------------------------------------ + tc.Run("AA6: host thread TerminateThread blocks on g_sched_cv until fiber exits", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00818000u, &stepSlowThenExit); + + uint32_t zero = 0u; + gAAStarted.store(0u, std::memory_order_release); + gAAWoken.store(0u, std::memory_order_release); + gAASeq.store(0u, std::memory_order_release); + + const int32_t tid = startSchedWorker(rdram.data(), &runtime, + 0x00818000u, 10, + 0x00618000u, 0x2000u); + t.IsTrue(tid > 0, "AA6: long-running fiber started"); + if (tid <= 0) + { + return; + } + + // Wait for the fiber to signal it has started + const bool started = waitUntil([&]() + { + return gAAStarted.load(std::memory_order_acquire) != 0u; + }, std::chrono::milliseconds(500)); + t.IsTrue(started, "AA6: fiber signaled kAAStartedFlag"); + + // A non-fiber host thread calls TerminateThread — internally calls join_fiber + // on the non-fiber path (ps2fiber_current()==nullptr), which blocks on + // g_sched_cv.wait() until the fiber's finished flag is set. + std::atomic hostJoinDone{false}; + std::atomic hostThrew{false}; + std::thread hostThread([&]() + { + try + { + // Signal the fiber to stop spinning first so it can exit + gAAWoken.store(1u, std::memory_order_release); + + R5900Context tc{}; + setRegU32(tc, 4, static_cast(tid)); + ps2_syscalls::TerminateThread(rdram.data(), &tc, &runtime); + } + catch (...) + { + hostThrew.store(true, std::memory_order_release); + } + hostJoinDone.store(true, std::memory_order_release); + }); + + const bool joined = waitUntil([&]() + { + return hostJoinDone.load(std::memory_order_acquire); + }, std::chrono::milliseconds(2000)); + + if (hostThread.joinable()) hostThread.join(); + + t.IsTrue(joined, "AA6: host thread TerminateThread returned within 2s"); + t.IsFalse(hostThrew.load(), "AA6: host thread TerminateThread did not throw"); + t.Equals(g_activeThreads.load(std::memory_order_acquire), 0, + "AA6: g_activeThreads == 0 after host join"); + + }); + + }); // MiniTest::Case("SchedulerJoinHost") +} + +// --------------------------------------------------------------------------- +// register_scheduler_fiber_alloc_tests — AA7 +// --------------------------------------------------------------------------- +void register_scheduler_fiber_alloc_tests() +{ + MiniTest::Case("SchedulerFiberAlloc", [](TestCase &tc) + { + // ------------------------------------------------------------------ + // AA7: ps2fiber_free on an unstarted fiber does not crash + // ------------------------------------------------------------------ + tc.Run("AA7: ps2fiber_free before first resume releases stack without crash", [](TestCase &t) + { + // No scheduler needed — this tests the ps2fiber lifecycle directly. + // Allocate a fiber with a trivial no-op function and free it immediately + // without ever calling ps2fiber_resume. + static bool noopCalled = false; + auto noopFn = [](void *) { noopCalled = true; }; + + // Use a small stack (4 KiB + one page for guard). + PS2Fiber *f = ps2fiber_alloc(noopFn, nullptr, 8192u); + t.IsTrue(f != nullptr, "AA7: ps2fiber_alloc should return non-null"); + + if (f) + { + // ps2fiber_free must release the mapping and the struct without crashing. + // On the ucontext backend this must NOT call swapcontext. + bool freed = false; + bool threw = false; + try + { + ps2fiber_free(f); + freed = true; + } + catch (...) + { + threw = true; + } + t.IsTrue(freed, "AA7: ps2fiber_free returned without throwing"); + t.IsFalse(threw, "AA7: ps2fiber_free must not throw"); + t.IsFalse(noopCalled, "AA7: fiber function must not have been called"); + } + }); + + }); // MiniTest::Case("SchedulerFiberAlloc") +} + +// --------------------------------------------------------------------------- +// register_scheduler_reinit_tests — AA8 +// --------------------------------------------------------------------------- +void register_scheduler_reinit_tests() +{ + MiniTest::Case("SchedulerReinit", [](TestCase &tc) + { + // ------------------------------------------------------------------ + // AA8: g_activeThreads resets to 0 between scheduler_init cycles + // ------------------------------------------------------------------ + tc.Run("AA8: g_activeThreads is zero at the start of a second scheduler_init cycle", [](TestCase &t) + { + // Cycle 1 + notifyRuntimeStop(); + ps2sched::scheduler_init(); + PS2Runtime runtime1; + std::vector rdram(PS2_RAM_SIZE, 0u); + + runtime1.registerFunction(0x00820000u, &stepAA8LogAndExit); + + uint32_t zero = 0u; + gAASeq.store(0u, std::memory_order_release); + + const int32_t tid1 = startSchedWorker(rdram.data(), &runtime1, 0x00820000u, 10, 0x00620000u, 0x2000u); + const int32_t tid2 = startSchedWorker(rdram.data(), &runtime1, 0x00820000u, 10, 0x00622000u, 0x2000u); + t.IsTrue(tid1 > 0, "AA8: cycle1 fiber1 started"); + t.IsTrue(tid2 > 0, "AA8: cycle1 fiber2 started"); + + const bool cycle1Done = waitUntil([&]() + { + return gAASeq.load(std::memory_order_acquire) >= 2u; + }, std::chrono::milliseconds(1000)); + t.IsTrue(cycle1Done, "AA8: cycle1: both fibers exited"); + + const bool cycle1Drained = waitUntil([&]() + { + return g_activeThreads.load(std::memory_order_acquire) == 0; + }, std::chrono::milliseconds(1000)); + t.IsTrue(cycle1Drained, "AA8: cycle1: g_activeThreads drained to 0"); + + ps2sched::scheduler_shutdown(); + runtime1.requestStop(); + t.Equals(g_activeThreads.load(std::memory_order_acquire), 0, + "AA8: g_activeThreads == 0 after cycle1 shutdown"); + notifyRuntimeStop(); + + // Cycle 2: scheduler_init must start with g_activeThreads == 0 + ps2sched::scheduler_init(); + PS2Runtime runtime2; + std::vector rdram2(PS2_RAM_SIZE, 0u); + + runtime2.registerFunction(0x00820000u, &stepAA8LogAndExit); + + // Assert g_activeThreads is zero at init time (before starting any fibers) + t.Equals(g_activeThreads.load(std::memory_order_acquire), 0, + "AA8: g_activeThreads == 0 at start of cycle2 (before any fibers)"); + + uint32_t zero2 = 0u; + gAASeq.store(0u, std::memory_order_release); + + const int32_t tid3 = startSchedWorker(rdram2.data(), &runtime2, 0x00820000u, 10, 0x00624000u, 0x2000u); + t.IsTrue(tid3 > 0, "AA8: cycle2 fiber started"); + + const bool cycle2Done = waitUntil([&]() + { + return gAASeq.load(std::memory_order_acquire) >= 1u; + }, std::chrono::milliseconds(1000)); + t.IsTrue(cycle2Done, "AA8: cycle2: fiber exited"); + + const bool cycle2Drained = waitUntil([&]() + { + return g_activeThreads.load(std::memory_order_acquire) == 0; + }, std::chrono::milliseconds(1000)); + t.IsTrue(cycle2Drained, "AA8: cycle2: g_activeThreads drained to 0"); + + ps2sched::scheduler_shutdown(); + runtime2.requestStop(); + }); + + }); // MiniTest::Case("SchedulerReinit") +} + +// --------------------------------------------------------------------------- +// register_scheduler_join_priority_tests — AA9 +// --------------------------------------------------------------------------- +void register_scheduler_join_priority_tests() +{ + MiniTest::Case("SchedulerJoinPriority", [](TestCase &tc) + { + // ------------------------------------------------------------------ + // AA9: join_fiber priority floor is applied while joining and restored after + // ------------------------------------------------------------------ + tc.Run("AA9: joiner priority is floored during join and restored to original after", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00828000u, &stepAA9Joiner); + runtime.registerFunction(0x00828100u, &stepAA9Target); + + uint32_t zero = 0u; + gAAStarted.store(0u, std::memory_order_release); + gAASeq.store(0u, std::memory_order_release); + int32_t bad = -9999; + std::memcpy(rdram.data() + kAASlotResult, &bad, 4); + + // Start target first (prio 50) + const int32_t targetTid = startSchedWorker(rdram.data(), &runtime, + 0x00828100u, 50, + 0x0062A000u, 0x2000u); + t.IsTrue(targetTid > 0, "AA9: target fiber started"); + if (targetTid <= 0) + { + return; + } + + // Store target tid in kAASlotEid so stepAA9Joiner can read it + { + int32_t tv = targetTid; + std::memcpy(rdram.data() + kAASlotEid, &tv, 4); + } + + // Wait for target to signal it started + const bool targetStarted = waitUntil([&]() + { + return gAAStarted.load(std::memory_order_acquire) != 0u; + }, std::chrono::milliseconds(500)); + t.IsTrue(targetStarted, "AA9: target fiber signaled started"); + + // Now start joiner at prio 10 — it will TerminateThread(target) -> join_fiber + const int32_t joinerTid = startSchedWorker(rdram.data(), &runtime, + 0x00828000u, 10, + 0x00628000u, 0x2000u); + t.IsTrue(joinerTid > 0, "AA9: joiner fiber started"); + if (joinerTid <= 0) + { + return; + } + + // Wait for the joiner to complete (it records priority after join) + const bool joinerDone = waitUntil([&]() + { + return gAASeq.load(std::memory_order_acquire) >= 1u; + }, std::chrono::milliseconds(3000)); + t.IsTrue(joinerDone, "AA9: joiner fiber completed"); + + // Joiner's final priority must be restored to its original value (10) + { + int32_t finalPrio = -9999; + std::memcpy(&finalPrio, rdram.data() + kAASlotResult, 4); + t.Equals(finalPrio, 10, "AA9: joiner priority restored to original 10 after join"); + } + + waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + }); + + }); // MiniTest::Case("SchedulerJoinPriority") +} + +// --------------------------------------------------------------------------- +// Regression tests for 3 scheduler race conditions: +// T-DEF1: request_terminate() racing the publish->arm_park() window +// (state still Running when a host thread's TerminateThread fires). +// T-DEF2: suspend_self() racing a concurrent ResumeThread -> clear_suspend() +// (syscall-surface stress variant + a deterministic +// direct-scheduler-API variant). +// T-DEF3: join_fiber()'s ABA on tid recycling (generation-token check). +// --------------------------------------------------------------------------- +namespace +{ + // ----------------------------------------------------------------------- + // T-DEF1: TerminateThread racing the publish->arm_park() window. + // Deterministic handshake: the fiber publishes "ready" (it is still the + // running fiber -- state==Running -- and has NOT called arm_park() yet), + // then spins on "go" doing NO scheduler calls. The host polls + // ThreadInfo::terminated (set by TerminateThread itself, under info->m, + // strictly before it calls request_terminate()) and releases "go" the + // moment it observes it -- request_terminate()'s own critical section + // (a handful of field writes under g_sched_mutex, no locks contended, no + // sleeps) completes in nanoseconds, vastly faster than the polling + // cadence below, so in practice request_terminate() has already run + // by the time "go" is observed and arm_park() is called. This lands the + // terminate squarely in the state==Running window under test. + // ----------------------------------------------------------------------- + static std::atomic gDef1Ready{0}; + static std::atomic gDef1Go{0}; + + static void stepDef1PublishThenBlock(uint8_t * /*rdram*/, R5900Context *ctx, PS2Runtime *runtime) + { + gDef1Ready.store(1u, std::memory_order_release); + while (!gDef1Go.load(std::memory_order_acquire)) + { + std::this_thread::yield(); + } + + // Transition into the park exactly like any real object wait + // (WaitSema, WaitEventFlag, ...) does: publish (already done above, + // logically) -> arm_park() -> block_current(). + ps2sched::arm_park(); + ps2sched::block_current(); + + // A terminate that lands in the window above (state still Running) + // makes request_terminate()'s unconditional wake_locked() record + // wake_pending, so block_current() returns WokenInWindow without ever + // parking. Drive the real production unwind path: yield_point() + // (invoked via shouldPreemptGuestExecution(), exactly like a + // recompiled back-edge) throws ThreadExitException once it observes + // terminateRequested(). + for (int i = 0; i < 300; ++i) + { + runtime->shouldPreemptGuestExecution(); + } + ctx->pc = 0u; // unreachable in the expected path + } + + // ----------------------------------------------------------------------- + // T-DEF2a: SuspendThread(self)/ResumeThread race, syscall-surface stress + // variant. The fiber loops kDef2aIterations times calling the real + // SuspendThread(TH_SELF) syscall; the host polls ThreadInfo::suspendCount + // (no sleep, tightest possible loop) and calls ResumeThread the moment it + // observes suspendCount > 0, retrying ResumeThread until it returns + // KE_OK (a KE_NOT_SUSPEND means the poll was too early relative to + // Thread.cpp's own suspendCount++, not that anything is wrong). This is + // best-effort (not a guaranteed reproduction of the exact race window -- + // see T-DEF2b for that), but 150 back-to-back iterations give a good + // chance of landing the race at least once. + // ----------------------------------------------------------------------- + constexpr int kDef2aIterations = 150; + static std::atomic gDef2aIterStarted{0}; + static std::atomic gDef2aIterCompleted{0}; + static std::atomic gDef2aDone{0}; + + static void stepDef2aSuspendLoop(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + for (int i = 0; i < kDef2aIterations; ++i) + { + gDef2aIterStarted.fetch_add(1, std::memory_order_release); + R5900Context sc{}; + setRegU32(sc, 4, 0u); // TH_SELF + ps2_syscalls::SuspendThread(rdram, &sc, runtime); + gDef2aIterCompleted.fetch_add(1, std::memory_order_release); + } + gDef2aDone.store(1u, std::memory_order_release); + ctx->pc = 0u; + } + + // ----------------------------------------------------------------------- + // T-DEF2b: suspend_self()/clear_suspend() race, deterministic direct- + // scheduler-API variant. The fiber publishes "ready" (fc->suspendCount is + // still 0, state is Running -- it has not called suspend_self() yet) and + // spins on "go" with no scheduler calls. The host calls clear_suspend() + // BEFORE releasing "go", so the fiber cannot possibly call suspend_self() + // until after clear_suspend() has already run. This deterministically + // exercises the window T-DEF2a only lands probabilistically: + // clear_suspend() finds suspendCount already 0 (its store(0) changes + // nothing) and, since the fiber is still Running (mid the conceptual + // "ThreadInfo::suspendCount++ -> suspend_self()" window), cannot enqueue + // it -- it must route through wake_locked(), which records the + // cancellation in wake_pending. suspend_self()'s block_current() consumes + // that flag and returns WokenInWindow instead of parking with no one left + // to wake it. + // ----------------------------------------------------------------------- + static std::atomic gDef2bReady{0}; + static std::atomic gDef2bGo{0}; + static std::atomic gDef2bReturned{0}; + + static void stepDef2bSuspendSelfDirect(uint8_t * /*rdram*/, R5900Context *ctx, PS2Runtime * /*runtime*/) + { + gDef2bReady.store(1u, std::memory_order_release); + while (!gDef2bGo.load(std::memory_order_acquire)) + { + std::this_thread::yield(); + } + + ps2sched::suspend_self(); + + // Only reached once block_current() honours the pre-recorded wake + // (WokenInWindow) -- an unrecorded wake would park this fiber forever. + gDef2bReturned.store(1u, std::memory_order_release); + ctx->pc = 0u; + } + + // ----------------------------------------------------------------------- + // T-DEF3: join_fiber() ABA on tid recycling. Uses ps2sched::create_fiber() + // directly (bypassing CreateThread's own tid allocator, which does not + // deterministically reuse a specific tid) so the SAME tid can be forced + // to identify two completely unrelated fibers in sequence. + // + // Getting the joiner to observe the recycled tid *deterministically* (not + // as a rare timing coincidence) needs care: as soon as the old fiber (A) + // transitions to state==Finished, ANY join_fiber() call already waiting on + // A -- fiber or host-thread cv-wait -- immediately sees that state (or + // `!t` once erased) and returns correctly, since the new fiber (B) does + // not exist yet at that instant. The ABA bug can only be exercised if the + // JOINER's very first look at tid happens strictly AFTER both A's + // erasure and B's creation. A plain fixed delay before creating B does + // not reliably arrange this (the joiner, once notified of A's finish, + // typically reacquires the scheduler lock and returns long before any + // sleep elapses), and racing to create B the instant A's `finished` flag + // is observed is unsafe (A's FiberContext -- and its still-in-flight + // ucontext swap back to the executor -- may not have been fully retired + // yet; overwriting g_fiber_map[tid] at that point could destroy a fiber + // whose stack the executor is still using). + // + // So this test uses a third fiber, GATE, purely to get *scheduling* + // (not timing) guarantees, with zero risk to A's teardown. Three + // priority-driven phases (lower priority number == more urgent): + // + // Phase 1 -- the joiner captures A's identity while A is still alive. + // The joiner (JOINER) is created FIRST, at a priority strictly more + // urgent than A's, so A's own yield_point() preempts to it at least + // once. JOINER's first iteration (inside join_fiber(), the fiber-path + // / yield-loop branch under test) captures A's {tid, generation}, + // finds "not done", and -- per join_fiber()'s own existing + // join-priority-floor logic -- demotes ITSELF to (A's priority + 1), + // i.e. strictly less urgent than A, before yielding. From then on A + // runs uninterrupted (JOINER never preempts it again) until A decides + // to exit on its own. + // + // Phase 2 -- GATE proves A is fully retired. GATE is created at THE + // SAME priority as A (equal priority never preempts, only strictly + // lower numbers do), so it cannot run even once while A is executing + // -- it just sits Ready behind A. Telling A to exit lets it finish and + // be erased; GATE (now the only thing at that priority tier, since + // JOINER demoted itself below it) is dispatched next. GATE's body + // starting to run (signalled via gDef3GateRunning) is therefore proof + // -- a scheduling guarantee, not a race -- that A is completely + // retired and it is safe to recycle its tid. GATE then bumps its OWN + // priority to the highest value so nothing created afterward can + // preempt it, and holds the executor exclusively until told to + // release. + // + // Phase 3 -- recycle and release. Only while GATE holds the executor + // do we create B (same tid as A, at a low priority). Releasing GATE + // then lets JOINER (waiting at A's priority + 1, still more urgent + // than B) run next and make its SECOND check -- the first one able to + // observe the recycled tid. + // ----------------------------------------------------------------------- + static std::atomic gDef3AStarted{0}; + static std::atomic gDef3AShouldExit{0}; + static std::atomic gDef3GateRunning{0}; + static std::atomic gDef3GateShouldRelease{0}; + static std::atomic gDef3NewStarted{0}; + static std::atomic gDef3NewShouldExit{0}; + static std::atomic gDef3JoinerStarted{0}; + static std::atomic gDef3JoinerDone{0}; + static std::atomic gDef3JoinerThrew{0}; + static int gDef3JoinerTargetTid = 0; + + static void stepDef3AFiber(uint8_t * /*rdram*/, R5900Context *ctx, PS2Runtime *runtime) + { + gDef3AStarted.store(1u, std::memory_order_release); + for (;;) + { + if (gDef3AShouldExit.load(std::memory_order_acquire) != 0u) break; + runtime->shouldPreemptGuestExecution(); + } + ctx->pc = 0u; + } + + static void stepDef3Gate(uint8_t * /*rdram*/, R5900Context *ctx, PS2Runtime *runtime) + { + // First execution of this body is only reachable once A is gone + // (see rationale above). Bump self to the highest priority so + // nothing created after this point (B, the joiner) can preempt us. + ps2sched::update_priority(g_currentThreadId, 1); + gDef3GateRunning.store(1u, std::memory_order_release); + for (;;) + { + if (gDef3GateShouldRelease.load(std::memory_order_acquire) != 0u) break; + runtime->shouldPreemptGuestExecution(); + } + ctx->pc = 0u; + } + + static void stepDef3NewFiber(uint8_t * /*rdram*/, R5900Context *ctx, PS2Runtime *runtime) + { + gDef3NewStarted.store(1u, std::memory_order_release); + for (;;) + { + if (gDef3NewShouldExit.load(std::memory_order_acquire) != 0u) break; + runtime->shouldPreemptGuestExecution(); + } + ctx->pc = 0u; + } + + static void stepDef3Joiner(uint8_t * /*rdram*/, R5900Context *ctx, PS2Runtime * /*runtime*/) + { + // Signalling BEFORE calling join_fiber() only proves this fiber body + // was dispatched at least once; join_fiber()'s own generation + // capture (a lock/read/unlock, a handful of instructions) then runs + // synchronously, on this same executor thread, microseconds later -- + // vastly faster than the host thread's poll granularity below, so by + // the time the host observes this flag the capture has, in + // practice, already happened. + gDef3JoinerStarted.store(1u, std::memory_order_release); + try + { + ps2sched::join_fiber(gDef3JoinerTargetTid); + } + catch (...) + { + gDef3JoinerThrew.store(1u, std::memory_order_release); + } + gDef3JoinerDone.store(1u, std::memory_order_release); + ctx->pc = 0u; + } + +} // anonymous namespace + +// --------------------------------------------------------------------------- +// register_scheduler_park_window_tests — AA10 +// --------------------------------------------------------------------------- +void register_scheduler_park_window_tests() +{ + MiniTest::Case("SchedulerParkWindow", [](TestCase &tc) + { + // ------------------------------------------------------------------ + // AA10: wakeup arriving between arm_park() and block_current() is not lost + // ------------------------------------------------------------------ + tc.Run("AA10: make_ready after arm_park but before block_current returns WokenInWindow", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00830000u, &stepArmThenSignal); + + uint32_t zero = 0u; + gAAWoken.store(0u, std::memory_order_release); + gAASeq.store(0u, std::memory_order_release); + gAA10Armed.store(false, std::memory_order_relaxed); + gAA10Go.store(false, std::memory_order_relaxed); + + const int32_t tid = startSchedWorker(rdram.data(), &runtime, + 0x00830000u, 10, + 0x00630000u, 0x2000u); + t.IsTrue(tid > 0, "AA10: fiber started"); + if (tid <= 0) + { + return; + } + + // Wait until the fiber has called arm_park(); it spins on gAA10Go + // after publishing gAA10Armed, so it is guaranteed to still be the + // running fiber (not yet in block_current) when we inject the wake. + const bool armed = waitUntil([&]() + { + return gAA10Armed.load(std::memory_order_acquire); + }, std::chrono::milliseconds(2000)); + t.IsTrue(armed, "AA10: fiber signaled arm_park() done"); + + // Inject wake AFTER arm_park but BEFORE block_current (while still g_running_fiber). + // wake_locked sees g_running_fiber==fc and sets wake_pending; block_current + // consumes it and returns WokenInWindow. Only then release the fiber. + ps2sched::make_ready(tid); + gAA10Go.store(true, std::memory_order_release); + + // The fiber must complete without hanging + const bool completed = waitUntil([&]() + { + return gAASeq.load(std::memory_order_acquire) >= 1u; + }, std::chrono::milliseconds(2000)); + t.IsTrue(completed, "AA10: fiber completed without hanging"); + + // Check that the fiber observed WokenInWindow + { + const uint32_t woken = gAAWoken.load(std::memory_order_acquire); + t.IsTrue(woken != 0u, "AA10: fiber observed WokenInWindow result from block_current"); + } + + waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + }); + + // ------------------------------------------------------------------ + // T-DEF1: TerminateThread racing the publish->arm_park() window must + // still wake and unwind the fiber promptly. + // ------------------------------------------------------------------ + tc.Run("T-DEF1: TerminateThread racing publish->arm_park wakes and unwinds the fiber", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00860000u, &stepDef1PublishThenBlock); + + gDef1Ready.store(0u, std::memory_order_release); + gDef1Go.store(0u, std::memory_order_release); + + const int32_t tid = startSchedWorker(rdram.data(), &runtime, + 0x00860000u, 10, + 0x00660000u, 0x2000u); + t.IsTrue(tid > 0, "T-DEF1: fiber started"); + if (tid <= 0) + { + return; + } + + const bool ready = waitUntil([&]() + { + return gDef1Ready.load(std::memory_order_acquire) != 0u; + }, std::chrono::milliseconds(500)); + t.IsTrue(ready, "T-DEF1: fiber published ready (still Running, has not called arm_park yet)"); + + std::shared_ptr info; + { + std::lock_guard lk(g_thread_map_mutex); + auto it = g_threads.find(tid); + if (it != g_threads.end()) info = it->second; + } + t.IsTrue(info != nullptr, "T-DEF1: ThreadInfo exists for the worker"); + + // Host thread issues a FULL TerminateThread syscall (not raw + // request_terminate()) while the fiber is guaranteed to still be + // spinning (Running, never Blocked). + std::atomic hostDone{false}; + std::atomic hostThrew{false}; + std::thread hostThread([&]() + { + try + { + R5900Context tcx{}; + setRegU32(tcx, 4, static_cast(tid)); + ps2_syscalls::TerminateThread(rdram.data(), &tcx, &runtime); + } + catch (...) { hostThrew.store(true, std::memory_order_release); } + hostDone.store(true, std::memory_order_release); + }); + + // Release the fiber into arm_park()/block_current() the moment + // TerminateThread has set ThreadInfo::terminated (which happens + // strictly before it calls request_terminate() -- see + // TerminateThread in Thread.cpp). request_terminate()'s own + // critical section is a handful of unlocked-contention field + // writes; it completes long before this poll's granularity, so + // in practice it has already run by the time we release "go". + const bool terminatedFlagSet = info && waitUntil([&]() + { + return info->terminated.load(std::memory_order_acquire); + }, std::chrono::milliseconds(2000)); + t.IsTrue(terminatedFlagSet, "T-DEF1: TerminateThread set ThreadInfo::terminated"); + gDef1Go.store(1u, std::memory_order_release); + + const bool hostCompleted = waitUntil([&]() + { + return hostDone.load(std::memory_order_acquire); + }, std::chrono::milliseconds(2000)); + if (hostThread.joinable()) hostThread.join(); + + t.IsTrue(hostCompleted, "T-DEF1: TerminateThread's join_fiber completed promptly"); + t.IsFalse(hostThrew.load(), "T-DEF1: TerminateThread did not throw"); + t.Equals(g_activeThreads.load(std::memory_order_acquire), 0, + "T-DEF1: fiber actually finished (g_activeThreads == 0)"); + + }); + + // ------------------------------------------------------------------ + // T-DEF2a: SuspendThread(self)/ResumeThread race, syscall-surface + // stress variant (best-effort). + // ------------------------------------------------------------------ + tc.Run("T-DEF2a: SuspendThread(self)/ResumeThread race never loses a resume (syscall surface, 150 iterations)", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00910000u, &stepDef2aSuspendLoop); + + gDef2aIterStarted.store(0, std::memory_order_release); + gDef2aIterCompleted.store(0, std::memory_order_release); + gDef2aDone.store(0u, std::memory_order_release); + + const int32_t tid = startSchedWorker(rdram.data(), &runtime, + 0x00910000u, 10, + 0x00794000u, 0x2000u); + t.IsTrue(tid > 0, "T-DEF2a: fiber started"); + if (tid <= 0) + { + return; + } + + std::shared_ptr info; + { + std::lock_guard lk(g_thread_map_mutex); + auto it = g_threads.find(tid); + if (it != g_threads.end()) info = it->second; + } + t.IsTrue(info != nullptr, "T-DEF2a: ThreadInfo exists for the worker"); + + bool allAdvanced = true; + for (int i = 0; i < kDef2aIterations && allAdvanced; ++i) + { + const bool iterStarted = waitUntil([&]() + { + return gDef2aIterStarted.load(std::memory_order_acquire) > i; + }, std::chrono::milliseconds(1000)); + if (!iterStarted) { allAdvanced = false; break; } + + // Tight (no-sleep) poll on ThreadInfo::suspendCount to + // maximize the chance of calling ResumeThread while the race + // window is open. + if (info) + { + const auto pollDeadline = std::chrono::steady_clock::now() + std::chrono::milliseconds(200); + for (;;) + { + { + std::lock_guard lk(info->m); + if (info->suspendCount > 0) break; + } + if (std::chrono::steady_clock::now() > pollDeadline) break; + } + } + + // Retry ResumeThread until it actually succeeds: a + // KE_NOT_SUSPEND just means the poll above gave up before + // Thread.cpp's own suspendCount++ landed, not a failure. + { + const auto retryDeadline = std::chrono::steady_clock::now() + std::chrono::milliseconds(1000); + for (;;) + { + R5900Context rc{}; + setRegU32(rc, 4, static_cast(tid)); + ps2_syscalls::ResumeThread(rdram.data(), &rc, &runtime); + if (getRegS32(rc, 2) == KE_OK) break; + if (std::chrono::steady_clock::now() > retryDeadline) break; + } + } + + const bool iterCompleted = waitUntil([&]() + { + return gDef2aIterCompleted.load(std::memory_order_acquire) > i; + }, std::chrono::milliseconds(1000)); + if (!iterCompleted) { allAdvanced = false; break; } + } + + t.IsTrue(allAdvanced, + "T-DEF2a: every SuspendThread/ResumeThread pair completed promptly"); + + waitUntil([&]() { return gDef2aDone.load(std::memory_order_acquire) != 0u; }, std::chrono::milliseconds(2000)); + waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + }); + + // ------------------------------------------------------------------ + // T-DEF2b: clear_suspend() firing before suspend_self() must be + // recorded as wake_pending (wake-in-window), deterministic + // direct-scheduler-API variant (guaranteed reproduction). + // ------------------------------------------------------------------ + tc.Run("T-DEF2b: clear_suspend before suspend_self records wake_pending so the park is woken in-window (deterministic handshake)", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00920000u, &stepDef2bSuspendSelfDirect); + + gDef2bReady.store(0u, std::memory_order_release); + gDef2bGo.store(0u, std::memory_order_release); + gDef2bReturned.store(0u, std::memory_order_release); + + const int32_t tid = startSchedWorker(rdram.data(), &runtime, + 0x00920000u, 10, + 0x00798000u, 0x2000u); + t.IsTrue(tid > 0, "T-DEF2b: fiber started"); + if (tid <= 0) + { + return; + } + + const bool ready = waitUntil([&]() + { + return gDef2bReady.load(std::memory_order_acquire) != 0u; + }, std::chrono::milliseconds(500)); + t.IsTrue(ready, "T-DEF2b: fiber published ready (still Running, has not called suspend_self yet)"); + + // Fire the resume BEFORE releasing "go": the fiber cannot call + // suspend_self() until we set gDef2bGo below, so this + // deterministically reproduces the race every time. + ps2sched::clear_suspend(tid); + gDef2bGo.store(1u, std::memory_order_release); + + const bool returned = waitUntil([&]() + { + return gDef2bReturned.load(std::memory_order_acquire) != 0u; + }, std::chrono::milliseconds(2000)); + t.IsTrue(returned, "T-DEF2b: suspend_self() returned promptly instead of parking forever"); + + waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + }); + + // ------------------------------------------------------------------ + // T-DEF3: join_fiber() ABA on tid recycling. + // ------------------------------------------------------------------ + tc.Run("T-DEF3: join_fiber does not latch onto a recycled tid's new fiber (ABA)", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00900000u, &stepDef3AFiber); + runtime.registerFunction(0x00901000u, &stepDef3Gate); + runtime.registerFunction(0x00902000u, &stepDef3NewFiber); + runtime.registerFunction(0x00903000u, &stepDef3Joiner); + + gDef3AStarted.store(0u, std::memory_order_release); + gDef3AShouldExit.store(0u, std::memory_order_release); + gDef3GateRunning.store(0u, std::memory_order_release); + gDef3GateShouldRelease.store(0u, std::memory_order_release); + gDef3NewStarted.store(0u, std::memory_order_release); + gDef3NewShouldExit.store(0u, std::memory_order_release); + gDef3JoinerStarted.store(0u, std::memory_order_release); + gDef3JoinerDone.store(0u, std::memory_order_release); + gDef3JoinerThrew.store(0u, std::memory_order_release); + + // Arbitrary fixed tids, well outside CreateThread's own 2..0xFF + // allocator range, forced directly via ps2sched::create_fiber() + // (CreateThread's own tid allocator does not deterministically + // reuse a specific tid, so it cannot force this ABA scenario). + constexpr int kDef3Tid = 0x654321; // A, later recycled to B + constexpr int kDef3GateTid = 0x654322; + constexpr int kDef3JoinerTid = 0x654323; + gDef3JoinerTargetTid = kDef3Tid; + + // A: priority 10. + ps2sched::create_fiber(kDef3Tid, 10, 0x00900000u, 0x00790000u, 0u, 0u, &runtime, rdram.data()); + const bool aStarted = waitUntil([&]() + { + return gDef3AStarted.load(std::memory_order_acquire) != 0u; + }, std::chrono::milliseconds(500)); + t.IsTrue(aStarted, "T-DEF3: A started"); + + // Phase 1: JOINER, priority 5 (strictly more urgent than A=10), + // created while A is running. A's own yield_point() preempts to + // it; JOINER's first join_fiber() iteration captures A's + // {tid, generation}, finds "not done", and (via join_fiber()'s + // own existing priority-floor logic) demotes ITSELF to + // A's-priority+1 = 11 before yielding -- so A then runs + // uninterrupted from here on. + ps2sched::create_fiber(kDef3JoinerTid, 5, 0x00903000u, 0x00796000u, 0u, 0u, &runtime, rdram.data()); + const bool joinerStarted = waitUntil([&]() + { + return gDef3JoinerStarted.load(std::memory_order_acquire) != 0u; + }, std::chrono::milliseconds(500)); + t.IsTrue(joinerStarted, "T-DEF3: joiner dispatched at least once (captured A's generation) before A exits"); + + // Phase 2: GATE, priority 10 (EQUAL to A's -- equal priority + // never preempts, only strictly lower numbers do), created + // while A is still running. It sits Ready behind A and cannot + // run even once until A actually stops being the running fiber. + ps2sched::create_fiber(kDef3GateTid, 10, 0x00901000u, 0x00792000u, 0u, 0u, &runtime, rdram.data()); + + // Tell A to exit. It finishes and is fully erased by the + // executor; GATE (now the only Ready fiber at that priority + // tier -- JOINER demoted itself to 11) is dispatched next. + gDef3AShouldExit.store(1u, std::memory_order_release); + + const bool gateRunning = waitUntil([&]() + { + return gDef3GateRunning.load(std::memory_order_acquire) != 0u; + }, std::chrono::milliseconds(2000)); + t.IsTrue(gateRunning, + "T-DEF3: GATE started (proves A finished and was erased -- scheduling guarantee, not a race)"); + + // Phase 3: safe to recycle kDef3Tid now: A is confirmed gone, + // and GATE (now at priority 1, the highest) will not let B + // preempt it before it is fully created. + // + // B: priority 90 (very low) -- may never even need to run. + ps2sched::create_fiber(kDef3Tid, 90, 0x00902000u, 0x00794000u, 0u, 0u, &runtime, rdram.data()); + + // Release GATE: the executor dispatches JOINER next (still at + // priority 11, more urgent than B's 90) for its SECOND + // join_fiber() iteration -- the first one able to observe the + // recycled tid. + gDef3GateShouldRelease.store(1u, std::memory_order_release); + + // The joiner must detect that its ORIGINAL target (A) is gone -- + // by generation, not just by tid -- and return promptly. It must + // NOT silently latch onto B (which never finishes on its own, + // and is even lower priority than the joiner) and hang forever. + const bool joinerCompleted = waitUntil([&]() + { + return gDef3JoinerDone.load(std::memory_order_acquire) != 0u; + }, std::chrono::milliseconds(2000)); + t.IsTrue(joinerCompleted, + "T-DEF3: joiner detected tid recycling and returned promptly"); + t.IsFalse(gDef3JoinerThrew.load(std::memory_order_acquire) != 0u, "T-DEF3: joiner did not throw"); + + // Clean up B: let it exit and join it (now legitimately targeting + // B, the current occupant of kDef3Tid) before shutdown. + gDef3NewShouldExit.store(1u, std::memory_order_release); + ps2sched::join_fiber(kDef3Tid); + + }); + + }); // MiniTest::Case("SchedulerParkWindow") +} + +// --------------------------------------------------------------------------- +// register_scheduler_fiber_ptr_tests — AA11 +// --------------------------------------------------------------------------- +void register_scheduler_fiber_ptr_tests() +{ + MiniTest::Case("SchedulerFiberPtr", [](TestCase &tc) + { + // ------------------------------------------------------------------ + // AA11: ps2fiber_current() returns same fiber pointer on entry and resume + // ------------------------------------------------------------------ + tc.Run("AA11: ps2fiber_current() is non-null and stable across a yield/resume cycle", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00838000u, &stepRecordFiberPtr); + + uint32_t zero = 0u; + gAAStarted.store(0u, std::memory_order_release); + std::memcpy(rdram.data() + kAAFiberPtrSlot, &zero, 4); + gAAWoken.store(0u, std::memory_order_release); + gAASeq.store(0u, std::memory_order_release); + + const int32_t tid = startSchedWorker(rdram.data(), &runtime, + 0x00838000u, 10, + 0x00638000u, 0x2000u); + t.IsTrue(tid > 0, "AA11: fiber started"); + if (tid <= 0) + { + return; + } + + // Wait for the fiber to record first-entry ptr and park + const bool firstEntry = waitUntil([&]() + { + return gAAStarted.load(std::memory_order_acquire) != 0u; + }, std::chrono::milliseconds(500)); + t.IsTrue(firstEntry, "AA11: fiber recorded first-entry ptr and parked"); + + // Read first-entry fiber pointer + uint32_t ptrOnEntry = 0u; + std::memcpy(&ptrOnEntry, rdram.data() + kAAFiberPtrSlot, 4); + t.IsTrue(ptrOnEntry != 0u, "AA11: ps2fiber_current() was non-null on first entry"); + + // Wake the fiber so it records the second ptr and exits + ps2sched::make_ready(tid); + + const bool completed = waitUntil([&]() + { + return gAASeq.load(std::memory_order_acquire) >= 1u; + }, std::chrono::milliseconds(1000)); + t.IsTrue(completed, "AA11: fiber completed after wake"); + + // Read second ptr (after resume) + const uint32_t ptrOnResume = gAAWoken.load(std::memory_order_acquire); + t.IsTrue(ptrOnResume != 0u, "AA11: ps2fiber_current() was non-null after resume"); + t.Equals(ptrOnEntry, ptrOnResume, "AA11: ps2fiber_current() is the same pointer before and after yield"); + + waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + }); + + }); // MiniTest::Case("SchedulerFiberPtr") +} diff --git a/ps2xTest/src/ps2_runtime_kernel_tests.cpp b/ps2xTest/src/ps2_runtime_kernel_tests.cpp index 0b75c063f..fa4422175 100644 --- a/ps2xTest/src/ps2_runtime_kernel_tests.cpp +++ b/ps2xTest/src/ps2_runtime_kernel_tests.cpp @@ -4,6 +4,7 @@ #include "ps2_syscalls.h" #include "ps2_stubs.h" +#include #include #include #include @@ -51,9 +52,14 @@ namespace constexpr uint32_t TSW_SEMA = 2u; constexpr uint32_t TSW_EVENT = 3u; - constexpr uint32_t K_EVENT_WAIT_READY_ADDR = 0x1800u; - constexpr uint32_t K_EVENT_WAIT_GATE_ADDR = 0x1804u; - constexpr uint32_t K_TERMINATE_SEMA_WAIT_READY_ADDR = 0x1810u; + // waitEventAfterSuspendHandler runs on a guest fiber thread while the test + // body polls from the host test thread; these rendezvous flags must be + // atomics with acquire/release ordering rather than plain rdram words — a + // plain memcpy poll across that thread boundary is a data race even though + // the two threads never touch memory at the same instant. + std::atomic gEventWaitReady{0}; + std::atomic gEventWaitGate{0}; + std::atomic gTerminateSemaWaitReady{0}; struct EeThreadStatus { @@ -196,8 +202,8 @@ namespace return; } - writeGuestU32(rdram, K_EVENT_WAIT_READY_ADDR, 1u); - while (readGuestU32(rdram, K_EVENT_WAIT_GATE_ADDR) == 0u) + gEventWaitReady.store(1u, std::memory_order_release); + while (gEventWaitGate.load(std::memory_order_acquire) == 0u) { if (runtime && runtime->isStopRequested()) { @@ -223,7 +229,7 @@ namespace return; } - writeGuestU32(rdram, K_TERMINATE_SEMA_WAIT_READY_ADDR, 1u); + gTerminateSemaWaitReady.store(1u, std::memory_order_release); WaitSema(rdram, ctx, runtime); ctx->pc = 0u; } @@ -692,7 +698,7 @@ void register_ps2_runtime_kernel_tests() "sub-case H: count must be exactly 0 after single token consumed (not -1)"); } - // Sub-case I: blocking WaitSema woken by SignalSema returns sid (the DQ8 scenario). + // Sub-case I: blocking WaitSema woken by SignalSema returns sid (the sid-on-success wake scenario). // init=0 forces the worker to block; SignalSema uses cv.notify_one() (not // ReleaseWaitThread), so the worker needs no g_currentThreadId identity. { @@ -757,6 +763,13 @@ void register_ps2_runtime_kernel_tests() tc.Run("WaitEventFlag preserves waitsuspend state when a suspended thread blocks", [](TestCase &t) { + // StartThread enqueues a guest fiber; nothing executes it unless the + // fiber scheduler's executor thread is running, so this test must + // bracket itself with scheduler_init()/scheduler_shutdown() like the + // Scheduler* suites do. + notifyRuntimeStop(); + ps2sched::scheduler_init(); + TestEnv env; constexpr uint32_t kEventParamAddr = 0x1600u; @@ -768,8 +781,8 @@ void register_ps2_runtime_kernel_tests() 0u }; std::memcpy(env.rdram.data() + kEventParamAddr, eventParam, sizeof(eventParam)); - writeGuestU32(env.rdram.data(), K_EVENT_WAIT_READY_ADDR, 0u); - writeGuestU32(env.rdram.data(), K_EVENT_WAIT_GATE_ADDR, 0u); + gEventWaitReady.store(0u, std::memory_order_release); + gEventWaitGate.store(0u, std::memory_order_release); R5900Context createEventCtx{}; setRegU32(createEventCtx, 4, kEventParamAddr); @@ -802,7 +815,7 @@ void register_ps2_runtime_kernel_tests() const bool ready = waitUntil([&]() { - return readGuestU32(env.rdram.data(), K_EVENT_WAIT_READY_ADDR) == 1u; + return gEventWaitReady.load(std::memory_order_acquire) == 1u; }, std::chrono::milliseconds(200)); t.IsTrue(ready, "waiter thread should reach the suspend gate before blocking"); @@ -810,7 +823,7 @@ void register_ps2_runtime_kernel_tests() SuspendThread(env.rdram.data(), &env.ctx, &env.runtime); t.Equals(getRegS32(env.ctx, 2), KE_OK, "SuspendThread should succeed for the running waiter"); - writeGuestU32(env.rdram.data(), K_EVENT_WAIT_GATE_ADDR, 1u); + gEventWaitGate.store(1u, std::memory_order_release); const bool waiting = waitUntil([&]() { @@ -840,22 +853,26 @@ void register_ps2_runtime_kernel_tests() SetEventFlag(env.rdram.data(), &signalCtx, &env.runtime); t.Equals(getRegS32(signalCtx, 2), KE_OK, "SetEventFlag should wake the waiting thread"); - const bool suspended = waitUntil([&]() + // Fiber-scheduler semantics: a wakeup that arrives while the fiber is + // suspended is deferred by the suspendCount gate — the fiber cannot run + // (or update its own wait bookkeeping) until ResumeThread, so the thread + // remains THS_WAITSUSPEND here. (The original EE kernel moved the TCB to + // THS_SUSPEND as soon as the wait was released; the cooperative fiber + // model applies that transition when the resumed fiber re-checks its + // now-satisfied wait — see the SchedulerProtocol suspend/resume tests.) { R5900Context statusCtx{}; setRegU32(statusCtx, 4, static_cast(tid)); setRegU32(statusCtx, 5, K_STATUS_ADDR); ReferThreadStatus(env.rdram.data(), &statusCtx, &env.runtime); - if (getRegS32(statusCtx, 2) != KE_OK) - { - return false; - } + t.Equals(getRegS32(statusCtx, 2), KE_OK, + "ReferThreadStatus should succeed after SetEventFlag"); EeThreadStatus status{}; std::memcpy(&status, env.rdram.data() + K_STATUS_ADDR, sizeof(status)); - return status.status == THS_SUSPEND && status.waitType == 0u; - }, std::chrono::milliseconds(200)); - t.IsTrue(suspended, "after wake, a still-suspended waiter should move to THS_SUSPEND"); + t.Equals(status.status, THS_WAITSUSPEND, + "a suspended waiter stays THS_WAITSUSPEND until ResumeThread lifts the suspend gate"); + } setRegU32(env.ctx, 4, static_cast(tid)); ResumeThread(env.rdram.data(), &env.ctx, &env.runtime); @@ -885,10 +902,20 @@ void register_ps2_runtime_kernel_tests() setRegU32(env.ctx, 4, static_cast(tid)); DeleteThread(env.rdram.data(), &env.ctx, &env.runtime); t.Equals(getRegS32(env.ctx, 2), KE_OK, "DeleteThread should clean up the waiter thread"); + + ps2sched::scheduler_shutdown(); + env.runtime.requestStop(); + notifyRuntimeStop(); }); tc.Run("TerminateThread unwinds semaphore wait as a normal thread exit", [](TestCase &t) { + // Bracket with scheduler_init()/scheduler_shutdown() so the enqueued + // guest fiber runs (see "WaitEventFlag preserves waitsuspend + // state..." above). + notifyRuntimeStop(); + ps2sched::scheduler_init(); + TestEnv env; constexpr uint32_t kWaitThreadEntry = 0x00261000u; @@ -920,7 +947,7 @@ void register_ps2_runtime_kernel_tests() 0u }; - writeGuestU32(env.rdram.data(), K_TERMINATE_SEMA_WAIT_READY_ADDR, 0u); + gTerminateSemaWaitReady.store(0u, std::memory_order_release); writeGuestWords(env.rdram.data(), K_PARAM_ADDR, threadParam, std::size(threadParam)); setRegU32(env.ctx, 4, K_PARAM_ADDR); CreateThread(env.rdram.data(), &env.ctx, &env.runtime); @@ -937,7 +964,7 @@ void register_ps2_runtime_kernel_tests() const bool waiting = waitUntil([&]() { - if (readGuestU32(env.rdram.data(), K_TERMINATE_SEMA_WAIT_READY_ADDR) != 1u) + if (gTerminateSemaWaitReady.load(std::memory_order_acquire) != 1u) { return false; } @@ -984,6 +1011,10 @@ void register_ps2_runtime_kernel_tests() setRegU32(env.ctx, 4, static_cast(sid)); DeleteSema(env.rdram.data(), &env.ctx, &env.runtime); t.Equals(getRegS32(env.ctx, 2), sid, "DeleteSema should return sid while cleaning up the waiter semaphore"); + + ps2sched::scheduler_shutdown(); + env.runtime.requestStop(); + notifyRuntimeStop(); }); tc.Run("setup heap and allocator primitives track end-of-heap", [](TestCase &t) diff --git a/ps2xTest/src/ps2_scheduler_workload_regression_tests.cpp b/ps2xTest/src/ps2_scheduler_workload_regression_tests.cpp new file mode 100644 index 000000000..07c3c8899 --- /dev/null +++ b/ps2xTest/src/ps2_scheduler_workload_regression_tests.cpp @@ -0,0 +1,1635 @@ +// --------------------------------------------------------------------------- +// Scheduler / runtime regression tests grounded in how real recompiled PS2 +// guests exercise the N=1 fiber scheduler. Each suite here targets a bug that +// the synthetic scheduler suites cannot see because their fibers +// are well-behaved (they yield cooperatively, park promptly, and shut down +// from the main thread). Real recompiled guests spin across function +// dispatches, monopolize the executor, and call requestStop() from guest +// context — these tests reproduce those shapes with bounded waits (every wait +// has a timeout and an escape hatch: a regression FAILS, it does not hang). +// --------------------------------------------------------------------------- + +#include "MiniTest.h" +#include "SchedTestSupport.h" +#include "ps2_runtime.h" +#include "ps2_syscalls.h" +#include "ps2_stubs.h" +#include "ps2_runtime_macros.h" +#include "Kernel/Syscalls/Interrupt.h" +#include "Kernel/Syscalls/System.h" + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "Kernel/Syscalls/Helpers/State.h" // g_threads / g_thread_map_mutex / g_activeThreads + +using namespace ps2_syscalls; +using namespace ps2x_test; + +namespace +{ + // These helpers move test control/result words between the test main + // thread and guest fibers (which may run on other OS threads under the + // pthread fiber backend). Use atomic accesses so that cross-thread + // traffic is well-defined and the suite runs clean under TSan; the + // addresses used are all 4-byte aligned, as std::atomic_ref requires. + void rdramWrite32Raw(uint8_t *rdram, uint32_t addr, uint32_t val) + { + std::atomic_ref(*reinterpret_cast(rdram + addr)).store(val); + } + + uint32_t rdramRead32Raw(const uint8_t *rdram, uint32_t addr) + { + return std::atomic_ref( + *reinterpret_cast(const_cast(rdram) + addr)) + .load(); + } + + void rdramWrite32(std::vector &rdram, uint32_t addr, uint32_t val) + { + rdramWrite32Raw(rdram.data(), addr, val); + } + + uint32_t rdramRead32(const std::vector &rdram, uint32_t addr) + { + return rdramRead32Raw(rdram.data(), addr); + } + + // Create a semaphore via CreateSema (EE layout: count, max_count, init_count). + int32_t createSchedSema(uint8_t *rdram, PS2Runtime *runtime, int initCount, int maxCount) + { + constexpr uint32_t kSemaParamAddr = 0x2F40u; + const uint32_t params[6] = { + 0u, + static_cast(maxCount), + static_cast(initCount), + 0u, + 0u, + 0u, + }; + std::memcpy(rdram + kSemaParamAddr, params, sizeof(params)); + + R5900Context cCtx{}; + setRegU32(cCtx, 4, kSemaParamAddr); + ps2_syscalls::CreateSema(rdram, &cCtx, runtime); + return getRegS32(cCtx, 2); + } + + void deleteSchedSema(uint8_t *rdram, PS2Runtime *runtime, int32_t sid) + { + if (sid <= 0) + { + return; + } + R5900Context dCtx{}; + setRegU32(dCtx, 4, static_cast(sid)); + ps2_syscalls::DeleteSema(rdram, &dCtx, runtime); + } + + void signalSchedSema(uint8_t *rdram, PS2Runtime *runtime, int32_t sid) + { + if (sid <= 0) + { + return; + } + R5900Context sCtx{}; + setRegU32(sCtx, 4, static_cast(sid)); + ps2_syscalls::SignalSema(rdram, &sCtx, runtime); + } + + // Create + start a fiber worker thread; returns tid or -1. + int32_t startSchedWorker(uint8_t *rdram, PS2Runtime *runtime, + uint32_t entryAddr, int priority, + uint32_t stackAddr, uint32_t stackSize) + { + constexpr uint32_t kThreadParamAddr = 0x2E40u; + const uint32_t threadParam[7] = { + 0u, + entryAddr, + stackAddr, + stackSize, + 0u, + static_cast(priority), + 0u, + }; + std::memcpy(rdram + kThreadParamAddr, threadParam, sizeof(threadParam)); + + R5900Context createCtx{}; + setRegU32(createCtx, 4, kThreadParamAddr); + ps2_syscalls::CreateThread(rdram, &createCtx, runtime); + const int32_t tid = getRegS32(createCtx, 2); + if (tid <= 0) + { + return -1; + } + + R5900Context startCtx{}; + setRegU32(startCtx, 4, static_cast(tid)); + setRegU32(startCtx, 5, 0u); + ps2_syscalls::StartThread(rdram, &startCtx, runtime); + if (getRegS32(startCtx, 2) != 0) + { + return -1; + } + return tid; + } + + // ------------------------------------------------------------------ + // Control words (kernel-area scratch, disjoint from other suites) + // ------------------------------------------------------------------ + constexpr uint32_t kThStop = 0x00060000u; // 1 => guest loops exit + constexpr uint32_t kThSidX = 0x00060010u; // ping-pong sema X + constexpr uint32_t kThSidY = 0x00060014u; // ping-pong sema Y + constexpr uint32_t kThFlag = 0x00060020u; // starvation probe flag + constexpr uint32_t kThCount = 0x00060024u; // ping-pong loop counter + constexpr uint32_t kThSent = 0x00060028u; // DMAC handler sentinel + + // ------------------------------------------------------------------ + // Guest step functions (registered as recompiled functions) + // ------------------------------------------------------------------ + + // Ping-pong pair: one permit circulates between semas X and Y, so the run + // queue is (almost) never empty and the executor never runs out of ready + // fibers — the shape that starved parked host workers before the + // g_host_token_waiters executor gate. + static void stepPingA(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + while (rdramRead32Raw(rdram, kThStop) == 0u) + { + const int32_t sidX = static_cast(rdramRead32Raw(rdram, kThSidX)); + const int32_t sidY = static_cast(rdramRead32Raw(rdram, kThSidY)); + R5900Context w{}; + setRegU32(w, 4, static_cast(sidX)); + ps2_syscalls::WaitSema(rdram, &w, runtime); + rdramWrite32Raw(rdram, kThCount, rdramRead32Raw(rdram, kThCount) + 1u); + R5900Context s{}; + setRegU32(s, 4, static_cast(sidY)); + ps2_syscalls::SignalSema(rdram, &s, runtime); + } + ctx->pc = 0u; + } + + // A guest loop that never blocks and never returns to the dispatch loop: + // its only scheduling hook is the recompiler-emitted back-edge call to + // shouldPreemptGuestExecution() (yield_point). Without yield_point step 4 + // this fiber holds the executor inside ps2fiber_resume forever and a + // parked host worker starves no matter what the executor predicate does. + static void stepSpinShouldPreempt(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + while (rdramRead32Raw(rdram, kThStop) == 0u) + { + rdramWrite32Raw(rdram, kThCount, rdramRead32Raw(rdram, kThCount) + 1u); + runtime->shouldPreemptGuestExecution(); // recompiled back-edge hook + } + ctx->pc = 0u; + } + + // Cross-dispatch spin pair: each function returns to the dispatch loop + // with ctx->pc aimed at the other. Neither body ever calls the emitted + // back-edge hook (mimicking generated code whose loop back-edge is a + // call/return across function boundaries), so the ONLY place a yield can + // happen is the dispatch loop itself. + static void stepCrossDispatchA(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + rdramWrite32Raw(rdram, kThCount, rdramRead32Raw(rdram, kThCount) + 1u); + ctx->pc = (rdramRead32Raw(rdram, kThStop) == 0u) ? 0x00700400u : 0u; + } + + static void stepCrossDispatchB(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + ctx->pc = (rdramRead32Raw(rdram, kThStop) == 0u) ? 0x00700300u : 0u; + } + + // RPC-server shape (sceSifRpcInit + sceSifSetRpcQueue + sceSifRpcLoop): + // the generated caller leaves ctx->pc at the loop entry, so if the stub + // returns without blocking the dispatch loop re-enters it forever. + static void stepRpcServerLoop(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + ps2_stubs::sceSifRpcLoop(rdram, ctx, runtime); + // ctx->pc intentionally left at this function's address: the real + // guest loop re-enters sceSifRpcLoop after any (stray) wakeup. + } + + static void stepWriteFlagAndExit(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + rdramWrite32Raw(rdram, kThFlag, 1u); + ctx->pc = 0u; + } + + // Guest-context stop shape: spin (no yields) until the REAL interrupt + // worker parks for the guest token, then call requestStop() from inside + // the fiber - exactly what the unimplemented-function default handler + // does when a guest thread faults. kThStop is a spin escape hatch. + static void stepGuestContextStop(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + while (ps2sched::host_token_waiters() == 0 && + rdramRead32Raw(rdram, kThStop) == 0u) + { + // busy spin: no yield points, so the parked worker stays parked + } + runtime->requestStop(); // notifyRuntimeStop() from GUEST context + rdramWrite32Raw(rdram, kThFlag, 1u); + ctx->pc = 0u; + } + + // DMAC handler body: bump the sentinel and finish. + static void stepDmacHandler(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + rdramWrite32Raw(rdram, kThSent, rdramRead32Raw(rdram, kThSent) + 1u); + ctx->pc = 0u; + } + + // sceSifSetDma shape: a guest syscall path that dispatches DMAC handlers + // SYNCHRONOUSLY from the fiber servicing the syscall. + static void stepDmacFromGuest(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + ps2_syscalls::dispatchDmacHandlersForCause(rdram, runtime, 5u); + rdramWrite32Raw(rdram, kThFlag, 1u); + ctx->pc = 0u; + } + + static void stepPingB(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + while (rdramRead32Raw(rdram, kThStop) == 0u) + { + const int32_t sidX = static_cast(rdramRead32Raw(rdram, kThSidX)); + const int32_t sidY = static_cast(rdramRead32Raw(rdram, kThSidY)); + R5900Context w{}; + setRegU32(w, 4, static_cast(sidY)); + ps2_syscalls::WaitSema(rdram, &w, runtime); + R5900Context s{}; + setRegU32(s, 4, static_cast(sidX)); + ps2_syscalls::SignalSema(rdram, &s, runtime); + } + ctx->pc = 0u; + } + + // ------------------------------------------------------------------ + // SchedulerRecoveryIsolation helpers + // + // The diagnostic dispatch-trace ring (formerly a single process-wide + // `thread_local DispatchHistory`) is now owned per-fiber by FiberContext. + // These step functions call PS2Runtime::debugCurrentDispatchTrace() - + // which resolves to whichever fiber (or per-OS-thread fallback) is + // running on the calling thread - so they MUST run inside a fiber's step + // function. Calling it from the test/main thread instead would silently + // observe the host fallback ring, not any fiber's history, and the tests + // would pass vacuously without ever exercising the fix. All assertions + // are therefore made on rdram words written by the fibers themselves; + // the main thread only reads rdram after the fibers finish. + // ------------------------------------------------------------------ + + // No-op registrant for marker PCs: never dispatched via ctx->pc, only + // looked up directly (lookupFunction) to push a PC into the calling + // fiber's dispatch-trace ring. Registered purely to avoid the noisy + // "no exact recompiled function" cerr log that an unregistered PC would + // trigger (harmless to test state, but kept clean per the design's risk + // list). + static void stepNoop(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + (void)rdram; (void)ctx; (void)runtime; + } + + // Fiber A: push its own marker, hand off to B via sema, then park until B + // hands back. On resume A inspects its OWN trace. A fiber resuming out of + // WaitSema continues at the point of the call - it does NOT re-enter + // dispatchLoop - so the only writer into A's ring between A's push and + // A's post-resume read is B, and (under the fix) B writes into its OWN + // ring, never A's. + static void stepIsoA(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + runtime->lookupFunction(0x00700A80u); // push A's marker into A's ring + + R5900Context s{}; + setRegU32(s, 4, rdramRead32Raw(rdram, 0x00061004u)); // sidB + ps2_syscalls::SignalSema(rdram, &s, runtime); // let B run + + R5900Context w{}; + setRegU32(w, 4, rdramRead32Raw(rdram, 0x00061000u)); // sidA + ps2_syscalls::WaitSema(rdram, &w, runtime); // park; B runs meanwhile + + const std::string trace = runtime->debugCurrentDispatchTrace(); // A resumes: inspect A's own ring + rdramWrite32Raw(rdram, 0x00061010u, + (trace.find("700b80") != std::string::npos || + trace.find("700b00") != std::string::npos) ? 1u : 0u); // kRiAForeign + rdramWrite32Raw(rdram, 0x00061018u, + (trace.find("700a80") != std::string::npos) ? 1u : 0u); // kRiAOwn + ctx->pc = 0u; + } + + // Fiber B: wait for A's handoff, push its own marker, inspect its OWN + // trace, then signal A back. + static void stepIsoB(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + R5900Context w{}; + setRegU32(w, 4, rdramRead32Raw(rdram, 0x00061004u)); // sidB + ps2_syscalls::WaitSema(rdram, &w, runtime); + + runtime->lookupFunction(0x00700B80u); // push B's marker into B's ring + const std::string trace = runtime->debugCurrentDispatchTrace(); + rdramWrite32Raw(rdram, 0x00061014u, + (trace.find("700a80") != std::string::npos || + trace.find("700a00") != std::string::npos) ? 1u : 0u); // kRiBForeign + rdramWrite32Raw(rdram, 0x0006101Cu, + (trace.find("700b80") != std::string::npos) ? 1u : 0u); // kRiBOwn + + R5900Context s{}; + setRegU32(s, 4, rdramRead32Raw(rdram, 0x00061000u)); // sidA + ps2_syscalls::SignalSema(rdram, &s, runtime); // wake A + ctx->pc = 0u; + } + + // R2 probe: same entry PC for both the original fiber (run==0) and the + // fiber that reuses its tid after teardown (run==1). Both fibers get + // 0x00700C00 pushed into THEIR OWN ring automatically by dispatchLoop + // before this body runs, so "the ring started empty" cannot be tested as + // trace=="(empty)" (it never is, even under the fix). Instead we check + // whether the ring, at entry, already contains the OTHER fiber's distinct + // marker (0x00700C80, pushed only by run==0 after this check) - that is + // the signal that would only appear via cross-fiber leakage of the old + // shared thread_local. + static void stepReuseProbe(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + constexpr uint32_t kReuseMarker = 0x00700C80u; + const uint32_t run = rdramRead32Raw(rdram, 0x00062000u); // kReuseRun + const bool inheritedPrevMarker = + (runtime->debugCurrentDispatchTrace().find("700c80") != std::string::npos); + + if (run == 0u) + { + rdramWrite32Raw(rdram, 0x00062004u, inheritedPrevMarker ? 0u : 1u); // kReuseNotInherited0 + runtime->lookupFunction(kReuseMarker); + rdramWrite32Raw(rdram, 0x00062008u, + (runtime->debugCurrentDispatchTrace().find("700c80") != std::string::npos) ? 1u : 0u); // kReuseNonEmpty0 + } + else + { + rdramWrite32Raw(rdram, 0x0006200Cu, inheritedPrevMarker ? 0u : 1u); // kReuseNotInherited1 + } + ctx->pc = 0u; + } + + // ------------------------------------------------------------------ + // SchedulerStackIsolation control words and addresses (disjoint from + // other suites, which use <= 0x00062xxx) + // ------------------------------------------------------------------ + constexpr uint32_t kSiSemA = 0x00063000u; // ping-pong sema A + constexpr uint32_t kSiSemB = 0x00063004u; // ping-pong sema B + constexpr uint32_t kSiSysA = 0x00063008u; // override syscall number, fiber A + constexpr uint32_t kSiSysB = 0x0006300Cu; // override syscall number, fiber B + constexpr uint32_t kSiTopA = 0x00063010u; // B1: invoke $sp seen by A + constexpr uint32_t kSiTopB = 0x00063014u; // B1: invoke $sp seen by B + constexpr uint32_t kSiDmacTopA = 0x00063018u; // B3: handler $sp seen by A + constexpr uint32_t kSiDmacTopB = 0x0006301Cu; // B3: handler $sp seen by B + + // ---- I1 (B1): RPC/override invoke stack isolation ---------------------- + // Invoked BY rpcInvokeFunction; ctx is its internal `tmp`, so $29 is the + // scratch-stack top and $31 is rpcInvokeFunction's return sentinel. + static void stepInvokeRecordA(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + rdramWrite32Raw(rdram, kSiTopA, getRegU32(ctx, 29)); // record A's scratch top + R5900Context s{}; setRegU32(s, 4, rdramRead32Raw(rdram, kSiSemB)); + ps2_syscalls::SignalSema(rdram, &s, runtime); // let B proceed + R5900Context w{}; setRegU32(w, 4, rdramRead32Raw(rdram, kSiSemA)); + ps2_syscalls::WaitSema(rdram, &w, runtime); // PARK inside invoke; A's stack stays live + ctx->pc = getRegU32(ctx, 31); // resume: return to sentinel -> invoke loop ends + } + static void stepInvokeRecordB(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + rdramWrite32Raw(rdram, kSiTopB, getRegU32(ctx, 29)); // A's scratch STILL live here + R5900Context s{}; setRegU32(s, 4, rdramRead32Raw(rdram, kSiSemA)); + ps2_syscalls::SignalSema(rdram, &s, runtime); // wake A + ctx->pc = getRegU32(ctx, 31); + } + static void stepInvokeEntryA(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + ps2_syscalls::dispatchNumericSyscall(rdramRead32Raw(rdram, kSiSysA), rdram, ctx, runtime); + ctx->pc = 0u; + } + static void stepInvokeEntryB(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + R5900Context w{}; setRegU32(w, 4, rdramRead32Raw(rdram, kSiSemB)); + ps2_syscalls::WaitSema(rdram, &w, runtime); // wait until A recorded + parked + ps2_syscalls::dispatchNumericSyscall(rdramRead32Raw(rdram, kSiSysB), rdram, ctx, runtime); + ctx->pc = 0u; + } + + // ---- I2 (B3): inline DMAC handler stack isolation ---------------------- + // Runs AS a DMAC handler; ctx is runHandlers' irqCtx, so $29 is the + // reserved handler-stack top. + static void stepDmacRecordA(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + rdramWrite32Raw(rdram, kSiDmacTopA, getRegU32(ctx, 29)); + R5900Context s{}; setRegU32(s, 4, rdramRead32Raw(rdram, kSiSemB)); + ps2_syscalls::SignalSema(rdram, &s, runtime); + R5900Context w{}; setRegU32(w, 4, rdramRead32Raw(rdram, kSiSemA)); + ps2_syscalls::WaitSema(rdram, &w, runtime); // PARK inside dispatch; A's handler stack live + ctx->pc = 0u; // handler returns + } + static void stepDmacRecordB(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + rdramWrite32Raw(rdram, kSiDmacTopB, getRegU32(ctx, 29)); // A's handler stack STILL live + R5900Context s{}; setRegU32(s, 4, rdramRead32Raw(rdram, kSiSemA)); + ps2_syscalls::SignalSema(rdram, &s, runtime); // wake A + ctx->pc = 0u; + } + static void stepDmacDispatchA(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + ps2_syscalls::dispatchDmacHandlersForCause(rdram, runtime, 5u); // inline on fiber + ctx->pc = 0u; + } + static void stepDmacDispatchB(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + R5900Context w{}; setRegU32(w, 4, rdramRead32Raw(rdram, kSiSemB)); + ps2_syscalls::WaitSema(rdram, &w, runtime); // wait until A recorded + parked + ps2_syscalls::dispatchDmacHandlersForCause(rdram, runtime, 6u); + ctx->pc = 0u; + } + + // ------------------------------------------------------------------ + // SchedulerOverrideIsolation helpers + // + // dispatchSyscallOverride's reentrancy guard (System.cpp) used to be one + // process-wide `thread_local std::vector`. Under the N=1 fiber scheduler + // that vector is keyed to the ONE guest executor OS thread, shared by every + // fiber. So while fiber A is parked INSIDE its override for syscall N (N + // still on the shared vector because A hasn't returned to pop it), fiber B + // issuing the same N sees N "active", is treated as reentrant, and its + // override is SILENTLY SKIPPED in favor of the builtin. The fix moves the + // stack into FiberContext (per-fiber), so B has its own empty stack and its + // override runs. Wrong-dispatch, not memory-safety. + // + // The handler PARKS via WaitSema from inside rpcInvokeFunction (inside the + // handler invoke). On ucontext the fiber's whole C++ stack is frozen at the + // swap, so nesting depth is irrelevant (same mechanism stepIsoA relies on). + // RED ONLY ON build-ucontext: on the pthread backend each fiber is its own + // OS thread, the old thread_local is accidentally per-fiber, and this test + // is vacuously green even unfixed. + // ------------------------------------------------------------------ + constexpr uint32_t kOvSyscall = 0x79u; // free syscall #, benign TODO fallback + constexpr uint32_t kOvHandler = 0x00700D00u; // registered override handler + constexpr uint32_t kOvEntryA = 0x00700D80u; + constexpr uint32_t kOvEntryB = 0x00700E00u; + constexpr uint32_t kOvSidStartB = 0x00063020u; // A -> B: "A has parked inside its override" + constexpr uint32_t kOvSidResumeA = 0x00063024u; // B -> A: "you may resume" + constexpr uint32_t kOvAEntered = 0x00063028u; // A's handler was entered (liveness) + constexpr uint32_t kOvARan = 0x0006302Cu; // A's handler resumed & completed (liveness) + constexpr uint32_t kOvBRan = 0x00063030u; // B's handler ran => override NOT skipped (THE signal) + + // One handler for both fibers; $a0 selects mode. mode 1 = A (park), 2 = B (mark). + static void stepOverrideHandler(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + const uint32_t mode = getRegU32(ctx, 4); // $a0, forwarded by rpcInvokeFunction + if (mode == 1u) + { + rdramWrite32Raw(rdram, kOvAEntered, 1u); + R5900Context s{}; setRegU32(s, 4, rdramRead32Raw(rdram, kOvSidStartB)); + ps2_syscalls::SignalSema(rdram, &s, runtime); // release B + R5900Context w{}; setRegU32(w, 4, rdramRead32Raw(rdram, kOvSidResumeA)); + ps2_syscalls::WaitSema(rdram, &w, runtime); // PARK: N is on A's override stack + rdramWrite32Raw(rdram, kOvARan, 1u); // resumed + } + else + { + rdramWrite32Raw(rdram, kOvBRan, 1u); // B's override actually ran + } + setReturnU32(ctx, 0u); + ctx->pc = getRegU32(ctx, 31); // return through rpcInvoke sentinel + } + + static void stepOverrideA(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + setRegU32(*ctx, 4, 1u); // mode A (park) + runtime->handleSyscall(rdram, ctx, kOvSyscall); // -> dispatchSyscallOverride -> handler parks + ctx->pc = 0u; + } + + static void stepOverrideB(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + R5900Context w{}; setRegU32(w, 4, rdramRead32Raw(rdram, kOvSidStartB)); + ps2_syscalls::WaitSema(rdram, &w, runtime); // proceed only after A has parked + setRegU32(*ctx, 4, 2u); // mode B (mark) + runtime->handleSyscall(rdram, ctx, kOvSyscall); // fix: B's override runs; bug: skipped -> builtin + R5900Context s{}; setRegU32(s, 4, rdramRead32Raw(rdram, kOvSidResumeA)); + ps2_syscalls::SignalSema(rdram, &s, runtime); // wake A regardless -> clean drain in both cases + ctx->pc = 0u; + } + + // ------------------------------------------------------------------ + // SchedulerJoinStarvation control words + step functions. + // + // Reproduces the TerminateThread hang caused by join_fiber flooring the + // joiner one level BELOW the target (t->priority + 1) instead of AT the + // target's level. With the +1 floor, an equal-priority sibling that stays + // runnable after the target exits keeps the joiner (now one level lower) + // permanently off the run-queue head, so join_fiber never re-observes the + // target as finished and TerminateThread never returns. + // ------------------------------------------------------------------ + constexpr uint32_t kJsTargetTid = 0x00062000u; // int32: target (B) tid, read by joiner A + constexpr uint32_t kJsJoinReturned = 0x00062004u; // 1 => A's TerminateThread(B) returned + constexpr uint32_t kJsBStarted = 0x00062008u; // 1 => target B is running + + // Target B (prio 10): announce running, then spin on the recompiler + // back-edge hook until terminated. B never self-exits — it is killed by + // A's TerminateThread(B), which guarantees B is still joinable when A + // enters join_fiber (so the priority floor is actually exercised). The + // yield_point inside shouldPreemptGuestExecution throws ThreadExitException + // once terminateRequested is set (by A, or by scheduler_shutdown on + // teardown), unwinding B cleanly. kThStop is a belt-and-suspenders escape. + static void stepJoinStarveTarget(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + rdramWrite32Raw(rdram, kJsBStarted, 1u); + while (rdramRead32Raw(rdram, kThStop) == 0u) + { + runtime->shouldPreemptGuestExecution(); // yield_point: throws on terminate + } + ctx->pc = 0u; + } + + // Joiner A (prio 5, strictly higher than B and the prio-10 pingers): + // TerminateThread(B) routes through request_terminate + join_fiber. The + // floor demotes A to the target's level (10, fixed) or one below (11, + // buggy). Only the fixed floor lets A rotate back to the run-queue head + // (FIFO among the prio-10 group) to observe B finished and return; the + // buggy floor strands A below the ever-runnable pingers forever. A sets + // kJsJoinReturned only AFTER TerminateThread returns. + static void stepJoinStarveJoiner(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + const int32_t targetTid = static_cast(rdramRead32Raw(rdram, kJsTargetTid)); + R5900Context tcx{}; + setRegU32(tcx, 4, static_cast(targetTid)); + ps2_syscalls::TerminateThread(rdram, &tcx, runtime); // -> join_fiber(targetTid) + rdramWrite32Raw(rdram, kJsJoinReturned, 1u); + ctx->pc = 0u; + } + +} // anonymous namespace + +// --------------------------------------------------------------------------- +// Token handoff: a host worker parked in async_guest_begin() must win the +// guest token within bounded time even while guest fibers keep the run queue +// non-empty. Regression test for the executor resume predicate: without the +// g_host_token_waiters gate the executor re-resumes fibers without ever +// releasing g_sched_mutex into a cv wait, so the parked worker never runs +// (the VBlank-starvation shape: a guest polling for vsync ticks in a +// semaphore/flag main loop keeps the run queue non-empty, so the starved +// interrupt worker cannot deliver the ticks the guest is waiting on). +// --------------------------------------------------------------------------- +void register_scheduler_token_handoff_tests() +{ + MiniTest::Case("SchedulerTokenHandoff", [](TestCase &tc) + { + tc.Run("H1: parked host worker wins the token while fibers ping-pong", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00700000u, &stepPingA); + runtime.registerFunction(0x00700100u, &stepPingB); + + const int32_t sidX = createSchedSema(rdram.data(), &runtime, 1, 1); // seeded permit + const int32_t sidY = createSchedSema(rdram.data(), &runtime, 0, 1); + t.IsTrue(sidX > 0 && sidY > 0, "H1: semas created"); + if (sidX <= 0 || sidY <= 0) + { + return; + } + rdramWrite32(rdram, kThStop, 0u); + rdramWrite32(rdram, kThSidX, static_cast(sidX)); + rdramWrite32(rdram, kThSidY, static_cast(sidY)); + rdramWrite32(rdram, kThCount, 0u); + + const int32_t tidA = startSchedWorker(rdram.data(), &runtime, + 0x00700000u, 10, 0x00510000u, 0x2000u); + const int32_t tidB = startSchedWorker(rdram.data(), &runtime, + 0x00700100u, 10, 0x00514000u, 0x2000u); + t.IsTrue(tidA > 0 && tidB > 0, "H1: ping-pong fibers started"); + if (tidA <= 0 || tidB <= 0) + { + rdramWrite32(rdram, kThStop, 1u); + signalSchedSema(rdram.data(), &runtime, sidX); + signalSchedSema(rdram.data(), &runtime, sidY); + deleteSchedSema(rdram.data(), &runtime, sidX); + deleteSchedSema(rdram.data(), &runtime, sidY); + return; + } + + // Wait until the ping-pong is demonstrably live. + const bool spinning = waitUntil([&]() + { + return rdramRead32(rdram, kThCount) >= 3u; + }, std::chrono::milliseconds(2000)); + t.IsTrue(spinning, "H1: ping-pong fibers are running"); + + // Host worker (interrupt-worker shape): parks for the guest token. + std::atomic tokenAcquired{false}; + std::atomic workerDone{false}; + std::thread worker([&]() + { + g_currentThreadId = -1; // non-fiber host worker + ps2sched::async_guest_begin(); + tokenAcquired.store(true, std::memory_order_release); + ps2sched::async_guest_end(); + workerDone.store(true, std::memory_order_release); + }); + + // THE regression assertion: the worker must win the token while the + // fibers are still ping-ponging (bounded wait — a starved worker + // fails here instead of hanging the binary). + const bool acquiredWhileBusy = waitUntil([&]() + { + return tokenAcquired.load(std::memory_order_acquire); + }, std::chrono::milliseconds(3000)); + + // Escape hatch + orderly teardown (runs regardless of the verdict: + // once the fibers exit, the executor idles and the worker's + // async_guest_begin proceeds, so join() below cannot hang). + rdramWrite32(rdram, kThStop, 1u); + signalSchedSema(rdram.data(), &runtime, sidX); + signalSchedSema(rdram.data(), &runtime, sidY); + + const bool drained = waitUntil([&]() + { + return g_activeThreads.load(std::memory_order_acquire) <= 0; + }, std::chrono::milliseconds(3000)); + + const bool workerFinished = waitUntil([&]() + { + return workerDone.load(std::memory_order_acquire); + }, std::chrono::milliseconds(3000)); + if (worker.joinable()) + { + worker.join(); + } + + t.IsTrue(acquiredWhileBusy, + "H1: parked host worker acquired the guest token while fibers were busy " + "(executor must sleep while g_host_token_waiters > 0)"); + t.IsTrue(drained, "H1: ping-pong fibers drained after stop"); + t.IsTrue(workerFinished, "H1: host worker completed"); + + const uint32_t loops = rdramRead32(rdram, kThCount); + t.IsTrue(loops >= 3u, "H1: guest made progress during the test"); + + deleteSchedSema(rdram.data(), &runtime, sidX); + deleteSchedSema(rdram.data(), &runtime, sidY); + }); + tc.Run("H2: spinning fiber yields at yield_point when a host worker is parked", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00700200u, &stepSpinShouldPreempt); + rdramWrite32(rdram, kThStop, 0u); + rdramWrite32(rdram, kThCount, 0u); + + const int32_t tid = startSchedWorker(rdram.data(), &runtime, + 0x00700200u, 10, 0x00518000u, 0x2000u); + t.IsTrue(tid > 0, "H2: spin fiber started"); + if (tid <= 0) + { + return; + } + + const bool spinning = waitUntil([&]() + { + return rdramRead32(rdram, kThCount) >= 1000u; + }, std::chrono::milliseconds(2000)); + t.IsTrue(spinning, "H2: fiber is spinning through yield_point samples"); + + std::atomic tokenAcquired{false}; + std::atomic workerDone{false}; + std::thread worker([&]() + { + g_currentThreadId = -1; // non-fiber host worker + ps2sched::async_guest_begin(); + tokenAcquired.store(true, std::memory_order_release); + ps2sched::async_guest_end(); + workerDone.store(true, std::memory_order_release); + }); + + // THE regression assertion: without yield_point step 4 the fiber + // never leaves ps2fiber_resume, so the worker cannot win the token + // while the spin is live (bounded wait; escape hatch below). + const bool acquiredWhileSpinning = waitUntil([&]() + { + return tokenAcquired.load(std::memory_order_acquire); + }, std::chrono::milliseconds(3000)); + + // Escape hatch + teardown: end the spin; the fiber exits, the + // executor idles, and the parked worker (if still parked) proceeds. + rdramWrite32(rdram, kThStop, 1u); + + const bool drained = waitUntil([&]() + { + return g_activeThreads.load(std::memory_order_acquire) <= 0; + }, std::chrono::milliseconds(3000)); + const bool workerFinished = waitUntil([&]() + { + return workerDone.load(std::memory_order_acquire); + }, std::chrono::milliseconds(3000)); + if (worker.joinable()) + { + worker.join(); + } + + t.IsTrue(acquiredWhileSpinning, + "H2: parked host worker acquired the token while the fiber was spinning " + "(yield_point must yield when host_token_waiters > 0)"); + t.IsTrue(drained, "H2: spin fiber drained after stop"); + t.IsTrue(workerFinished, "H2: host worker completed"); + }); + tc.Run("H3: cross-dispatch spin yields via the dispatch loop preempt hook", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00700300u, &stepCrossDispatchA); + runtime.registerFunction(0x00700400u, &stepCrossDispatchB); + rdramWrite32(rdram, kThStop, 0u); + rdramWrite32(rdram, kThCount, 0u); + + const int32_t tid = startSchedWorker(rdram.data(), &runtime, + 0x00700300u, 10, 0x0051C000u, 0x2000u); + t.IsTrue(tid > 0, "H3: cross-dispatch fiber started"); + if (tid <= 0) + { + return; + } + + const bool spinning = waitUntil([&]() + { + return rdramRead32(rdram, kThCount) >= 1000u; + }, std::chrono::milliseconds(2000)); + t.IsTrue(spinning, "H3: fiber is spinning across function dispatches"); + + std::atomic tokenAcquired{false}; + std::atomic workerDone{false}; + std::thread worker([&]() + { + g_currentThreadId = -1; // non-fiber host worker + ps2sched::async_guest_begin(); + tokenAcquired.store(true, std::memory_order_release); + ps2sched::async_guest_end(); + workerDone.store(true, std::memory_order_release); + }); + + // THE regression assertion: the function bodies never call the + // back-edge hook, so only the dispatch-loop preempt can yield. + const bool acquiredWhileSpinning = waitUntil([&]() + { + return tokenAcquired.load(std::memory_order_acquire); + }, std::chrono::milliseconds(3000)); + + // Escape hatch + teardown. + rdramWrite32(rdram, kThStop, 1u); + + const bool drained = waitUntil([&]() + { + return g_activeThreads.load(std::memory_order_acquire) <= 0; + }, std::chrono::milliseconds(3000)); + const bool workerFinished = waitUntil([&]() + { + return workerDone.load(std::memory_order_acquire); + }, std::chrono::milliseconds(3000)); + if (worker.joinable()) + { + worker.join(); + } + + t.IsTrue(acquiredWhileSpinning, + "H3: parked host worker acquired the token during a cross-dispatch spin " + "(dispatchLoop must call shouldPreemptGuestExecution)"); + t.IsTrue(drained, "H3: cross-dispatch fiber drained after stop"); + t.IsTrue(workerFinished, "H3: host worker completed"); + }); + }); // MiniTest::Case("SchedulerTokenHandoff") +} + +// --------------------------------------------------------------------------- +// sceSifRpcLoop must PARK its thread, not return. The real kernel loop is +// `while (1) { SleepThread(); serve; }`; with all SIF RPC HLE'd host-side no +// wakeup ever arrives, so a stub that returns turns the RPC server thread +// into a hot spin that monopolizes the N=1 executor (any SIF-RPC server +// thread — pad, memcard, audio, filesystem — then starves the main thread +// forever). +// --------------------------------------------------------------------------- +void register_scheduler_rpc_loop_park_tests() +{ + MiniTest::Case("SchedulerRpcLoopPark", [](TestCase &tc) + { + tc.Run("R1: sceSifRpcLoop parks its fiber and does not monopolize the executor", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00700500u, &stepRpcServerLoop); + runtime.registerFunction(0x00700600u, &stepWriteFlagAndExit); + rdramWrite32(rdram, kThFlag, 0u); + + // RPC server first (it runs first and, unfixed, never lets go). + const int32_t serverTid = startSchedWorker(rdram.data(), &runtime, + 0x00700500u, 10, 0x00520000u, 0x2000u); + // Probe fiber at the SAME priority: only a genuine park (not a + // priority preempt) can let it run. + const int32_t probeTid = startSchedWorker(rdram.data(), &runtime, + 0x00700600u, 10, 0x00524000u, 0x2000u); + t.IsTrue(serverTid > 0 && probeTid > 0, "R1: both fibers started"); + if (serverTid <= 0 || probeTid <= 0) + { + return; + } + + // THE regression assertion: the probe fiber runs within bounded + // time, i.e. the RPC server fiber parked in SleepThread instead of + // re-entering the stub forever. + const bool probeRan = waitUntil([&]() + { + return rdramRead32(rdram, kThFlag) == 1u; + }, std::chrono::milliseconds(3000)); + + t.IsTrue(probeRan, + "R1: equal-priority fiber ran while sceSifRpcLoop was active " + "(the RPC server fiber must park via SleepThread)"); + + // Teardown: scheduler_shutdown() terminates the parked server + // fiber (SleepThread observes terminateRequested and unwinds). + // Bounded even on regression: shutdown's terminate also unwinds a + // hot-spinning server at its next dispatch-loop yield point. + ps2sched::scheduler_shutdown(); + t.Equals(g_activeThreads.load(std::memory_order_acquire), 0, + "R1: all fibers terminated after shutdown"); + }); + }); // MiniTest::Case("SchedulerRpcLoopPark") +} + +// --------------------------------------------------------------------------- +// requestStop() from GUEST context (a fiber on the executor thread) must not +// join host workers: a worker parked in async_guest_begin() waits for +// g_running_fiber == nullptr, which can never happen while the joining fiber +// IS the running fiber - a deadlock (reached whenever the unimplemented- +// function default fault handler calls requestStop from a fiber while an +// interrupt worker is parked for the token). notifyRuntimeStop must +// signal-only in that case; scheduler_shutdown() joins later on the main +// thread. +// --------------------------------------------------------------------------- +void register_scheduler_guest_context_stop_tests() +{ + MiniTest::Case("SchedulerGuestContextStop", [](TestCase &tc) + { + tc.Run("G1: requestStop from a fiber does not deadlock against a parked worker", [](TestCase &t) + { + notifyRuntimeStop(); + ps2sched::scheduler_init(); + PS2Runtime runtime; + std::vector rdram(PS2_RAM_SIZE, 0u); + + runtime.registerFunction(0x00700700u, &stepGuestContextStop); + rdramWrite32(rdram, kThStop, 0u); + rdramWrite32(rdram, kThFlag, 0u); + + // Real interrupt worker: each VBlank tick it takes AsyncGuestScope + // (even with zero INTC handlers), parking in async_guest_begin() + // while our spinning fiber holds the executor. + EnsureVSyncWorkerRunning(rdram.data(), &runtime); + + const int32_t tid = startSchedWorker(rdram.data(), &runtime, + 0x00700700u, 10, 0x00528000u, 0x2000u); + t.IsTrue(tid > 0, "G1: fiber started"); + if (tid <= 0) + { + ps2sched::scheduler_shutdown(); runtime.requestStop(); return; + } + + // THE regression assertion: the fiber observes the parked worker, + // calls requestStop() from guest context, and RETURNS (writing the + // flag). Unfixed, notifyRuntimeStop joins the parked worker from + // the fiber and wedges before the flag write. + const bool stopReturned = waitUntil([&]() + { + return rdramRead32(rdram, kThFlag) == 1u; + }, std::chrono::milliseconds(5000)); + + t.IsTrue(stopReturned, + "G1: requestStop() invoked on the guest executor returned " + "(worker stop must be signal-only from guest context)"); + + if (!stopReturned) + { + // The fiber is wedged inside the join; the executor cannot be + // shut down. Return WITHOUT scheduler_shutdown so the failure + // above is reported instead of hanging here. (The next suite's + // scheduler_init will SCHED_REQUIRE-abort the process - a loud + // bounded failure, never a silent hang.) + return; + } + + const bool drained = waitUntil([&]() + { + return g_activeThreads.load(std::memory_order_acquire) <= 0; + }, std::chrono::milliseconds(3000)); + t.IsTrue(drained, "G1: fiber drained after guest-context stop"); + + // Main-thread shutdown performs the real joins (idempotent stops). + ps2sched::scheduler_shutdown(); + runtime.requestStop(); + }); + }); // MiniTest::Case("SchedulerGuestContextStop") +} + +// --------------------------------------------------------------------------- +// Async callback stacks must live in kernel-reserved memory, never at +// top-of-RAM. Games place their main stack at top-of-RAM via SetupThread +// (SDK crt0 places the main stack at the top of user RAM, and the boot $sp +// defaults to PS2_RAM_SIZE - 0x10); the old pool +// carved down from PS2_RAM_SIZE, so host-dispatched guest callbacks (GS +// vsync / INTC / alarms) ran ON the guest's live main stack and both sides' +// register spills corrupted each other (garbage callback pointers, $ra +// clobbers near the stack top). +// --------------------------------------------------------------------------- +void register_runtime_async_stack_pool_tests() +{ + MiniTest::Case("RuntimeAsyncStackPool", [](TestCase &tc) + { + tc.Run("S1: callback stack pool is kernel-area and disjoint from top-of-RAM", [](TestCase &t) + { + constexpr uint32_t kPoolFloor = 0x00080000u; + constexpr uint32_t kPoolTop = 0x00100000u; + constexpr uint32_t kStackSize = 0x4000u; + + PS2Runtime runtime; + + // First reservation: this is the stack the GS vsync-callback path + // grabs at boot. Under the old top-of-RAM pool it was 0x1FFFFF0 - + // inside the guest's main stack. + const uint32_t first = runtime.reserveAsyncCallbackStack(kStackSize, 16u); + t.Equals(first, kPoolTop - 0x10u, "S1: first reservation tops the kernel pool"); + t.IsTrue(first < kPoolTop, "S1: first reservation is below the ELF load base"); + t.IsTrue(first >= kPoolFloor, "S1: first reservation is above the pool floor"); + + // Exhaust the pool: every reservation must stay inside + // [kPoolFloor, kPoolTop), and the pool must hold exactly + // (kPoolTop - kPoolFloor) / kStackSize stacks before returning 0 (bounded loop). + uint32_t count = 1u; + bool allInPool = (first >= kPoolFloor && first < kPoolTop); + for (uint32_t i = 0u; i < 64u; ++i) + { + const uint32_t top = runtime.reserveAsyncCallbackStack(kStackSize, 16u); + if (top == 0u) + { + break; + } + ++count; + allInPool = allInPool && (top >= kPoolFloor && top < kPoolTop); + } + t.IsTrue(allInPool, "S1: every reservation stays inside the kernel pool"); + t.Equals(count, (kPoolTop - kPoolFloor) / kStackSize, + "S1: pool capacity matches [0x80000, 0x100000) / 16KB"); + + runtime.requestStop(); + }); + }); // MiniTest::Case("RuntimeAsyncStackPool") +} + +// --------------------------------------------------------------------------- +// dispatchDmacHandlersForCause is reachable from BOTH sides of the guest +// token: from host workers (IRQ worker, DMA completion paths) AND +// synchronously from guest syscall context (sceSifSetDma calls it inline +// while a fiber services the syscall). It borrowed the token via +// AsyncGuestScope unconditionally, and async_guest_begin() aborts by design +// on the guest executor thread - so a guest program arming a DMAC handler +// and then calling sceSifSetDma killed the process deterministically +// ("FATAL [ps2sched]: async_guest_begin from guest executor thread"; any +// guest that arms a DMAC handler and then issues a SIF transfer — e.g. a +// sound-driver init path — hits this). +// --------------------------------------------------------------------------- +void register_scheduler_dmac_guest_dispatch_tests() +{ + MiniTest::Case("SchedulerDmacGuestDispatch", [](TestCase &tc) + { + tc.Run("M1: DMAC dispatch from guest syscall context runs handlers without aborting", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00700800u, &stepDmacHandler); + runtime.registerFunction(0x00700900u, &stepDmacFromGuest); + rdramWrite32(rdram, kThFlag, 0u); + rdramWrite32(rdram, kThSent, 0u); + + // Arm a DMAC handler on cause 5 (SIF0), like sceSifSetDma clients do. + R5900Context ac{}; + setRegU32(ac, 4, 5u); // cause + setRegU32(ac, 5, 0x00700800u); // handler + setRegU32(ac, 6, 0u); // next + setRegU32(ac, 7, 0u); // arg + ps2_syscalls::AddDmacHandler(rdram.data(), &ac, &runtime); + t.IsTrue(getRegS32(ac, 2) > 0, "M1: DMAC handler registered"); + + // Guest-context dispatch: the fiber calls + // dispatchDmacHandlersForCause synchronously. Against the + // unconditional AsyncGuestScope this std::terminate()s the + // process - a loud bounded failure, never a hang. + const int32_t tid = startSchedWorker(rdram.data(), &runtime, + 0x00700900u, 10, 0x0052C000u, 0x2000u); + t.IsTrue(tid > 0, "M1: dispatching fiber started"); + if (tid <= 0) + { + return; + } + + const bool done = waitUntil([&]() + { + return rdramRead32(rdram, kThFlag) == 1u; + }, std::chrono::milliseconds(3000)); + t.IsTrue(done, "M1: guest-context dispatch returned"); + t.Equals(rdramRead32(rdram, kThSent), 1u, + "M1: handler ran exactly once from guest context"); + + const bool drained = waitUntil([&]() + { + return g_activeThreads.load(std::memory_order_acquire) <= 0; + }, std::chrono::milliseconds(3000)); + t.IsTrue(drained, "M1: fiber drained"); + + // Host-thread path must still borrow the token and work: dispatch + // the same cause from this (non-executor) thread. + ps2_syscalls::dispatchDmacHandlersForCause(rdram.data(), &runtime, 5u); + t.Equals(rdramRead32(rdram, kThSent), 2u, + "M1: handler also runs via the borrowed-token host path"); + }); + }); // MiniTest::Case("SchedulerDmacGuestDispatch") +} + +// --------------------------------------------------------------------------- +// The diagnostic dispatch-trace ring (the "trace=..." string logged by +// lookupFunction/reportMissingFunction/dispatchLoop) used to be a single +// process-wide `thread_local DispatchHistory` in ps2_runtime.cpp. Under the +// N=1 fiber scheduler that thread_local is keyed to the ONE guest executor +// OS thread, not to any individual fiber - so EVERY fiber that ever ran on +// that thread shared and overwrote the same ring. Two concrete failures fall +// out of that: +// R1 (isolation): while fiber A is legitimately still alive (parked in +// WaitSema, about to resume), a second live fiber B pushes into the +// "same" ring, so A's post-resume trace is contaminated with B's PCs - +// a diagnostic reading a completely unrelated fiber's history. +// R2 (freshness on tid reuse): after a fiber exits and its tid is +// genuinely recycled by CreateThread's tid allocator, the new fiber at +// that tid inherits the old fiber's leftover ring contents, because the +// thread_local lives on the OS thread, not the (now-destroyed) fiber. +// The fix embeds the ring directly in FiberContext (fresh at construction, +// destroyed with the fiber; see ps2_scheduler_internal.h), so both failure +// modes are structurally impossible: there is no shared, outlives-the-fiber +// storage left to leak through. +// --------------------------------------------------------------------------- +void register_scheduler_recovery_isolation_tests() +{ + MiniTest::Case("SchedulerRecoveryIsolation", [](TestCase &tc) + { + tc.Run("R1: dispatch-trace ring is isolated across two live interleaved fibers", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + constexpr uint32_t kEntryA = 0x00700A00u; + constexpr uint32_t kMarkerA = 0x00700A80u; + constexpr uint32_t kEntryB = 0x00700B00u; + constexpr uint32_t kMarkerB = 0x00700B80u; + constexpr uint32_t kRiSidA = 0x00061000u; + constexpr uint32_t kRiSidB = 0x00061004u; + constexpr uint32_t kRiAForeign = 0x00061010u; // A saw B's marker? (bug => 1) + constexpr uint32_t kRiBForeign = 0x00061014u; // B saw A's marker? (bug => 1) + constexpr uint32_t kRiAOwn = 0x00061018u; // A saw its own marker? (must be 1) + constexpr uint32_t kRiBOwn = 0x0006101Cu; // B saw its own marker? (must be 1) + + runtime.registerFunction(kEntryA, &stepIsoA); + runtime.registerFunction(kEntryB, &stepIsoB); + runtime.registerFunction(kMarkerA, &stepNoop); + runtime.registerFunction(kMarkerB, &stepNoop); + + // Both semas start at 0: A signals B then waits on A; B waits on B + // then signals A - a one-shot ping-pong handoff, not a loop. + const int32_t sidA = createSchedSema(rdram.data(), &runtime, 0, 1); + const int32_t sidB = createSchedSema(rdram.data(), &runtime, 0, 1); + t.IsTrue(sidA > 0 && sidB > 0, "R1: semas created"); + if (sidA <= 0 || sidB <= 0) + { + return; + } + rdramWrite32(rdram, kRiSidA, static_cast(sidA)); + rdramWrite32(rdram, kRiSidB, static_cast(sidB)); + rdramWrite32(rdram, kRiAForeign, 0u); + rdramWrite32(rdram, kRiBForeign, 0u); + rdramWrite32(rdram, kRiAOwn, 0u); + rdramWrite32(rdram, kRiBOwn, 0u); + + const int32_t tidA = startSchedWorker(rdram.data(), &runtime, + kEntryA, 10, 0x00530000u, 0x2000u); + const int32_t tidB = startSchedWorker(rdram.data(), &runtime, + kEntryB, 10, 0x00534000u, 0x2000u); + t.IsTrue(tidA > 0 && tidB > 0, "R1: both fibers started"); + if (tidA <= 0 || tidB <= 0) + { + deleteSchedSema(rdram.data(), &runtime, sidA); + deleteSchedSema(rdram.data(), &runtime, sidB); + return; + } + + const bool drained = waitUntil([&]() + { + return g_activeThreads.load(std::memory_order_acquire) <= 0; + }, std::chrono::milliseconds(3000)); + t.IsTrue(drained, "R1: both fibers completed the handoff and exited"); + + // THE regression assertions: under the shared thread_local, A and + // B push into the SAME ring, so each fiber's post-handoff trace + // is contaminated with the other's markers. + t.Equals(rdramRead32(rdram, kRiAForeign), 0u, + "R1: fiber A's trace must not contain fiber B's markers"); + t.Equals(rdramRead32(rdram, kRiBForeign), 0u, + "R1: fiber B's trace must not contain fiber A's markers"); + t.Equals(rdramRead32(rdram, kRiAOwn), 1u, + "R1: fiber A's trace must contain fiber A's own marker"); + t.Equals(rdramRead32(rdram, kRiBOwn), 1u, + "R1: fiber B's trace must contain fiber B's own marker"); + + deleteSchedSema(rdram.data(), &runtime, sidA); + deleteSchedSema(rdram.data(), &runtime, sidB); + }); + + tc.Run("R2: dispatch-trace ring is fresh after genuine tid reuse", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + constexpr uint32_t kEntryReuse = 0x00700C00u; + constexpr uint32_t kReuseMarker = 0x00700C80u; + constexpr uint32_t kReuseRun = 0x00062000u; // 0 => first fiber, 1 => reused + constexpr uint32_t kReuseNotInherited0 = 0x00062004u; // fiber1 did not inherit a prior marker (expect 1) + constexpr uint32_t kReuseNonEmpty0 = 0x00062008u; // fiber1 pushed its own marker (expect 1) + constexpr uint32_t kReuseNotInherited1 = 0x0006200Cu; // fiber2 (reused tid) did not inherit fiber1's marker (expect 1) + + runtime.registerFunction(kEntryReuse, &stepReuseProbe); + runtime.registerFunction(kReuseMarker, &stepNoop); + rdramWrite32(rdram, kReuseRun, 0u); + rdramWrite32(rdram, kReuseNotInherited0, 0u); + rdramWrite32(rdram, kReuseNonEmpty0, 0u); + rdramWrite32(rdram, kReuseNotInherited1, 0u); + + const int32_t T1 = startSchedWorker(rdram.data(), &runtime, + kEntryReuse, 10, 0x00538000u, 0x2000u); + t.IsTrue(T1 > 0, "R2: fiber 1 started"); + if (T1 <= 0) + { + return; + } + + const bool drained1 = waitUntil([&]() + { + return g_activeThreads.load(std::memory_order_acquire) <= 0; + }, std::chrono::milliseconds(3000)); + t.IsTrue(drained1, "R2: fiber 1 fully torn down (FiberContext destroyed)"); + + // Force GENUINE tid reuse: erase T1 from the kernel thread map + // (DeleteThread requires THS_DORMANT, which on_fiber_exit already + // set), then seed the tid allocator so the next CreateThread hands + // out exactly T1 again. Both operations are serialized under + // g_thread_map_mutex, the same lock CreateThread's allocator holds. + { + R5900Context d{}; + setRegU32(d, 4, static_cast(T1)); + ps2_syscalls::DeleteThread(rdram.data(), &d, &runtime); + } + { + std::lock_guard lk(g_thread_map_mutex); + g_nextThreadId = T1; + } + + rdramWrite32(rdram, kReuseRun, 1u); + const int32_t T2 = startSchedWorker(rdram.data(), &runtime, + kEntryReuse, 10, 0x0053C000u, 0x2000u); + t.Equals(T2, T1, "R2: tid genuinely recycled (not merely a fresh, different tid)"); + + const bool drained2 = waitUntil([&]() + { + return g_activeThreads.load(std::memory_order_acquire) <= 0; + }, std::chrono::milliseconds(3000)); + t.IsTrue(drained2, "R2: fiber 2 (reused tid) completed"); + + // THE regression assertion: under the shared thread_local, fiber 2 + // (running on the same executor OS thread) sees fiber 1's leftover + // 0x00700c80 marker still sitting in the one global ring. + t.Equals(rdramRead32(rdram, kReuseNotInherited0), 1u, + "R2: fiber 1 (first ever on this tid) starts with no prior marker"); + t.Equals(rdramRead32(rdram, kReuseNonEmpty0), 1u, + "R2: fiber 1 successfully pushed its own marker"); + t.Equals(rdramRead32(rdram, kReuseNotInherited1), 1u, + "R2: fiber 2 (reused tid) must NOT inherit fiber 1's marker"); + }); + }); // MiniTest::Case("SchedulerRecoveryIsolation") +} + +// --------------------------------------------------------------------------- +// Shared-guest-stack bug class (B1/B3/B4): rpcInvokeFunction (RPC/override +// invoke), inline DMAC handler dispatch, and MPEG stream callbacks each ran +// recompiled guest code on a per-OS-thread `thread_local` scratch stack. Under +// the N=1 fiber scheduler every fiber runs on the ONE guest-executor OS +// thread, so that thread_local is a SINGLE stack shared by every fiber (and +// by one fiber re-entering the same path). A fiber can yield mid-invoke (the +// invoked guest body hits a back-edge / blocking syscall), so a second fiber +// entering the same path sets $sp to the SAME top and overwrites the parked +// fiber's live frames. Fixed by GuestScratchStack: a fresh, RAII-released +// guest-heap reservation per invocation (see ps2_runtime.h), so interleaved +// or re-entrant invocations always run on disjoint stacks. +// +// I1/I2 are RED on build-ucontext against the old shared thread_local +// (topA == topB) and GREEN with the fix (disjoint reservations). On the +// pthread backend each fiber is its own OS thread, so the old thread_local is +// already per-fiber and both tests pass even unfixed — mirrors the existing +// SchedulerRecoveryIsolation banner note. I3 exercises the GuestScratchStack +// contract directly (stands in for B4 — dispatchGuestStreamCallback is +// anon-namespace-internal and reachable only through the full MPEG decode +// path, disproportionate to stand up for a unit test; B4's site uses the same +// helper proven here and at I1/I2). +// +// LOAD-BEARING CHOREOGRAPHY: a sequential (non-overlapping) two-fiber test +// FALSE-PASSES even on the fix, because guestMalloc recycles a just-freed +// address — if fiber A frees before B allocates, topA == topB regardless of +// the fix. Both reservations must be LIVE SIMULTANEOUSLY: fiber A parks +// INSIDE its invoke/dispatch (its GuestScratchStack not yet destructed) via a +// semaphore wait executed from within the invoked guest body / handler body, +// while fiber B allocates and records. Do not "simplify" this overlap away. +// --------------------------------------------------------------------------- +void register_scheduler_stack_isolation_tests() +{ + MiniTest::Case("SchedulerStackIsolation", [](TestCase &tc) + { + // I1 --- B1: two fibers interleaving through rpcInvokeFunction must run + // on disjoint scratch stacks. rpcInvokeFunction gets its stack from a + // fresh, per-invocation GuestScratchStack reservation (see + // ps2_runtime.h), so two interleaved invocations never share $sp + // (topA != topB). + tc.Run("I1: rpc/override invoke stack is isolated across interleaved fibers", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + constexpr uint32_t kSysA = 0x00005A01u, kSysB = 0x00005A02u; + constexpr uint32_t kEntryA = 0x00701000u, kEntryB = 0x00701100u; + constexpr uint32_t kInvA = 0x00701200u, kInvB = 0x00701300u; + + runtime.registerFunction(kEntryA, &stepInvokeEntryA); + runtime.registerFunction(kEntryB, &stepInvokeEntryB); + runtime.registerFunction(kInvA, &stepInvokeRecordA); + runtime.registerFunction(kInvB, &stepInvokeRecordB); + + // Register two DISTINCT syscall overrides (distinct numbers dodge + // B2's shared s_activeSyscallOverrides reentrancy guard; that bug + // is tracked separately by SchedulerOverrideIsolation). + { R5900Context s{}; setRegU32(s,4,kSysA); setRegU32(s,5,kInvA); + ps2_syscalls::SetSyscall(rdram.data(), &s, &runtime); } + { R5900Context s{}; setRegU32(s,4,kSysB); setRegU32(s,5,kInvB); + ps2_syscalls::SetSyscall(rdram.data(), &s, &runtime); } + + const int32_t sidA = createSchedSema(rdram.data(), &runtime, 0, 1); + const int32_t sidB = createSchedSema(rdram.data(), &runtime, 0, 1); + t.IsTrue(sidA > 0 && sidB > 0, "I1: semas created"); + rdramWrite32(rdram, kSiSemA, static_cast(sidA)); + rdramWrite32(rdram, kSiSemB, static_cast(sidB)); + rdramWrite32(rdram, kSiSysA, kSysA); + rdramWrite32(rdram, kSiSysB, kSysB); + rdramWrite32(rdram, kSiTopA, 0u); + rdramWrite32(rdram, kSiTopB, 0u); + + const int32_t tidA = startSchedWorker(rdram.data(), &runtime, kEntryA, 10, 0x00540000u, 0x2000u); + const int32_t tidB = startSchedWorker(rdram.data(), &runtime, kEntryB, 10, 0x00544000u, 0x2000u); + t.IsTrue(tidA > 0 && tidB > 0, "I1: both fibers started"); + + const bool drained = waitUntil([&]{ return g_activeThreads.load(std::memory_order_acquire) <= 0; }, + std::chrono::milliseconds(3000)); + t.IsTrue(drained, "I1: both fibers completed the handoff and exited"); + + const uint32_t topA = rdramRead32(rdram, kSiTopA); + const uint32_t topB = rdramRead32(rdram, kSiTopB); + t.IsTrue(topA != 0u && topB != 0u, "I1: both invokes reserved a scratch stack"); + t.IsTrue(topA != topB, + "I1: interleaved invokes must run on DISJOINT scratch stacks"); + + // Cleanup: erase the global overrides so later suites are unaffected. + { R5900Context s{}; setRegU32(s,4,kSysA); setRegU32(s,5,0u); + ps2_syscalls::SetSyscall(rdram.data(), &s, &runtime); } + { R5900Context s{}; setRegU32(s,4,kSysB); setRegU32(s,5,0u); + ps2_syscalls::SetSyscall(rdram.data(), &s, &runtime); } + deleteSchedSema(rdram.data(), &runtime, sidA); + deleteSchedSema(rdram.data(), &runtime, sidB); + }); + + // I2 --- B3: two fibers interleaving through inline DMAC dispatch must + // run handlers on disjoint stacks. RED on ucontext against the shared + // getAsyncHandlerStackTop cache (dmacTopA == dmacTopB). + tc.Run("I2: inline DMAC handler stack is isolated across interleaved fibers", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + constexpr uint32_t kEntryA = 0x00701400u, kEntryB = 0x00701500u; + constexpr uint32_t kHndA = 0x00701600u, kHndB = 0x00701700u; + + runtime.registerFunction(kEntryA, &stepDmacDispatchA); + runtime.registerFunction(kEntryB, &stepDmacDispatchB); + runtime.registerFunction(kHndA, &stepDmacRecordA); + runtime.registerFunction(kHndB, &stepDmacRecordB); + + int32_t hidA = -1, hidB = -1; + { R5900Context a{}; setRegU32(a,4,5u); setRegU32(a,5,kHndA); setRegU32(a,6,0u); setRegU32(a,7,0u); + ps2_syscalls::AddDmacHandler(rdram.data(), &a, &runtime); hidA = getRegS32(a, 2); } + { R5900Context a{}; setRegU32(a,4,6u); setRegU32(a,5,kHndB); setRegU32(a,6,0u); setRegU32(a,7,0u); + ps2_syscalls::AddDmacHandler(rdram.data(), &a, &runtime); hidB = getRegS32(a, 2); } + t.IsTrue(hidA > 0 && hidB > 0, "I2: DMAC handlers registered"); + + const int32_t sidA = createSchedSema(rdram.data(), &runtime, 0, 1); + const int32_t sidB = createSchedSema(rdram.data(), &runtime, 0, 1); + rdramWrite32(rdram, kSiSemA, static_cast(sidA)); + rdramWrite32(rdram, kSiSemB, static_cast(sidB)); + rdramWrite32(rdram, kSiDmacTopA, 0u); + rdramWrite32(rdram, kSiDmacTopB, 0u); + + const int32_t tidA = startSchedWorker(rdram.data(), &runtime, kEntryA, 10, 0x00548000u, 0x2000u); + const int32_t tidB = startSchedWorker(rdram.data(), &runtime, kEntryB, 10, 0x0054C000u, 0x2000u); + t.IsTrue(tidA > 0 && tidB > 0, "I2: both fibers started"); + + const bool drained = waitUntil([&]{ return g_activeThreads.load(std::memory_order_acquire) <= 0; }, + std::chrono::milliseconds(3000)); + t.IsTrue(drained, "I2: both fibers completed dispatch and exited"); + + const uint32_t topA = rdramRead32(rdram, kSiDmacTopA); + const uint32_t topB = rdramRead32(rdram, kSiDmacTopB); + t.IsTrue(topA != 0u && topB != 0u, "I2: both dispatches reserved a handler stack"); + t.IsTrue(topA != topB, + "I2: interleaved inline DMAC dispatches must run on DISJOINT stacks"); + + // Cleanup: RemoveDmacHandler reads cause=$a0, handlerId=$a1. + { R5900Context r{}; setRegU32(r,4,5u); setRegU32(r,5,static_cast(hidA)); + ps2_syscalls::RemoveDmacHandler(rdram.data(), &r, &runtime); } + { R5900Context r{}; setRegU32(r,4,6u); setRegU32(r,5,static_cast(hidB)); + ps2_syscalls::RemoveDmacHandler(rdram.data(), &r, &runtime); } + deleteSchedSema(rdram.data(), &runtime, sidA); + deleteSchedSema(rdram.data(), &runtime, sidB); + }); + + // I3 --- shared-mechanism contract (stands in for B4). dispatchGuest- + // StreamCallback is anon-namespace-internal and standing up MPEG demux + // state to trigger it from a fiber is disproportionate; instead assert + // the GuestScratchStack contract directly: two LIVE reservations are + // disjoint. B4's site uses this exact helper, and the fiber-level proof + // that it isolates under interleaving is I1/I2. + tc.Run("I3: two live GuestScratchStack reservations are disjoint", [](TestCase &t) + { + PS2Runtime runtime; + std::vector rdram(PS2_RAM_SIZE, 0u); + constexpr uint32_t kSize = 0x4000u; + + GuestScratchStack a(&runtime, kSize); + GuestScratchStack b(&runtime, kSize); // b alive WHILE a is alive + t.IsTrue(a.valid() && b.valid(), "I3: both reservations succeeded"); + t.IsTrue(a.top() != b.top(), "I3: live reservations have distinct tops"); + const uint32_t hi = (a.top() > b.top()) ? a.top() : b.top(); + const uint32_t lo = (a.top() > b.top()) ? b.top() : a.top(); + t.IsTrue(hi - lo >= kSize, "I3: reserved ranges do not overlap"); + // After scope exit both free; a subsequent reservation may recycle + // an address (expected) — which is exactly why I1/I2 keep both live. + + runtime.requestStop(); + }); + }); // MiniTest::Case("SchedulerStackIsolation") +} + +// --------------------------------------------------------------------------- +// B2: dispatchSyscallOverride reentrancy-guard bug -- see the helper +// banner above for the shared-thread_local mechanism and pthread-backend +// caveat. This case proves fiber B's override actually runs while A is +// parked inside its own override. +// --------------------------------------------------------------------------- +void register_scheduler_override_isolation_tests() +{ + MiniTest::Case("SchedulerOverrideIsolation", [](TestCase &tc) + { + tc.Run("O1: a fiber parked inside a syscall override must not suppress another fiber's override for the same syscall", + [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(kOvEntryA, &stepOverrideA); + runtime.registerFunction(kOvEntryB, &stepOverrideB); + runtime.registerFunction(kOvHandler, &stepOverrideHandler); + + // Register the override directly (avoids SetSyscall's guest-memory + // mirroring side effects); State.h is already included. + { + std::lock_guard lk(g_syscall_override_mutex); + g_syscall_overrides[kOvSyscall] = kOvHandler; + } + + const int32_t sidStartB = createSchedSema(rdram.data(), &runtime, 0, 1); + const int32_t sidResumeA = createSchedSema(rdram.data(), &runtime, 0, 1); + t.IsTrue(sidStartB > 0 && sidResumeA > 0, "O1: semas created"); + if (sidStartB <= 0 || sidResumeA <= 0) + { + { std::lock_guard lk(g_syscall_override_mutex); g_syscall_overrides.erase(kOvSyscall); } + return; + } + rdramWrite32(rdram, kOvSidStartB, static_cast(sidStartB)); + rdramWrite32(rdram, kOvSidResumeA, static_cast(sidResumeA)); + rdramWrite32(rdram, kOvAEntered, 0u); + rdramWrite32(rdram, kOvARan, 0u); + rdramWrite32(rdram, kOvBRan, 0u); + + const int32_t tidA = startSchedWorker(rdram.data(), &runtime, + kOvEntryA, 10, 0x00550000u, 0x2000u); + const int32_t tidB = startSchedWorker(rdram.data(), &runtime, + kOvEntryB, 10, 0x00554000u, 0x2000u); + t.IsTrue(tidA > 0 && tidB > 0, "O1: both fibers started"); + if (tidA <= 0 || tidB <= 0) + { + deleteSchedSema(rdram.data(), &runtime, sidStartB); + deleteSchedSema(rdram.data(), &runtime, sidResumeA); + { std::lock_guard lk(g_syscall_override_mutex); g_syscall_overrides.erase(kOvSyscall); } + return; + } + + const bool drained = waitUntil([&]() + { + return g_activeThreads.load(std::memory_order_acquire) <= 0; + }, std::chrono::milliseconds(3000)); + // A timeout here means the PARK mechanism failed (WaitSema nested in + // the override invoke) — NOT the B2 guard. Debug the park, not the fix. + t.IsTrue(drained, "O1: both fibers completed the handoff and exited"); + + // Liveness: prove A really parked inside its override and resumed — + // otherwise kOvBRan could be satisfied vacuously. + t.Equals(rdramRead32(rdram, kOvAEntered), 1u, + "O1: fiber A entered its override handler"); + t.Equals(rdramRead32(rdram, kOvARan), 1u, + "O1: fiber A resumed from its park inside the override"); + + // THE regression assertion. Bug: B's override is skipped as + // 'reentrant' (A's N still on the shared stack) => kOvBRan == 0. + t.Equals(rdramRead32(rdram, kOvBRan), 1u, + "O1: fiber B's own override must run while fiber A is parked inside the same syscall's override"); + + deleteSchedSema(rdram.data(), &runtime, sidStartB); + deleteSchedSema(rdram.data(), &runtime, sidResumeA); + { std::lock_guard lk(g_syscall_override_mutex); g_syscall_overrides.erase(kOvSyscall); } + }); + }); // MiniTest::Case("SchedulerOverrideIsolation") +} + +// --------------------------------------------------------------------------- +// TerminateThread must not hang when the target has an equal-priority sibling +// that stays runnable after the target exits. +// +// join_fiber() applies a temporary priority floor to the joiner so it runs +// strictly AFTER the target. The floor must be the target's OWN level, not +// one below it: PS2 EE priority is "lower number = higher priority", and the +// run queue is FIFO within a level. Flooring the joiner one level below the +// target (t->priority + 1) drops it beneath every equal-priority sibling of +// the target. If any such sibling stays runnable after the target finishes, +// the joiner never regains the run-queue head, never re-observes the target +// as finished, and join_fiber() — hence TerminateThread — never returns. +// +// Shape (all guest fibers, N=1 cooperative executor): +// S1,S2 (prio 10): a one-permit semaphore ping-pong. Exactly one of the two +// is runnable at any instant, so the prio-10 level is +// continuously "hot" for the whole test, including after +// the target exits. +// B (prio 10): the terminate target. Spins until killed by A. +// A (prio 5): calls TerminateThread(B). join_fiber floors A to B's +// level. Fixed (floor = 10): A ties the pingers and, by +// FIFO rotation, cycles back to the head to see B finished +// and returns. Buggy (floor = 11): A sits below the ever- +// runnable pingers forever and TerminateThread hangs. +// +// The assertion is a bounded-time poll on A's "join returned" flag, so the +// buggy build FAILS via timeout instead of hanging the binary. Teardown stops +// the pingers and drains to zero active threads in BOTH outcomes (and the +// SchedFixture destructor's scheduler_shutdown is the final backstop), so no +// runnable fiber is ever leaked into the rest of the suite. +// --------------------------------------------------------------------------- +void register_scheduler_join_starvation_tests() +{ + MiniTest::Case("SchedulerJoinStarvation", [](TestCase &tc) + { + tc.Run("J1: TerminateThread returns when the target has an equal-priority sibling that outlives it", + [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + constexpr uint32_t kEntryPingA = 0x00760000u; + constexpr uint32_t kEntryPingB = 0x00760100u; + constexpr uint32_t kEntryTarget = 0x00760200u; + constexpr uint32_t kEntryJoiner = 0x00760300u; + + runtime.registerFunction(kEntryPingA, &stepPingA); + runtime.registerFunction(kEntryPingB, &stepPingB); + runtime.registerFunction(kEntryTarget, &stepJoinStarveTarget); + runtime.registerFunction(kEntryJoiner, &stepJoinStarveJoiner); + + // One permit circulates X -> Y -> X, keeping the prio-10 level hot. + const int32_t sidX = createSchedSema(rdram.data(), &runtime, 1, 1); + const int32_t sidY = createSchedSema(rdram.data(), &runtime, 0, 1); + t.IsTrue(sidX > 0 && sidY > 0, "J1: ping-pong semas created"); + if (sidX <= 0 || sidY <= 0) + { + deleteSchedSema(rdram.data(), &runtime, sidX); + deleteSchedSema(rdram.data(), &runtime, sidY); + return; + } + rdramWrite32(rdram, kThSidX, static_cast(sidX)); + rdramWrite32(rdram, kThSidY, static_cast(sidY)); + rdramWrite32(rdram, kThStop, 0u); + rdramWrite32(rdram, kThCount, 0u); + rdramWrite32(rdram, kJsBStarted, 0u); + rdramWrite32(rdram, kJsJoinReturned, 0u); + rdramWrite32(rdram, kJsTargetTid, 0u); + + // Pingers first (prio 10), then confirm the level is circulating. + const int32_t tidS1 = startSchedWorker(rdram.data(), &runtime, kEntryPingA, 10, 0x00560000u, 0x2000u); + const int32_t tidS2 = startSchedWorker(rdram.data(), &runtime, kEntryPingB, 10, 0x00562000u, 0x2000u); + t.IsTrue(tidS1 > 0 && tidS2 > 0, "J1: ping-pong pair started"); + + const bool pingLive = waitUntil([&]() + { + return rdramRead32(rdram, kThCount) >= 3u; + }, std::chrono::milliseconds(1000)); + t.IsTrue(pingLive, "J1: prio-10 ping-pong is circulating"); + + // Target B (prio 10) — never self-exits; killed by A's Terminate. + const int32_t tidB = startSchedWorker(rdram.data(), &runtime, kEntryTarget, 10, 0x00564000u, 0x2000u); + t.IsTrue(tidB > 0, "J1: target fiber started"); + const bool bRunning = waitUntil([&]() + { + return rdramRead32(rdram, kJsBStarted) == 1u; + }, std::chrono::milliseconds(1000)); + t.IsTrue(bRunning, "J1: target fiber is running"); + + // Joiner A (prio 5) terminates B; join_fiber floors A to B's level. + rdramWrite32(rdram, kJsTargetTid, static_cast(tidB)); + const int32_t tidA = startSchedWorker(rdram.data(), &runtime, kEntryJoiner, 5, 0x00566000u, 0x2000u); + t.IsTrue(tidA > 0, "J1: joiner fiber started"); + + // THE regression assertion: TerminateThread(B) returns within a + // bounded number of scheduler rounds. Buggy (+1 floor): A is + // stranded below the pingers and this never flips -> timeout -> + // FAIL (binary still exits; teardown below quiesces everything). + const bool joinReturned = waitUntil([&]() + { + return rdramRead32(rdram, kJsJoinReturned) == 1u; + }, std::chrono::milliseconds(3000)); + t.IsTrue(joinReturned, + "J1: TerminateThread(target) returned — join floor must be the " + "target's own priority level, not target+1"); + + // ---- Teardown (runs in BOTH pass and fail outcomes) ---- + // Stop the pingers and release any WaitSema-parked pinger so it + // re-checks kThStop and exits. Once the prio-10 level drains, a + // stranded (buggy-case) joiner also regains the head, observes B + // finished, and returns — so g_activeThreads reaches zero here, + // proving clean quiescence without leaking a runnable fiber. + rdramWrite32(rdram, kThStop, 1u); + for (int i = 0; i < 4; ++i) + { + signalSchedSema(rdram.data(), &runtime, sidX); + signalSchedSema(rdram.data(), &runtime, sidY); + } + const bool drained = waitUntil([&]() + { + return g_activeThreads.load(std::memory_order_acquire) <= 0; + }, std::chrono::milliseconds(3000)); + t.IsTrue(drained, "J1: all fibers quiesced after teardown"); + + deleteSchedSema(rdram.data(), &runtime, sidX); + deleteSchedSema(rdram.data(), &runtime, sidY); + }); + }); // MiniTest::Case("SchedulerJoinStarvation") +} diff --git a/ps2xTest/tsan.supp b/ps2xTest/tsan.supp new file mode 100644 index 000000000..a7ff09755 --- /dev/null +++ b/ps2xTest/tsan.supp @@ -0,0 +1,13 @@ +# ThreadSanitizer suppressions for ps2x_tests. +# +# Usage: +# TSAN_OPTIONS="suppressions=$PWD/ps2xTest/tsan.supp" ./ps2x_tests +# +# The vsync flag/tick words and the GS CSR vsync field model memory-mapped +# PS2 hardware: the interrupt worker stores them and guest/test code polls +# the same words raw, with no interlock — exactly as on real hardware. This +# is a data race under the C++ memory model and is left unsynchronized +# intentionally; see the comment in signalVSyncFlag +# (ps2xRuntime/src/lib/Kernel/Syscalls/Interrupt.cpp) for the full rationale. +race:signalVSyncFlag +race:updateGsCsrFieldForVSync From f4a02c6182fa385c073d8f74ad49095c771846a4 Mon Sep 17 00:00:00 2001 From: Shane Michael Mathews Date: Tue, 7 Jul 2026 18:36:15 -0400 Subject: [PATCH 02/15] =?UTF-8?q?simplify(fiber):=20dedup=20backends=20?= =?UTF-8?q?=E2=80=94=20shared=20GuardedStack=20RAII,=20shared=20TLS/curren?= =?UTF-8?q?t,=20drop=20SceFiber=20tls=5Fpending=5Fself?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ps2xRuntime/src/lib/ps2_fiber.cpp | 246 +++++++++++++++--------------- 1 file changed, 122 insertions(+), 124 deletions(-) diff --git a/ps2xRuntime/src/lib/ps2_fiber.cpp b/ps2xRuntime/src/lib/ps2_fiber.cpp index 03c111e70..cbc38a463 100644 --- a/ps2xRuntime/src/lib/ps2_fiber.cpp +++ b/ps2xRuntime/src/lib/ps2_fiber.cpp @@ -6,6 +6,7 @@ #include #include #include +#include // Abort if a fiber context switch is attempted off the guest executor thread. // Used by the backends (ucontext, SceFiber, Win32 Fibers) where exactly one @@ -23,33 +24,117 @@ static inline void ps2fiber_require_executor_thread(const char* who) } } +// The fiber currently running on this thread, or nullptr. PS2Fiber* is the +// same opaque pointer type across all four backends (only the struct it +// points to differs per backend), so the TLS slot and its accessor are +// defined once here instead of once per backend. +static thread_local PS2Fiber* tls_current_fiber_ptr = nullptr; + +PS2Fiber* ps2fiber_current() +{ + return tls_current_fiber_ptr; +} + +#if !defined(PS2X_FIBER_PTHREAD) +// ucontext, SceFiber, and Win32 Fibers backends: no separate OS thread to +// outlive the PS2Fiber, so the executor relies on FiberContext::state == +// Finished instead of polling this function. (The pthread backend below has +// a real per-fiber OS thread that can already be dead when polled, so it +// defines its own ps2fiber_finished under its branch.) +bool ps2fiber_finished(PS2Fiber* /*f*/) +{ + return false; +} +#endif + +#if !defined(PLATFORM_VITA) && !defined(_WIN32) +// ============================================================================ +// Shared guard-paged stack allocator — ucontext and pthread backends only. +// Both are POSIX mmap/mprotect based; Vita and Win32 use their own +// platform-native stack allocation and are excluded here. +// ============================================================================ +#include +#include // sysconf(_SC_PAGESIZE) + +// Owns an mmap'd stack with a single low guard page (PROT_NONE) below the +// usable region, so a stack overflow faults instead of silently corrupting +// adjacent memory. Move-only; unmaps on destruction. +struct GuardedStack +{ + uint8_t* base = nullptr; // mmap base (guard page first) + size_t total = 0; // total mapping incl. guard page + size_t usable = 0; // usable stack size (page-rounded) + + uint8_t* stack() const { return base + total - usable; } // region above the low guard page + + // Allocate `want` bytes of usable stack (rounded up to a page) plus one + // low guard page below it. Returns false on failure; *out is untouched. + static bool make(size_t want, GuardedStack& out) + { + long pageQuery = sysconf(_SC_PAGESIZE); + const size_t page = (pageQuery > 0) ? static_cast(pageQuery) : 4096u; + size_t usable = (want + page - 1) & ~(page - 1); + size_t total = usable + page; // 1 guard page at low address + + void* base = mmap(nullptr, total, PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + if (base == MAP_FAILED) + { + std::fprintf(stderr, "[ps2fiber] mmap failed\n"); + return false; + } + if (mprotect(base, page, PROT_NONE) != 0) + { + munmap(base, total); + std::fprintf(stderr, "[ps2fiber] mprotect guard failed\n"); + return false; + } + + out.base = static_cast(base); + out.total = total; + out.usable = usable; + return true; + } + + GuardedStack() = default; + GuardedStack(GuardedStack&& o) noexcept { *this = std::move(o); } + GuardedStack& operator=(GuardedStack&& o) noexcept + { + if (this != &o) + { + if (base) munmap(base, total); + base = o.base; total = o.total; usable = o.usable; + o.base = nullptr; o.total = 0; o.usable = 0; + } + return *this; + } + GuardedStack(const GuardedStack&) = delete; + GuardedStack& operator=(const GuardedStack&) = delete; + ~GuardedStack() { if (base) munmap(base, total); } +}; +#endif // !PLATFORM_VITA && !_WIN32 + #if !defined(PLATFORM_VITA) && !defined(PS2X_FIBER_PTHREAD) && !defined(_WIN32) // ============================================================================ // POSIX path — ucontext_t // ============================================================================ #include -#include -#include // sysconf(_SC_PAGESIZE) #include #include static_assert(sizeof(void*) == 8, "ucontext pointer split requires exactly 64-bit pointers (LP64)"); static_assert(sizeof(unsigned int) == 4, "makecontext int args must be 32 bits"); -// Per-guest-executor-thread state. These are thread_local but in practice -// only ever touched by the single g_guest_thread. +// Per-guest-executor-thread state. This is thread_local but in practice only +// ever touched by the single g_guest_thread. static thread_local ucontext_t tls_guest_main_ctx; -static thread_local PS2Fiber* tls_current_fiber_ptr = nullptr; struct PS2Fiber { - ucontext_t ctx; - uint8_t* map_base = nullptr; // mmap base (guard page first) - size_t map_size = 0; // total mapping incl. guard page - uint8_t* stack = nullptr; // usable stack region (after guard page) - size_t stackSize = 0; - void (*fn)(void*) = nullptr; - void* arg = nullptr; + ucontext_t ctx; + GuardedStack stack; + void (*fn)(void*) = nullptr; + void* arg = nullptr; }; // makecontext can only pass int-sized arguments portably. Split the 64-bit @@ -70,48 +155,26 @@ static void ps2fiber_trampoline(unsigned int self_hi, unsigned int self_lo) PS2Fiber* ps2fiber_alloc(void (*fn)(void*), void* arg, size_t stack_bytes) { if (stack_bytes == 0) { std::fprintf(stderr, "ps2fiber_alloc: zero stack size\n"); return nullptr; } - long pageQuery = sysconf(_SC_PAGESIZE); - const size_t page = (pageQuery > 0) ? static_cast(pageQuery) : 4096u; - size_t usable = (stack_bytes + page - 1) & ~(page - 1); - size_t total = usable + page; // 1 guard page at low address - - void* base = mmap(nullptr, total, PROT_READ | PROT_WRITE, - MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); - if (base == MAP_FAILED) - { - std::fprintf(stderr, "[ps2fiber] mmap failed\n"); - return nullptr; - } - if (mprotect(base, page, PROT_NONE) != 0) - { - munmap(base, total); - std::fprintf(stderr, "[ps2fiber] mprotect guard failed\n"); - return nullptr; - } PS2Fiber* f = new (std::nothrow) PS2Fiber(); - if (!f) + if (!f) return nullptr; + + if (!GuardedStack::make(stack_bytes, f->stack)) { - munmap(base, total); + delete f; return nullptr; } - - f->map_base = static_cast(base); - f->map_size = total; - f->stack = static_cast(base) + page; - f->stackSize = usable; f->fn = fn; f->arg = arg; if (getcontext(&f->ctx) != 0) { std::fprintf(stderr, "[ps2fiber] getcontext failed: %s\n", std::strerror(errno)); - munmap(base, total); - delete f; + delete f; // GuardedStack destructor unmaps the stack return nullptr; } - f->ctx.uc_stack.ss_sp = f->stack; - f->ctx.uc_stack.ss_size = f->stackSize; + f->ctx.uc_stack.ss_sp = f->stack.stack(); + f->ctx.uc_stack.ss_size = f->stack.usable; f->ctx.uc_link = nullptr; // trampoline never returns via link uintptr_t raw = reinterpret_cast(f); @@ -124,8 +187,7 @@ PS2Fiber* ps2fiber_alloc(void (*fn)(void*), void* arg, size_t stack_bytes) void ps2fiber_free(PS2Fiber* f) { if (!f) return; - if (f->map_base) munmap(f->map_base, f->map_size); - delete f; + delete f; // GuardedStack destructor unmaps the stack } void ps2fiber_resume(PS2Fiber* f) @@ -163,18 +225,6 @@ void ps2fiber_yield() tls_current_fiber_ptr = self; } -PS2Fiber* ps2fiber_current() -{ - return tls_current_fiber_ptr; -} - -bool ps2fiber_finished(PS2Fiber* /*f*/) -{ - // ucontext backend: no separate OS thread to outlive the PS2Fiber. The - // executor relies on FiberContext::state == Finished instead. - return false; -} - #elif defined(PLATFORM_VITA) && !defined(PS2X_FIBER_PTHREAD) // ============================================================================ // SceFiber backend — Vita (PLATFORM_VITA and not PS2X_FIBER_PTHREAD) @@ -183,11 +233,6 @@ bool ps2fiber_finished(PS2Fiber* /*f*/) #include // sceKernelAllocMemBlock/GetMemBlockBase/FreeMemBlock #include "ps2_scheduler.h" // extern thread_local int g_currentThreadId -static thread_local PS2Fiber* tls_current_fiber_ptr = nullptr; -// Set by ps2fiber_resume immediately before sceFiberRun so the fiber entry -// can recover its PS2Fiber* without relying on the 32-bit argOnRunTo alone. -static thread_local PS2Fiber* tls_pending_self = nullptr; - struct PS2Fiber { SceFiber fiber; // SDK control block (value, not pointer) @@ -201,8 +246,9 @@ struct PS2Fiber static void ps2fiber_entry(SceUInt32 /*argOnInitialize*/, SceUInt32 /*argOnRunTo*/) { - PS2Fiber* self = tls_pending_self; - tls_current_fiber_ptr = self; + // ps2fiber_resume already sets tls_current_fiber_ptr = f immediately + // before sceFiberRun on this same thread, so it's already set here. + PS2Fiber* self = tls_current_fiber_ptr; g_currentThreadId = self->tid; self->fn(self->arg); // fn returned — hand control back to the thread. The entry must not return. @@ -275,7 +321,6 @@ void ps2fiber_set_tid(PS2Fiber* f, int tid) void ps2fiber_resume(PS2Fiber* f) { ps2fiber_require_executor_thread("ps2fiber_resume"); - tls_pending_self = f; PS2Fiber* prev = tls_current_fiber_ptr; tls_current_fiber_ptr = f; SceInt32 rc = sceFiberRun(&f->fiber, 0, nullptr); @@ -302,18 +347,6 @@ void ps2fiber_yield() tls_current_fiber_ptr = self; } -PS2Fiber* ps2fiber_current() -{ - return tls_current_fiber_ptr; -} - -bool ps2fiber_finished(PS2Fiber* /*f*/) -{ - // SceFiber backend: matches ucontext semantics — the scheduler relies on - // FiberContext::state == Finished rather than polling this function. - return false; -} - void ps2fiber_free(PS2Fiber* f) { if (!f) return; @@ -338,12 +371,8 @@ void ps2fiber_free(PS2Fiber* f) #endif #include #include -#include -#include #include "ps2_scheduler.h" // extern thread_local int g_currentThreadId -static thread_local PS2Fiber* tls_current_fiber_ptr = nullptr; - struct PS2Fiber { pthread_t thread; @@ -355,8 +384,7 @@ struct PS2Fiber std::atomic finished{false}; // fn returned; written on the fiber thread and read on the executor, so must be atomic std::atomic ever_resumed{false}; // set by ps2fiber_resume; distinguishes "never resumed" (free is legal) from "parked mid-run" (free is a caller bug) std::atomic abandoned{false}; // set by ps2fiber_free when freeing a never-resumed fiber; tells the worker to exit without running fn - uint8_t* map_base = nullptr; // mmap base (guard page first); freed after pthread_join - size_t map_size = 0; // total mapping incl. guard page + GuardedStack stack; // guard-paged stack backing `thread`; unmapped after pthread_join // Guest thread id for this fiber. Set by the scheduler via // ps2fiber_set_tid() after alloc; published onto the fiber's own pthread in // ps2fiber_thread_main so blocking syscalls see the right g_currentThreadId. @@ -399,25 +427,12 @@ PS2Fiber* ps2fiber_alloc(void (*fn)(void*), void* arg, size_t stack_bytes) // Allocate our own stack with an explicit guard page at the low address so // a stack overflow faults (SIGSEGV) rather than silently corrupting memory. - long pg_long = sysconf(_SC_PAGESIZE); - size_t page = (pg_long > 0) ? static_cast(pg_long) : 4096u; - size_t usable = stack_bytes ? stack_bytes : (512 * 1024); // smaller on Vita - size_t total = usable + page; - void* base = mmap(nullptr, total, PROT_READ | PROT_WRITE, - MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); - if (base == MAP_FAILED) + size_t want = stack_bytes ? stack_bytes : (512 * 1024); // smaller on Vita + if (!GuardedStack::make(want, f->stack)) { delete f; return nullptr; } - if (mprotect(base, page, PROT_NONE) != 0) - { - munmap(base, total); - delete f; - return nullptr; - } - f->map_base = static_cast(base); - f->map_size = total; sem_init(&f->resume_sem, 0, 0); sem_init(&f->yield_sem, 0, 0); @@ -425,18 +440,20 @@ PS2Fiber* ps2fiber_alloc(void (*fn)(void*), void* arg, size_t stack_bytes) pthread_attr_t attr; pthread_attr_init(&attr); // Belt-and-suspenders: ask pthread to set its own guard too (may be a no-op - // on some platforms, but the mmap guard above is the authoritative one). + // on some platforms, but the mmap guard in GuardedStack::make is the + // authoritative one). + long pg_long = sysconf(_SC_PAGESIZE); + size_t page = (pg_long > 0) ? static_cast(pg_long) : 4096u; pthread_attr_setguardsize(&attr, page); // Hand our pre-allocated, guard-paged stack to pthread. - pthread_attr_setstack(&attr, static_cast(base) + page, usable); + pthread_attr_setstack(&attr, f->stack.stack(), f->stack.usable); int rc = pthread_create(&f->thread, &attr, ps2fiber_thread_main, f); pthread_attr_destroy(&attr); if (rc != 0) { sem_destroy(&f->resume_sem); sem_destroy(&f->yield_sem); - munmap(base, total); - delete f; + delete f; // GuardedStack destructor unmaps the stack return nullptr; } f->started = true; @@ -463,11 +480,6 @@ void ps2fiber_yield() sem_wait(&self->resume_sem); // wait for next resume } -PS2Fiber* ps2fiber_current() -{ - return tls_current_fiber_ptr; -} - bool ps2fiber_finished(PS2Fiber* f) { // ps2fiber_thread_main sets finished=true just before the pthread exits. @@ -499,10 +511,10 @@ void ps2fiber_free(PS2Fiber* f) } } if (f->started) pthread_join(f->thread, nullptr); // joinable: no leak - // Unmap our guard-paged stack only after the thread is fully joined (exited). - if (f->map_base) munmap(f->map_base, f->map_size); sem_destroy(&f->resume_sem); sem_destroy(&f->yield_sem); + // GuardedStack destructor (in ~PS2Fiber via delete below) unmaps the + // guard-paged stack — only now that the thread is fully joined (exited). delete f; } @@ -518,10 +530,9 @@ void ps2fiber_free(PS2Fiber* f) #endif #include -// Per-guest-executor-thread state. These are thread_local but in practice -// only ever touched by the single g_guest_thread. -static thread_local LPVOID tls_guest_main_fiber = nullptr; // executor's own fiber (the "main context") -static thread_local PS2Fiber* tls_current_fiber_ptr = nullptr; +// Per-guest-executor-thread state. This is thread_local but in practice only +// ever touched by the single g_guest_thread. +static thread_local LPVOID tls_guest_main_fiber = nullptr; // executor's own fiber (the "main context") struct PS2Fiber { @@ -657,19 +668,6 @@ void ps2fiber_yield() tls_current_fiber_ptr = self; } -PS2Fiber* ps2fiber_current() -{ - return tls_current_fiber_ptr; -} - -bool ps2fiber_finished(PS2Fiber* /*f*/) -{ - // Win32 Fibers backend: matches ucontext semantics — no separate OS thread - // to outlive the PS2Fiber, so the executor relies on - // FiberContext::state == Finished instead of polling this function. - return false; -} - #else #error "Unknown fiber backend: define PLATFORM_VITA (SceFiber), PS2X_FIBER_PTHREAD (pthread), build for _WIN32 (Win32 Fibers), or use the default POSIX ucontext backend" #endif // fiber backend dispatch From e9e063a24818f87d380002c3e36ec70cf68859df Mon Sep 17 00:00:00 2001 From: Shane Michael Mathews Date: Tue, 7 Jul 2026 18:42:40 -0400 Subject: [PATCH 03/15] simplify(thread): activeCounted mint/consume pair, resolveSelfOrThread prologue, self-exit mark helper --- .../src/lib/Kernel/Syscalls/Helpers/Runtime.h | 39 +++++ .../src/lib/Kernel/Syscalls/Helpers/State.h | 19 +++ .../src/lib/Kernel/Syscalls/Lifecycle.cpp | 5 +- .../src/lib/Kernel/Syscalls/Thread.cpp | 152 ++++-------------- 4 files changed, 86 insertions(+), 129 deletions(-) diff --git a/ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/Runtime.h b/ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/Runtime.h index 50c92e7f3..d94a33c35 100644 --- a/ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/Runtime.h +++ b/ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/Runtime.h @@ -152,6 +152,45 @@ static std::shared_ptr ensureCurrentThreadInfo(R5900Context *ctx) return info; } +// Resolves tid==0 (TH_SELF) to g_currentThreadId, looks up the corresponding +// ThreadInfo, and writes the appropriate error return on failure. Callers +// pass tid by reference so the resolved (non-zero) id is visible afterward. +// Shared prologue for syscalls that operate on "self or an explicit tid": +// TerminateThread, SuspendThread, ResumeThread, ReferThreadStatus, +// CancelWakeupThread, ChangeThreadPriority. Do NOT use for syscalls with +// different tid==0 semantics (WakeupThread, ReleaseWaitThread, the +// i-prefixed variants, RotateThreadReadyQueue). +static std::shared_ptr resolveSelfOrThread(R5900Context *ctx, int &tid) +{ + if (tid == 0) + { + if (g_currentThreadId == -1) + { + setReturnS32(ctx, KE_ILLEGAL_THID); + return nullptr; + } + tid = g_currentThreadId; + } + auto info = (tid == g_currentThreadId) ? ensureCurrentThreadInfo(ctx) : lookupThreadInfo(tid); + if (!info) + setReturnS32(ctx, KE_UNKNOWN_THID); + return info; +} + +// Marks a thread as exiting itself (caller holds info.m). Shared by +// ExitThread and ExitDeleteThread only. TerminateThread must NOT route +// through this: it intentionally sets only terminated/forceRelease and +// leaves waitType/waitId/wakeupCount observable via ReferThreadStatus until +// the target's own wait loop clears them. +static void markSelfExitingLocked(ThreadInfo &info) +{ + info.terminated = true; + info.forceRelease = true; + info.waitType = TSW_NONE; + info.waitId = 0; + info.wakeupCount = 0; +} + static std::shared_ptr lookupSemaInfo(int sid) { std::lock_guard lock(g_sema_map_mutex); diff --git a/ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/State.h b/ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/State.h index fb7a310d0..3b9da6518 100644 --- a/ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/State.h +++ b/ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/State.h @@ -240,6 +240,25 @@ inline std::atomic g_alarm_stop_flag{false}; // Protected by g_alarm_mutex. inline bool g_alarm_worker_running{false}; inline std::atomic g_activeThreads{0}; + +// Mint/consume pair for ThreadInfo::activeCounted (see the field comment +// above). MINT bumps g_activeThreads FIRST, then publishes the token with +// release, so a consumer observing activeCounted==true is guaranteed to also +// see the matching +1. CONSUME is the sole arbiter for a given token: the +// exchange(false) only the winner of the true->false transition performs the +// matching fetch_sub, so concurrent consumers (e.g. on_fiber_exit racing +// notifyRuntimeStop over the same shared_ptr) can never double-decrement. +static inline void mintActiveToken(const std::shared_ptr &info) +{ + g_activeThreads.fetch_add(1, std::memory_order_relaxed); + info->activeCounted.store(true, std::memory_order_release); +} +static inline void consumeActiveToken(const std::shared_ptr &info) +{ + if (info && info->activeCounted.exchange(false, std::memory_order_acq_rel)) + g_activeThreads.fetch_sub(1, std::memory_order_release); +} + inline std::mutex g_fd_mutex; struct RpcServerState diff --git a/ps2xRuntime/src/lib/Kernel/Syscalls/Lifecycle.cpp b/ps2xRuntime/src/lib/Kernel/Syscalls/Lifecycle.cpp index f2726bede..24e0a5ab7 100644 --- a/ps2xRuntime/src/lib/Kernel/Syscalls/Lifecycle.cpp +++ b/ps2xRuntime/src/lib/Kernel/Syscalls/Lifecycle.cpp @@ -67,10 +67,7 @@ namespace ps2_syscalls // cleared above) and skips its own decrement. for (const auto &[tid, threadInfo] : threads) { - if (threadInfo->activeCounted.exchange(false, std::memory_order_acq_rel)) - { - g_activeThreads.fetch_sub(1, std::memory_order_release); - } + consumeActiveToken(threadInfo); } // -1 is the "not a guest fiber" sentinel: a host thread running // notifyRuntimeStop must never be mistaken for a real guest thread id diff --git a/ps2xRuntime/src/lib/Kernel/Syscalls/Thread.cpp b/ps2xRuntime/src/lib/Kernel/Syscalls/Thread.cpp index 7940401bc..01e3a78a2 100644 --- a/ps2xRuntime/src/lib/Kernel/Syscalls/Thread.cpp +++ b/ps2xRuntime/src/lib/Kernel/Syscalls/Thread.cpp @@ -94,12 +94,11 @@ namespace ps2_syscalls // null the g_threads entry was already removed by whoever also took the // token (notifyRuntimeStop reaping residual guest threads, or // ExitDeleteThread erasing its own entry), so this exit must NOT - // decrement again. When `info` is present the atomic exchange is the - // sole arbiter: a concurrent notifyRuntimeStop() holding the same - // shared_ptr races here and exactly one side wins the true->false - // transition and performs the single fetch_sub. - if (info && info->activeCounted.exchange(false, std::memory_order_acq_rel)) - g_activeThreads.fetch_sub(1, std::memory_order_release); + // decrement again. When `info` is present consumeActiveToken's atomic + // exchange is the sole arbiter: a concurrent notifyRuntimeStop() holding + // the same shared_ptr races here and exactly one side wins the + // true->false transition and performs the single fetch_sub. + consumeActiveToken(info); } static std::once_flag s_fiber_exit_hook_once; @@ -387,13 +386,9 @@ namespace ps2_syscalls } ensureFiberExitHookRegistered(); - // Mint this thread's active-thread token. Order matters: bump the - // counter FIRST, then publish the token with release. A consumer that - // observes activeCounted==true is therefore guaranteed to also see the - // matching +1, so the exchange/fetch_sub in a consumer can never run - // ahead of this fetch_add and transiently drive g_activeThreads negative. - g_activeThreads.fetch_add(1, std::memory_order_relaxed); - info->activeCounted.store(true, std::memory_order_release); + // Mint this thread's active-thread token (see mintActiveToken for the + // memory-ordering rationale). + mintActiveToken(info); // Create the fiber and enqueue it Ready. Throws on allocation failure or if the scheduler is shutting down; the catch below reports it as KE_NO_MEMORY. try @@ -406,10 +401,10 @@ namespace ps2_syscalls catch (const std::exception& e) { // Undo the g_activeThreads increment and reset thread state. Consume - // the token we just minted; the exchange guards against a concurrent - // notifyRuntimeStop() that reaped this same ThreadInfo. - if (info->activeCounted.exchange(false, std::memory_order_acq_rel)) - g_activeThreads.fetch_sub(1, std::memory_order_release); + // the token we just minted; consumeActiveToken's exchange guards + // against a concurrent notifyRuntimeStop() that reaped this same + // ThreadInfo. + consumeActiveToken(info); { std::lock_guard lock(info->m); info->started = false; @@ -442,11 +437,7 @@ namespace ps2_syscalls if (info) { std::lock_guard lock(info->m); - info->terminated = true; - info->forceRelease = true; - info->waitType = TSW_NONE; - info->waitId = 0; - info->wakeupCount = 0; + markSelfExitingLocked(*info); } throw ThreadExitException(); } @@ -459,11 +450,7 @@ namespace ps2_syscalls if (info) { std::lock_guard lock(info->m); - info->terminated = true; - info->forceRelease = true; - info->waitType = TSW_NONE; - info->waitId = 0; - info->wakeupCount = 0; + markSelfExitingLocked(*info); } { std::lock_guard lock(g_thread_map_mutex); @@ -473,30 +460,15 @@ namespace ps2_syscalls // after this throw will see a null ThreadInfo and skip the token consume. // Consume the active-thread token here instead, so g_activeThreads is // decremented exactly once for this started thread. - if (info && info->activeCounted.exchange(false, std::memory_order_acq_rel)) - g_activeThreads.fetch_sub(1, std::memory_order_release); + consumeActiveToken(info); throw ThreadExitException(); } void TerminateThread(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) { int tid = static_cast(getRegU32(ctx, 4)); - if (tid == 0) - { - if (g_currentThreadId == -1) - { - setReturnS32(ctx, KE_ILLEGAL_THID); - return; - } - tid = g_currentThreadId; - } - - auto info = (tid == g_currentThreadId) ? ensureCurrentThreadInfo(ctx) : lookupThreadInfo(tid); - if (!info) - { - setReturnS32(ctx, KE_UNKNOWN_THID); - return; - } + auto info = resolveSelfOrThread(ctx, tid); + if (!info) return; { std::lock_guard lock(info->m); @@ -526,22 +498,8 @@ namespace ps2_syscalls void SuspendThread(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) { int tid = static_cast(getRegU32(ctx, 4)); - if (tid == 0) - { - if (g_currentThreadId == -1) - { - setReturnS32(ctx, KE_ILLEGAL_THID); - return; - } - tid = g_currentThreadId; - } - - auto info = (tid == g_currentThreadId) ? ensureCurrentThreadInfo(ctx) : lookupThreadInfo(tid); - if (!info) - { - setReturnS32(ctx, KE_UNKNOWN_THID); - return; - } + auto info = resolveSelfOrThread(ctx, tid); + if (!info) return; { std::lock_guard lock(info->m); @@ -581,22 +539,8 @@ namespace ps2_syscalls void ResumeThread(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) { int tid = static_cast(getRegU32(ctx, 4)); - if (tid == 0) - { - if (g_currentThreadId == -1) - { - setReturnS32(ctx, KE_ILLEGAL_THID); - return; - } - tid = g_currentThreadId; - } - - auto info = (tid == g_currentThreadId) ? ensureCurrentThreadInfo(ctx) : lookupThreadInfo(tid); - if (!info) - { - setReturnS32(ctx, KE_UNKNOWN_THID); - return; - } + auto info = resolveSelfOrThread(ctx, tid); + if (!info) return; { std::lock_guard lock(info->m); @@ -653,22 +597,8 @@ namespace ps2_syscalls int tid = static_cast(getRegU32(ctx, 4)); uint32_t statusAddr = getRegU32(ctx, 5); - if (tid == 0) // TH_SELF - { - if (g_currentThreadId == -1) - { - setReturnS32(ctx, KE_ILLEGAL_THID); - return; - } - tid = g_currentThreadId; - } - - auto info = (tid == g_currentThreadId) ? ensureCurrentThreadInfo(ctx) : lookupThreadInfo(tid); - if (!info) - { - setReturnS32(ctx, KE_UNKNOWN_THID); - return; - } + auto info = resolveSelfOrThread(ctx, tid); + if (!info) return; ee_thread_status_t *status = reinterpret_cast(getMemPtr(rdram, statusAddr)); if (!status) @@ -884,22 +814,8 @@ namespace ps2_syscalls void CancelWakeupThread(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) { int tid = static_cast(getRegU32(ctx, 4)); - if (tid == 0) - { - if (g_currentThreadId == -1) - { - setReturnS32(ctx, KE_ILLEGAL_THID); - return; - } - tid = g_currentThreadId; - } - - auto info = (tid == g_currentThreadId) ? ensureCurrentThreadInfo(ctx) : lookupThreadInfo(tid); - if (!info) - { - setReturnS32(ctx, KE_UNKNOWN_THID); - return; - } + auto info = resolveSelfOrThread(ctx, tid); + if (!info) return; int previous = 0; { @@ -940,22 +856,8 @@ namespace ps2_syscalls int tid = static_cast(getRegU32(ctx, 4)); int newPrio = static_cast(getRegU32(ctx, 5)); - if (tid == 0) - { - if (g_currentThreadId == -1) - { - setReturnS32(ctx, KE_ILLEGAL_THID); - return; - } - tid = g_currentThreadId; - } - - auto info = (tid == g_currentThreadId) ? ensureCurrentThreadInfo(ctx) : lookupThreadInfo(tid); - if (!info) - { - setReturnS32(ctx, KE_UNKNOWN_THID); - return; - } + auto info = resolveSelfOrThread(ctx, tid); + if (!info) return; { std::lock_guard lock(info->m); From e010ebeec6bf2c166446529352d1bdc17c8ad325 Mon Sep 17 00:00:00 2001 From: Shane Michael Mathews Date: Tue, 7 Jul 2026 18:48:24 -0400 Subject: [PATCH 04/15] simplify(runtime): encapsulate async-callback stack pool as a KernelStackPool allocator --- ps2xRuntime/include/ps2_runtime.h | 24 +++++++++-- ps2xRuntime/src/lib/ps2_runtime.cpp | 67 +++++++++++++++++------------ 2 files changed, 60 insertions(+), 31 deletions(-) diff --git a/ps2xRuntime/include/ps2_runtime.h b/ps2xRuntime/include/ps2_runtime.h index 7d5c9ef3e..3c09cc451 100644 --- a/ps2xRuntime/include/ps2_runtime.h +++ b/ps2xRuntime/include/ps2_runtime.h @@ -342,6 +342,25 @@ constexpr uint32_t kAsyncCallbackStackTop = 0x00100000u; // running a handler there would corrupt live guest frames. constexpr uint32_t kAsyncCallbackFallbackSp = kAsyncCallbackStackTop - 0x10u; +// Encapsulates the async-callback stack pool's carve state. The floor is a +// compile-time constant (nothing ever moves it); `top` is the only runtime +// state — the downward-carving cursor — and is what loadELF() re-arms on +// each load via reset(). See the pool layout comment in ps2_runtime.cpp for +// the disjointness rationale. +struct KernelStackPool +{ + static constexpr uint32_t floor = kAsyncCallbackStackFloor; + uint32_t top = kAsyncCallbackStackTop; + + void reset() { top = kAsyncCallbackStackTop; } + + // Carves an aligned [base, top) region of `size` bytes off the top of the + // pool and returns the guest $sp (top - 0x10) for it, or 0 if the pool is + // exhausted. `size` must already be alignGuestHeapValue()-rounded by the + // caller; `align` is normalized here via PS2Runtime::normalizeGuestHeapAlignment. + uint32_t carve(uint32_t size, uint32_t align); +}; + class PS2Runtime { public: @@ -574,9 +593,8 @@ class PS2Runtime // Async callback stack pool [floor, top): kernel-reserved guest memory, // carved downward. See the pool layout comment in ps2_runtime.cpp (near // kAsyncCallbackStackFloor's namespace-level block) for the full - // disjointness rationale. loadELF() re-arms these on each load. - uint32_t m_asyncCallbackStackFloor = kAsyncCallbackStackFloor; - uint32_t m_asyncCallbackStackTop = kAsyncCallbackStackTop; + // disjointness rationale. loadELF() re-arms this on each load. + KernelStackPool m_asyncCallbackStack; std::atomic m_missingFunctionPolicy{static_cast(MissingFunctionPolicy::ContinueToTarget)}; std::atomic m_missingFunctionReported{false}; diff --git a/ps2xRuntime/src/lib/ps2_runtime.cpp b/ps2xRuntime/src/lib/ps2_runtime.cpp index 5e139a31b..5c4a57f18 100644 --- a/ps2xRuntime/src/lib/ps2_runtime.cpp +++ b/ps2xRuntime/src/lib/ps2_runtime.cpp @@ -473,8 +473,8 @@ PS2Runtime::PS2Runtime() m_guestHeapLimit = std::min(kGuestHeapHardLimit, PS2_RAM_SIZE); m_guestHeapSuggestedBase = kGuestHeapDefaultBase; m_guestHeapConfigured = false; - // m_asyncCallbackStackFloor/Top keep their kernel-pool default member - // initializers here; loadELF() re-arms them for the pool's next load + // m_asyncCallbackStack (KernelStackPool) keeps its kernel-pool default + // member initializer here; loadELF() re-arms it for the pool's next load // (see the layout comment at kAsyncCallbackStackFloor). // Claim the reserved main-thread identity (tid 1 — see State.h's @@ -849,8 +849,7 @@ bool PS2Runtime::loadELF(const std::string &elfPath) std::lock_guard lock(m_asyncCallbackStackMutex); // Re-arm the kernel pool for this load (see the layout comment at // kAsyncCallbackStackFloor); a prior run may have carved it down. - m_asyncCallbackStackFloor = kAsyncCallbackStackFloor; - m_asyncCallbackStackTop = kAsyncCallbackStackTop; + m_asyncCallbackStack.reset(); } LoadedModule module; @@ -1792,48 +1791,60 @@ uint32_t PS2Runtime::guestHeapLimit() const return m_guestHeapConfigured ? m_guestHeapLimit : m_guestHeapSuggestedBase; } -uint32_t PS2Runtime::reserveAsyncCallbackStack(uint32_t size, uint32_t alignment) +// Carve arithmetic lifted verbatim out of reserveAsyncCallbackStack(). Two +// clamps present in the pre-encapsulation version were confirmed dead and +// dropped here rather than ported: +// - `if (top > PS2_RAM_SIZE) top = PS2_RAM_SIZE;` — top starts at +// kAsyncCallbackStackTop (0x00100000) and every successful carve strictly +// decreases it (base < top is required to succeed), so top can never +// approach PS2_RAM_SIZE (0x02000000); the branch never fired. +// - the per-call `top &= ~(kGuestHeapDefaultAlignment - 1u)` re-align — +// top starts 16-aligned (0x00100000) and every stored top is a `base` +// that was itself masked to `align`, which normalizeGuestHeapAlignment() +// guarantees is a power of two >= kGuestHeapDefaultAlignment (16); a +// value aligned to a multiple of 16 is already 16-aligned, so the +// re-align was always a no-op. +uint32_t KernelStackPool::carve(uint32_t size, uint32_t align) { - if (size == 0u) + if (top <= size) { return 0u; } - const uint32_t normalizedAlignment = normalizeGuestHeapAlignment(alignment); - const uint32_t allocSize = alignGuestHeapValue(size, kGuestHeapDefaultAlignment); - if (allocSize == 0u) + uint32_t base = top - size; + base &= ~(align - 1u); + if (base < floor || base >= top) { return 0u; } - std::lock_guard lock(m_asyncCallbackStackMutex); - uint32_t top = m_asyncCallbackStackTop; - if (top > PS2_RAM_SIZE) - { - top = PS2_RAM_SIZE; - } - top &= ~(kGuestHeapDefaultAlignment - 1u); + const uint32_t reservedTop = top; + top = base; + // One line per reservation (a handful per boot): permanent evidence of + // where host-dispatched callback stacks live, so any future overlap with + // guest memory is visible in the boot log. + std::cerr << "[async-stack] reserved [0x" << std::hex << base + << ", 0x" << reservedTop << ") stackTop=0x" << (reservedTop - 0x10u) + << std::dec << '\n'; + return reservedTop - 0x10u; +} - if (top <= allocSize) +uint32_t PS2Runtime::reserveAsyncCallbackStack(uint32_t size, uint32_t alignment) +{ + if (size == 0u) { return 0u; } - uint32_t base = top - allocSize; - base &= ~(normalizedAlignment - 1u); - if (base < m_asyncCallbackStackFloor || base >= top) + const uint32_t normalizedAlignment = normalizeGuestHeapAlignment(alignment); + const uint32_t allocSize = alignGuestHeapValue(size, kGuestHeapDefaultAlignment); + if (allocSize == 0u) { return 0u; } - m_asyncCallbackStackTop = base; - // One line per reservation (a handful per boot): permanent evidence of - // where host-dispatched callback stacks live, so any future overlap with - // guest memory is visible in the boot log. - std::cerr << "[async-stack] reserved [0x" << std::hex << base - << ", 0x" << top << ") stackTop=0x" << (top - 0x10u) - << std::dec << '\n'; - return top - 0x10u; + std::lock_guard lock(m_asyncCallbackStackMutex); + return m_asyncCallbackStack.carve(allocSize, normalizedAlignment); } void PS2Runtime::dispatchLoop(uint8_t *rdram, R5900Context *ctx) From 0c06675beca0ad3ac5e5431e37d169ca521f1455 Mon Sep 17 00:00:00 2001 From: Shane Michael Mathews Date: Tue, 7 Jul 2026 18:52:30 -0400 Subject: [PATCH 05/15] simplify(scheduler): reuse maybe_yield in yield_point; drop redundant lock in block_current borrowed-worker path --- ps2xRuntime/src/lib/ps2_scheduler.cpp | 28 +++++++++++---------------- 1 file changed, 11 insertions(+), 17 deletions(-) diff --git a/ps2xRuntime/src/lib/ps2_scheduler.cpp b/ps2xRuntime/src/lib/ps2_scheduler.cpp index 6bcb715c6..41ecc24b2 100644 --- a/ps2xRuntime/src/lib/ps2_scheduler.cpp +++ b/ps2xRuntime/src/lib/ps2_scheduler.cpp @@ -794,13 +794,13 @@ ps2sched::BlockResult ps2sched::block_current() // AsyncGuestScope) called a blocking syscall. We cannot park a host // worker on the fiber scheduler. Report whether this worker actually holds // the guest token so the caller only drops/reacquires a token it owns. - bool ownsToken; - { - std::lock_guard lk(g_sched_mutex); - ownsToken = g_guest_token_held_by_host && tls_holds_guest_token; - } - return ownsToken ? BlockResult::NonFiberOwner - : BlockResult::NonFiberNoTok; + // tls_holds_guest_token and g_guest_token_held_by_host are set/cleared + // together under g_sched_mutex in async_guest_begin/async_guest_end, so + // on THIS thread tls_holds_guest_token==true already implies + // g_guest_token_held_by_host==true. No need to re-read the global or + // take the lock. + return tls_holds_guest_token ? BlockResult::NonFiberOwner + : BlockResult::NonFiberNoTok; } bool throwTerminate = false; bool wokenInWindow = false; @@ -1113,16 +1113,10 @@ bool ps2sched::yield_point() } // 3. Higher-priority fiber ready -> cooperative yield (enqueue Ready first). - bool yield = false; - { - std::lock_guard lk(g_sched_mutex); - if (g_run_queue && g_run_queue->priority < fc->priority) - { - enqueue_locked(fc); - yield = true; - } - } - if (yield) ps2fiber_yield(); + // Same logic as the standalone maybe_yield(): fc here is the same current + // fiber maybe_yield() would reload from tls_current_fiber, and it is + // already known non-null above, so behavior is identical. + maybe_yield(); // 4. A host worker is parked in async_guest_begin() waiting for the guest // token -> cooperatively yield so it can run. The executor-side half of From 78a2954137a6d37186b2741e376ba1605b6d5ca9 Mon Sep 17 00:00:00 2001 From: Shane Michael Mathews Date: Tue, 7 Jul 2026 18:57:59 -0400 Subject: [PATCH 06/15] simplify(sched): NonFiberBackoff::wait() primitive, single-source token-drop, fix log-once backoff warning - Extract withGuestTokenDropped() as the single place that knows the async_guest_end/begin bracketing for a non-fiber wait pause; both NonFiberBackoff::step's sleep and nonFiberBlockBackoff's yield now route through it instead of duplicating the drop/reacquire branch. - Fix a real bug in NonFiberBackoff::step: the cap-reached warning was meant to fire once, but spins froze at kMaxSpins so `spins == kMaxSpins` stayed true and it re-fired every iteration after the cap. Now increments-and-compares so it fires exactly once. Also drop the now-redundant spins < kMaxSpins guard around the delay ramp since delay already self-clamps at the 1ms cap. - Fold arm_park + block_current + the conditional step into NonFiberBackoff::wait(onFiber), and make step() private so no caller can feed it a fiber block result. Route WaitSema, WaitEventFlag, SleepThread, and WaitForNextVSyncTick's two block_current sites through it, preserving each site's exact lock handling and the arm-before-block ordering. --- .../src/lib/Kernel/Syscalls/Helpers/Runtime.h | 75 ++++++++++++------- .../src/lib/Kernel/Syscalls/Interrupt.cpp | 12 +-- ps2xRuntime/src/lib/Kernel/Syscalls/Sync.cpp | 16 +--- .../src/lib/Kernel/Syscalls/Thread.cpp | 13 +--- 4 files changed, 57 insertions(+), 59 deletions(-) diff --git a/ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/Runtime.h b/ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/Runtime.h index d94a33c35..9f0515554 100644 --- a/ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/Runtime.h +++ b/ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/Runtime.h @@ -24,6 +24,26 @@ static inline void wakeWaiters(std::mutex &m, std::vector +static inline void withGuestTokenDropped(ps2sched::BlockResult br, PauseFn pause) +{ + if (br == ps2sched::BlockResult::NonFiberOwner) + { + ps2sched::async_guest_end(); + pause(); + ps2sched::async_guest_begin(); + } + else // NonFiberNoTok (or, defensively, any non-Parked non-fiber result) + { + pause(); + } +} + // Bounded backoff for a borrowed host worker that hit a blocking syscall. // Translates a non-fiber BlockResult into the correct token handling, then // sleeps with exponential backoff (1us -> 1ms cap). After kMaxSpins iterations @@ -35,37 +55,45 @@ struct NonFiberBackoff int spins = 0; std::chrono::microseconds delay{1}; - // Sleeps this worker once with exponential backoff. The syscall's Mesa loop - // decides whether to re-check its wait condition and loop again. - void step(ps2sched::BlockResult br) + // arm_park (only for a real fiber) + block_current, then, if the wake was + // a non-fiber result, one backoff step. Never runs step() for a Parked + // (fiber) wake, since a fiber's block_current already did the real park. + ps2sched::BlockResult wait(bool onFiber) { - // Drop/reacquire the guest token only if we actually own it. - if (br == ps2sched::BlockResult::NonFiberOwner) + if (onFiber) { - ps2sched::async_guest_end(); - std::this_thread::sleep_for(delay); - ps2sched::async_guest_begin(); + ps2sched::arm_park(); } - else // NonFiberNoTok (or, defensively, any non-Parked non-fiber result) + const ps2sched::BlockResult br = ps2sched::block_current(); + if (br == ps2sched::BlockResult::NonFiberOwner || + br == ps2sched::BlockResult::NonFiberNoTok) { - std::this_thread::sleep_for(delay); + step(br); } + return br; + } + +private: + // Sleeps this worker once with exponential backoff. The syscall's Mesa loop + // decides whether to re-check its wait condition and loop again. + void step(ps2sched::BlockResult br) + { + withGuestTokenDropped(br, [&] { std::this_thread::sleep_for(delay); }); + + // delay self-clamps at the 1ms cap below, so no separate spins < kMaxSpins + // guard is needed to keep it from overflowing past the cap. + delay = std::min(delay * 2, std::chrono::microseconds(1000)); constexpr int kMaxSpins = 50; - if (spins == kMaxSpins) + // Fires exactly once, the iteration spins reaches kMaxSpins, then spins + // freezes at kMaxSpins so this can never become true again. + if (spins < kMaxSpins && ++spins == kMaxSpins) { std::fprintf(stderr, "[ps2sched] WARNING: borrowed host worker has blocked on a guest " "condition for %d retries; capping backoff at 1ms (possible " "interrupt-context deadlock)\n", kMaxSpins); } - if (spins < kMaxSpins) - { - ++spins; - delay *= 2; - if (delay > std::chrono::microseconds(1000)) - delay = std::chrono::microseconds(1000); - } } }; @@ -73,16 +101,7 @@ struct NonFiberBackoff // a borrowed worker has no wait-list to re-check, so it does not loop. inline void nonFiberBlockBackoff(ps2sched::BlockResult br) { - if (br == ps2sched::BlockResult::NonFiberOwner) - { - ps2sched::async_guest_end(); - std::this_thread::yield(); - ps2sched::async_guest_begin(); - } - else - { - std::this_thread::yield(); - } + withGuestTokenDropped(br, [] { std::this_thread::yield(); }); } static void throwIfTerminated(const std::shared_ptr &info) diff --git a/ps2xRuntime/src/lib/Kernel/Syscalls/Interrupt.cpp b/ps2xRuntime/src/lib/Kernel/Syscalls/Interrupt.cpp index 5b6126fd5..9505d6768 100644 --- a/ps2xRuntime/src/lib/Kernel/Syscalls/Interrupt.cpp +++ b/ps2xRuntime/src/lib/Kernel/Syscalls/Interrupt.cpp @@ -460,8 +460,7 @@ namespace ps2_syscalls NonFiberBackoff nfBackoff; for (;;) { - const ps2sched::BlockResult br = ps2sched::block_current(); - nfBackoff.step(br); + const ps2sched::BlockResult br = nfBackoff.wait(false); std::lock_guard lock(g_vsync_flag_mutex); if (g_vsync_tick_counter != entryTick) @@ -481,11 +480,12 @@ namespace ps2_syscalls std::lock_guard lock(g_vsync_flag_mutex); g_vsync_waitList.emplace_back(g_currentThreadId, selfToken); } - ps2sched::arm_park(); - // Block the current fiber; signalVSyncFlag calls the validated wakeup - // from the IRQ worker thread to wake us. - const ps2sched::BlockResult br = ps2sched::block_current(); + // from the IRQ worker thread to wake us. onFiber is always true here + // (the !onFiber path above already returned), so wait() never runs a + // backoff step for this park. + NonFiberBackoff nfBackoff; + const ps2sched::BlockResult br = nfBackoff.wait(true); // A fiber woken from a real park (Parked) may have been woken by // scheduler_shutdown / TerminateThread rather than a vsync tick. If so, diff --git a/ps2xRuntime/src/lib/Kernel/Syscalls/Sync.cpp b/ps2xRuntime/src/lib/Kernel/Syscalls/Sync.cpp index 2c41b76f3..d8ab144ba 100644 --- a/ps2xRuntime/src/lib/Kernel/Syscalls/Sync.cpp +++ b/ps2xRuntime/src/lib/Kernel/Syscalls/Sync.cpp @@ -413,20 +413,9 @@ namespace ps2_syscalls // Drop sema->m BEFORE any scheduler operation. lock.unlock(); - if (onFiber) - { - ps2sched::arm_park(); - } - - const ps2sched::BlockResult br = ps2sched::block_current(); - // Non-fiber (borrowed host worker) path: bounded exponential backoff // so a never-satisfied condition cannot busy-spin the CPU. - if (br == ps2sched::BlockResult::NonFiberOwner || - br == ps2sched::BlockResult::NonFiberNoTok) - { - nfBackoff.step(br); - } + const ps2sched::BlockResult br = nfBackoff.wait(onFiber); // === Woke up here === lock.lock(); @@ -836,8 +825,7 @@ namespace ps2_syscalls } lock.unlock(); - const ps2sched::BlockResult br = ps2sched::block_current(); - nfBackoff.step(br); + const ps2sched::BlockResult br = nfBackoff.wait(false); lock.lock(); if (tInfo) diff --git a/ps2xRuntime/src/lib/Kernel/Syscalls/Thread.cpp b/ps2xRuntime/src/lib/Kernel/Syscalls/Thread.cpp index 01e3a78a2..a67f5ca74 100644 --- a/ps2xRuntime/src/lib/Kernel/Syscalls/Thread.cpp +++ b/ps2xRuntime/src/lib/Kernel/Syscalls/Thread.cpp @@ -694,19 +694,10 @@ namespace ps2_syscalls // Arm on every iteration: block_current() consumes wake_pending, // so a wake arriving in the new publish/arm window would be missed // if we skipped re-arming on subsequent iterations. - if (onFiber) - { - ps2sched::arm_park(); - } - const ps2sched::BlockResult br = ps2sched::block_current(); - + // // Non-fiber (but identified) waiter: bounded exponential backoff // so a never-satisfied condition cannot busy-spin the CPU. - if (br == ps2sched::BlockResult::NonFiberOwner || - br == ps2sched::BlockResult::NonFiberNoTok) - { - nfBackoff.step(br); - } + const ps2sched::BlockResult br = nfBackoff.wait(onFiber); lock.lock(); From 1ff8068ceeb9caaec1688e6b0a8eafa00476594d Mon Sep 17 00:00:00 2001 From: Shane Michael Mathews Date: Tue, 7 Jul 2026 19:09:15 -0400 Subject: [PATCH 07/15] simplify(sched): BlockResult 4->3 + holds_guest_token, strong-typed FiberToken, context-carrying stop callback --- ps2xRuntime/include/ps2_scheduler.h | 49 ++++++++++++------ .../src/lib/Kernel/Syscalls/Helpers/Runtime.h | 27 +++++----- .../src/lib/Kernel/Syscalls/Helpers/State.h | 4 +- .../src/lib/Kernel/Syscalls/Interrupt.cpp | 12 ++--- .../src/lib/Kernel/Syscalls/Interrupt.h | 4 +- ps2xRuntime/src/lib/Kernel/Syscalls/RPC.cpp | 2 +- ps2xRuntime/src/lib/Kernel/Syscalls/Sync.cpp | 22 ++++---- .../src/lib/Kernel/Syscalls/Thread.cpp | 4 +- ps2xRuntime/src/lib/ps2_runtime.cpp | 10 +--- ps2xRuntime/src/lib/ps2_scheduler.cpp | 50 +++++++++++-------- ps2xTest/src/ps2_runtime_expansion_tests.cpp | 12 ++--- 11 files changed, 110 insertions(+), 86 deletions(-) diff --git a/ps2xRuntime/include/ps2_scheduler.h b/ps2xRuntime/include/ps2_scheduler.h index 33e4f2b49..f75e4df23 100644 --- a/ps2xRuntime/include/ps2_scheduler.h +++ b/ps2xRuntime/include/ps2_scheduler.h @@ -24,6 +24,16 @@ struct SyscallOverrideStack; // defined in ps2_syscall_override_state.h namespace ps2sched { + // Opaque identity token for a fiber, encoding its generation + tid (see + // current_fiber_token()). A distinct type (rather than a bare uint64_t) + // keeps a raw fiber-token bit pattern from being silently accepted wherever + // a uint64_t is expected. Only ever compared for equality, default- + // constructed (FiberToken{} — the "no fiber" sentinel), or passed back to + // enqueue_external_wakeup_validated. Code that genuinely needs the + // underlying bits (e.g. constructing/validating generation+tid) casts + // explicitly; the bit layout and validation logic are unchanged. + enum class FiberToken : uint64_t {}; + // --- Lifecycle --- // Initialise global scheduler state and create the single guest executor @@ -37,8 +47,11 @@ namespace ps2sched // Register a callback that scheduler_shutdown() invokes once, just before // joining the guest executor thread. The runtime sets this to request a // stop so dispatchLoop's while(!isStopRequested()) exits between dispatched - // functions. Pass nullptr to clear. May be called before scheduler_init(). - void scheduler_set_stop_callback(void (*fn)()); + // functions. `ctx` is passed back to `fn` unmodified (letting the callback + // carry its own context instead of relying on a file-static). Pass + // fn==nullptr to clear (ctx is ignored in that case). May be called before + // scheduler_init(). + void scheduler_set_stop_callback(void (*fn)(void*), void* ctx); // --- Thread lifecycle (called from Thread.cpp / Lifecycle.cpp) --- @@ -76,19 +89,27 @@ namespace ps2sched Parked, // a fiber actually parked and was later resumed by a waker. WokenInWindow, // a fiber: a wakeup arrived in the arm/publish window; do // not park. Caller re-checks its wait condition. - NonFiberOwner, // a borrowed host worker that HOLDS the guest token: caller - // must async_guest_end()/yield/async_guest_begin(), then - // re-check. - NonFiberNoTok // a borrowed host worker that does NOT hold the token: - // caller just yields the host thread and re-checks; it - // must NOT touch the guest token. + NonFiber // a borrowed host worker (not a fiber): caller cannot park + // on the fiber scheduler and must yield the host thread + // and re-check. Whether it must also drop/reacquire the + // guest token around that yield is a SEPARATE question, + // answered by holds_guest_token() (below), not by this + // result. }; // Park the currently-running fiber. Caller MUST have called arm_park() first // and then published itself to the object wait-list. The fiber must release any - // object mutexes BEFORE calling this. See BlockResult for the four outcomes. + // object mutexes BEFORE calling this. See BlockResult for the three outcomes. BlockResult block_current(); + // True iff the calling OS thread is a borrowed host worker that currently + // holds the guest token (acquired via async_guest_begin(), not yet released + // via async_guest_end()). Always false on a fiber. Callers that get + // BlockResult::NonFiber from block_current() use this to decide whether they + // must drop/reacquire the guest token around their backoff pause — dropping + // a token this thread does not hold would be a bug. + bool holds_guest_token(); + // Wake a blocked fiber from within another fiber (from a guest thread). // Gated on suspendCount == 0. void make_ready(int tid); @@ -98,12 +119,12 @@ namespace ps2sched // (from current_fiber_token()). The token encodes the fiber's generation, so // a recycled tid whose new fiber has a different generation will not match // and the stale wakeup is dropped. Validation happens under g_sched_mutex. - void enqueue_external_wakeup_validated(int tid, uint64_t token); + void enqueue_external_wakeup_validated(int tid, FiberToken token); - // Opaque identity token for the fiber currently running on this thread, or 0 - // if not on a fiber. Encodes generation + tid. Only ever compared for - // equality / passed back to enqueue_external_wakeup_validated. - uint64_t current_fiber_token(); + // Opaque identity token for the fiber currently running on this thread, or + // FiberToken{} if not on a fiber. Encodes generation + tid. Only ever + // compared for equality / passed back to enqueue_external_wakeup_validated. + FiberToken current_fiber_token(); // Yield if a higher-priority fiber is ready. Enqueues self Ready BEFORE // yielding, so the fiber is never 'Running but off-queue'. diff --git a/ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/Runtime.h b/ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/Runtime.h index 9f0515554..b0fe97c43 100644 --- a/ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/Runtime.h +++ b/ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/Runtime.h @@ -11,9 +11,9 @@ // shape: callers whose critical section also mutates other state alongside // the swap (e.g. marking the object deleted) must keep doing that inline // instead of calling this, so the lock is still held across both writes. -static inline void wakeWaiters(std::mutex &m, std::vector> &list) +static inline void wakeWaiters(std::mutex &m, std::vector> &list) { - std::vector> waiters; + std::vector> waiters; { std::lock_guard lk(m); waiters.swap(list); @@ -24,21 +24,21 @@ static inline void wakeWaiters(std::mutex &m, std::vector -static inline void withGuestTokenDropped(ps2sched::BlockResult br, PauseFn pause) +static inline void withGuestTokenDropped(PauseFn pause) { - if (br == ps2sched::BlockResult::NonFiberOwner) + if (ps2sched::holds_guest_token()) { ps2sched::async_guest_end(); pause(); ps2sched::async_guest_begin(); } - else // NonFiberNoTok (or, defensively, any non-Parked non-fiber result) + else { pause(); } @@ -65,10 +65,9 @@ struct NonFiberBackoff ps2sched::arm_park(); } const ps2sched::BlockResult br = ps2sched::block_current(); - if (br == ps2sched::BlockResult::NonFiberOwner || - br == ps2sched::BlockResult::NonFiberNoTok) + if (br == ps2sched::BlockResult::NonFiber) { - step(br); + step(); } return br; } @@ -76,9 +75,9 @@ struct NonFiberBackoff private: // Sleeps this worker once with exponential backoff. The syscall's Mesa loop // decides whether to re-check its wait condition and loop again. - void step(ps2sched::BlockResult br) + void step() { - withGuestTokenDropped(br, [&] { std::this_thread::sleep_for(delay); }); + withGuestTokenDropped([&] { std::this_thread::sleep_for(delay); }); // delay self-clamps at the 1ms cap below, so no separate spins < kMaxSpins // guard is needed to keep it from overflowing past the cap. @@ -99,9 +98,9 @@ struct NonFiberBackoff // One-shot: drop/reacquire token (if owned) and yield once. SleepThread for // a borrowed worker has no wait-list to re-check, so it does not loop. -inline void nonFiberBlockBackoff(ps2sched::BlockResult br) +inline void nonFiberBlockBackoff() { - withGuestTokenDropped(br, [] { std::this_thread::yield(); }); + withGuestTokenDropped([] { std::this_thread::yield(); }); } static void throwIfTerminated(const std::shared_ptr &info) diff --git a/ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/State.h b/ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/State.h index 3b9da6518..94a98a0ee 100644 --- a/ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/State.h +++ b/ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/State.h @@ -168,7 +168,7 @@ struct SemaInfo // Wait list of blocked guest threads. Each entry is {tid, generation token} // where the token was captured via ps2sched::current_fiber_token() at push // time. Protected by m; never hold m across a scheduling yield. - std::vector> waitList; + std::vector> waitList; }; struct EventFlagInfo @@ -181,7 +181,7 @@ struct EventFlagInfo bool deleted = false; std::mutex m; // See SemaInfo::waitList. - std::vector> waitList; + std::vector> waitList; }; struct AlarmInfo diff --git a/ps2xRuntime/src/lib/Kernel/Syscalls/Interrupt.cpp b/ps2xRuntime/src/lib/Kernel/Syscalls/Interrupt.cpp index 9505d6768..b318fba38 100644 --- a/ps2xRuntime/src/lib/Kernel/Syscalls/Interrupt.cpp +++ b/ps2xRuntime/src/lib/Kernel/Syscalls/Interrupt.cpp @@ -17,7 +17,7 @@ namespace ps2_syscalls std::mutex g_irq_worker_mutex; std::condition_variable g_irq_worker_cv; std::mutex g_vsync_flag_mutex; - std::vector> g_vsync_waitList; + std::vector> g_vsync_waitList; std::atomic g_irq_worker_stop{false}; std::atomic g_irq_worker_running{false}; std::thread g_irq_worker_thread; // joinable worker handle so stopInterruptWorker() can join it @@ -433,9 +433,9 @@ namespace ps2_syscalls ensureInterruptWorkerRunning(rdram, runtime); // Opaque identity of the fiber that is about to park. Non-fiber host - // workers get token 0 and never publish to the wait-list. - const uint64_t selfToken = ps2sched::current_fiber_token(); - const bool onFiber = (selfToken != 0u); + // workers get token FiberToken{} and never publish to the wait-list. + const ps2sched::FiberToken selfToken = ps2sched::current_fiber_token(); + const bool onFiber = (selfToken != ps2sched::FiberToken{}); // Snapshot the tick we are waiting to advance past. A non-fiber worker // never publishes to g_vsync_waitList (it cannot park), so it cannot @@ -502,7 +502,7 @@ namespace ps2_syscalls std::lock_guard clLock(g_vsync_flag_mutex); auto &wl = g_vsync_waitList; auto it = std::find_if(wl.begin(), wl.end(), - [selfToken](const std::pair &e) + [selfToken](const std::pair &e) { return e.second == selfToken; }); if (it != wl.end()) wl.erase(it); } @@ -518,7 +518,7 @@ namespace ps2_syscalls std::lock_guard lock(g_vsync_flag_mutex); auto &wl = g_vsync_waitList; auto it = std::find_if(wl.begin(), wl.end(), - [selfToken](const std::pair &e) + [selfToken](const std::pair &e) { return e.second == selfToken; }); if (it != wl.end()) { diff --git a/ps2xRuntime/src/lib/Kernel/Syscalls/Interrupt.h b/ps2xRuntime/src/lib/Kernel/Syscalls/Interrupt.h index fecb9b423..e39afe161 100644 --- a/ps2xRuntime/src/lib/Kernel/Syscalls/Interrupt.h +++ b/ps2xRuntime/src/lib/Kernel/Syscalls/Interrupt.h @@ -26,9 +26,9 @@ namespace ps2_syscalls // (from ps2sched::current_fiber_token(), which encodes the fiber's // generation). signalVSyncFlag delivers the wakeup only to the exact // fiber that parked, so a recycled tid cannot receive a stale tick. - // Borrowed host workers (g_currentThreadId == -1, token == 0) never park + // Borrowed host workers (g_currentThreadId == -1, FiberToken{}) never park // here, so every stored entry has a non-zero token. - extern std::vector> g_vsync_waitList; + extern std::vector> g_vsync_waitList; extern std::atomic g_irq_worker_stop; extern std::atomic g_irq_worker_running; extern std::thread g_irq_worker_thread; // joinable worker handle diff --git a/ps2xRuntime/src/lib/Kernel/Syscalls/RPC.cpp b/ps2xRuntime/src/lib/Kernel/Syscalls/RPC.cpp index 3dcd9da5e..c130a3293 100644 --- a/ps2xRuntime/src/lib/Kernel/Syscalls/RPC.cpp +++ b/ps2xRuntime/src/lib/Kernel/Syscalls/RPC.cpp @@ -612,7 +612,7 @@ namespace ps2_syscalls bool signaled = false; int wokenTid = 0; - uint64_t wokenToken = 0; + ps2sched::FiberToken wokenToken{}; { std::lock_guard lock(sema->m); if (!sema->deleted && sema->count < sema->maxCount) diff --git a/ps2xRuntime/src/lib/Kernel/Syscalls/Sync.cpp b/ps2xRuntime/src/lib/Kernel/Syscalls/Sync.cpp index d8ab144ba..792964c51 100644 --- a/ps2xRuntime/src/lib/Kernel/Syscalls/Sync.cpp +++ b/ps2xRuntime/src/lib/Kernel/Syscalls/Sync.cpp @@ -213,7 +213,7 @@ namespace ps2_syscalls } // Collect all waiting threads, then wake each with token validation. - std::vector> waiters; + std::vector> waiters; { std::lock_guard lk(sema->m); sema->deleted = true; @@ -250,7 +250,7 @@ namespace ps2_syscalls // PS2 EE BIOS returns sid on success; KE_SEMA_OVF overrides on overflow. int ret = sid; int wokenTid = 0; - uint64_t wokenToken = 0; + ps2sched::FiberToken wokenToken{}; { std::unique_lock lock(sema->m); if (sema->count >= sema->maxCount) @@ -390,8 +390,8 @@ namespace ps2_syscalls // target lookup and SignalSema/DeleteSema's targeted wakeup see a // non-fiber host thread carrying a real guest tid (e.g. a raw // std::thread standing in for a guest thread in tests). Off-fiber, - // current_fiber_token() is 0; SignalSema/DeleteSema's - // enqueue_external_wakeup_validated drops token==0 entries, so + // current_fiber_token() is FiberToken{}; SignalSema/DeleteSema's + // enqueue_external_wakeup_validated drops FiberToken{} entries, so // publishing here is harmless for such a waiter — it doesn't need // the wake, it re-polls via the backoff loop below. A fully // borrowed host worker (info==nullptr) would alias every other @@ -431,7 +431,7 @@ namespace ps2_syscalls { auto& wl = sema->waitList; auto it = std::find_if(wl.begin(), wl.end(), - [](const std::pair& e){ return e.first == g_currentThreadId; }); + [](const std::pair& e){ return e.first == g_currentThreadId; }); if (it != wl.end()) wl.erase(it); } sema->waiters--; @@ -612,7 +612,7 @@ namespace ps2_syscalls return; } - std::vector> evfWaiters; + std::vector> evfWaiters; { std::lock_guard lk(info->m); info->deleted = true; @@ -646,7 +646,7 @@ namespace ps2_syscalls return; } - std::vector> setEvfWaiters; + std::vector> setEvfWaiters; { std::unique_lock lock(info->m); info->bits |= bits; @@ -776,8 +776,8 @@ namespace ps2_syscalls // status in THS_WAIT/THS_WAITSUSPEND for the duration, so // ReferEventFlagStatus's numThreads count and ReleaseWaitThread's // target lookup both see it — matching WaitSema. Off-fiber, - // current_fiber_token() is 0; SetEventFlag's wake fan-out - // tolerates token==0 (enqueue_external_wakeup_validated drops it): + // current_fiber_token() is FiberToken{}; SetEventFlag's wake fan-out + // tolerates FiberToken{} (enqueue_external_wakeup_validated drops it): // this waiter doesn't need the wake, it re-polls every backoff // step. A true borrowed worker (tInfo == nullptr) never publishes // and relies solely on satisfied() re-checks, same as before. @@ -832,7 +832,7 @@ namespace ps2_syscalls { auto &wl = info->waitList; auto it = std::find_if(wl.begin(), wl.end(), - [](const std::pair& e){ return e.first == g_currentThreadId; }); + [](const std::pair& e){ return e.first == g_currentThreadId; }); if (it != wl.end()) wl.erase(it); } info->waiters--; @@ -925,7 +925,7 @@ namespace ps2_syscalls { auto &wl = info->waitList; auto it = std::find_if(wl.begin(), wl.end(), - [](const std::pair& e){ return e.first == g_currentThreadId; }); + [](const std::pair& e){ return e.first == g_currentThreadId; }); if (it != wl.end()) wl.erase(it); info->waiters--; } diff --git a/ps2xRuntime/src/lib/Kernel/Syscalls/Thread.cpp b/ps2xRuntime/src/lib/Kernel/Syscalls/Thread.cpp index a67f5ca74..07e1f0884 100644 --- a/ps2xRuntime/src/lib/Kernel/Syscalls/Thread.cpp +++ b/ps2xRuntime/src/lib/Kernel/Syscalls/Thread.cpp @@ -661,8 +661,8 @@ namespace ps2_syscalls // There is no ThreadInfo / wakeupCount to consult. Park-and-retry // once with bounded backoff, then return OK so the worker does // not livelock the emulator. - ps2sched::BlockResult br = ps2sched::block_current(); - nonFiberBlockBackoff(br); + ps2sched::block_current(); + nonFiberBlockBackoff(); setReturnS32(ctx, 0); return; } diff --git a/ps2xRuntime/src/lib/ps2_runtime.cpp b/ps2xRuntime/src/lib/ps2_runtime.cpp index 5c4a57f18..b417954fe 100644 --- a/ps2xRuntime/src/lib/ps2_runtime.cpp +++ b/ps2xRuntime/src/lib/ps2_runtime.cpp @@ -2112,10 +2112,6 @@ void PS2Runtime::HandleIntegerOverflow(R5900Context *ctx) raiseCop0Exception(ctx, EXCEPTION_INTEGER_OVERFLOW); } -// Trampoline pointer used by the scheduler stop callback (non-capturing lambda). -// Set in run() before scheduler_init(); cleared after scheduler_shutdown() returns. -static PS2Runtime* g_stopRuntime = nullptr; - void PS2Runtime::run() { m_stopRequested.store(false, std::memory_order_relaxed); @@ -2143,8 +2139,7 @@ void PS2Runtime::run() // Initialize the fiber/pool scheduler. ps2sched::scheduler_init(); - g_stopRuntime = this; - ps2sched::scheduler_set_stop_callback(+[]{ if (g_stopRuntime) g_stopRuntime->requestStopFlagOnly(); }); + ps2sched::scheduler_set_stop_callback(+[](void* p) { static_cast(p)->requestStopFlagOnly(); }, this); // Create the main guest fiber (tid=1). uint8_t *rdram = m_memory.getRDRAM(); @@ -2229,8 +2224,7 @@ void PS2Runtime::run() // Signal all guest fibers to stop and join the pool threads. ps2sched::scheduler_shutdown(); - ps2sched::scheduler_set_stop_callback(nullptr); - g_stopRuntime = nullptr; + ps2sched::scheduler_set_stop_callback(nullptr, nullptr); if (m_debugUiInitialized && m_debugUiShutdownCallback) { diff --git a/ps2xRuntime/src/lib/ps2_scheduler.cpp b/ps2xRuntime/src/lib/ps2_scheduler.cpp index 41ecc24b2..b5a09a853 100644 --- a/ps2xRuntime/src/lib/ps2_scheduler.cpp +++ b/ps2xRuntime/src/lib/ps2_scheduler.cpp @@ -79,8 +79,11 @@ void (*g_fiber_exit_hook)(int tid, uint8_t* rdram, R5900Context* ctx, PS2Runtime // Stop callback — set by scheduler_set_stop_callback(); invoked by // scheduler_shutdown() before joining g_guest_thread. Read/written under -// g_sched_mutex. nullptr-safe. -static void (*g_request_runtime_stop_fn)() = nullptr; +// g_sched_mutex. nullptr-safe. g_request_runtime_stop_ctx is passed back to +// the callback unmodified, letting the caller carry its own context instead +// of relying on a file-static (e.g. PS2Runtime::run()'s `this`). +static void (*g_request_runtime_stop_fn)(void*) = nullptr; +static void* g_request_runtime_stop_ctx = nullptr; static constexpr size_t kFiberStackBytes = 1024 * 1024; // 1 MiB @@ -447,10 +450,11 @@ void ps2sched::scheduler_init() g_guest_thread = std::thread(guest_executor_main); } -void ps2sched::scheduler_set_stop_callback(void (*fn)()) +void ps2sched::scheduler_set_stop_callback(void (*fn)(void*), void* ctx) { std::lock_guard lk(g_sched_mutex); g_request_runtime_stop_fn = fn; + g_request_runtime_stop_ctx = fn ? ctx : nullptr; } void ps2sched::scheduler_shutdown() @@ -501,12 +505,14 @@ void ps2sched::scheduler_shutdown() // between dispatched functions. Invoked OUTSIDE g_sched_mutex: the callback // runs runtime.requestStopFlagOnly(), which sets its own atomic and must not // nest under g_sched_mutex. - void (*stopFn)() = nullptr; + void (*stopFn)(void*) = nullptr; + void* stopCtx = nullptr; { std::lock_guard lk(g_sched_mutex); - stopFn = g_request_runtime_stop_fn; + stopFn = g_request_runtime_stop_fn; + stopCtx = g_request_runtime_stop_ctx; } - if (stopFn) stopFn(); + if (stopFn) stopFn(stopCtx); if (g_guest_thread.joinable()) g_guest_thread.join(); @@ -792,15 +798,10 @@ ps2sched::BlockResult ps2sched::block_current() { // A borrowed IRQ/alarm/RPC worker (running guest code under // AsyncGuestScope) called a blocking syscall. We cannot park a host - // worker on the fiber scheduler. Report whether this worker actually holds - // the guest token so the caller only drops/reacquires a token it owns. - // tls_holds_guest_token and g_guest_token_held_by_host are set/cleared - // together under g_sched_mutex in async_guest_begin/async_guest_end, so - // on THIS thread tls_holds_guest_token==true already implies - // g_guest_token_held_by_host==true. No need to re-read the global or - // take the lock. - return tls_holds_guest_token ? BlockResult::NonFiberOwner - : BlockResult::NonFiberNoTok; + // worker on the fiber scheduler. Whether this worker actually holds the + // guest token (and so must drop/reacquire it around its backoff pause) + // is answered separately by holds_guest_token(). + return BlockResult::NonFiber; } bool throwTerminate = false; bool wokenInWindow = false; @@ -862,6 +863,15 @@ ps2sched::BlockResult ps2sched::block_current() return BlockResult::Parked; } +// True on a host (non-fiber) worker thread that currently holds the guest +// token. Always false on a fiber (fibers never call async_guest_begin — see +// on_guest_execution_slot's abort guard). Just reads tls_holds_guest_token, +// which async_guest_begin/async_guest_end set/clear under g_sched_mutex. +bool ps2sched::holds_guest_token() +{ + return tls_holds_guest_token; +} + // make_ready (suspendCount gate) void ps2sched::make_ready(int tid) { @@ -893,10 +903,10 @@ static inline uint64_t make_fiber_token(const FiberContext* fc) static_cast(static_cast(fc->tid)); } -uint64_t ps2sched::current_fiber_token() +ps2sched::FiberToken ps2sched::current_fiber_token() { FiberContext* fc = tls_current_fiber; - return fc ? make_fiber_token(fc) : 0u; + return static_cast(fc ? make_fiber_token(fc) : 0u); } ::DispatchHistory& ps2sched::current_dispatch_history() noexcept @@ -925,7 +935,7 @@ ::SyscallOverrideStack& ps2sched::current_active_syscall_overrides() noexcept return s_nonFiberOverrideState; } -void ps2sched::enqueue_external_wakeup_validated(int tid, uint64_t token) +void ps2sched::enqueue_external_wakeup_validated(int tid, FiberToken token) { { std::lock_guard lk(g_sched_mutex); @@ -936,8 +946,8 @@ void ps2sched::enqueue_external_wakeup_validated(int tid, uint64_t token) // WaitForNextVSyncTick has the same publish/arm window as WaitSema, so a // tick that fires while the fiber is still Running must reach wake_locked. if (fc != nullptr && - token != 0u && - make_fiber_token(fc) == token && + token != FiberToken{} && + make_fiber_token(fc) == static_cast(token) && fc->suspendCount.load(std::memory_order_relaxed) == 0) { wake_locked(fc); diff --git a/ps2xTest/src/ps2_runtime_expansion_tests.cpp b/ps2xTest/src/ps2_runtime_expansion_tests.cpp index d446e6c4e..51c73548c 100644 --- a/ps2xTest/src/ps2_runtime_expansion_tests.cpp +++ b/ps2xTest/src/ps2_runtime_expansion_tests.cpp @@ -2585,7 +2585,7 @@ namespace // same SleepThread/record/signal sequence as stepSleepRecordSignal. static void stepSleepRecordSignalPublishToken(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) { - const uint64_t tok = ps2sched::current_fiber_token(); + const uint64_t tok = static_cast(ps2sched::current_fiber_token()); gT11TokenLo.store(static_cast(tok & 0xFFFFFFFFu), std::memory_order_relaxed); gT11TokenHi.store(static_cast(tok >> 32u), std::memory_order_release); stepSleepRecordSignal(rdram, ctx, runtime); @@ -3400,7 +3400,7 @@ void register_scheduler_protocol_tests() }, std::chrono::milliseconds(500)); t.IsTrue(isSuspended, "T11: fiber reached THS_WAITSUSPEND"); - std::thread foreign([&](){ ps2sched::enqueue_external_wakeup_validated(tid, token); }); + std::thread foreign([&](){ ps2sched::enqueue_external_wakeup_validated(tid, static_cast(token)); }); foreign.join(); std::this_thread::sleep_for(std::chrono::milliseconds(40)); @@ -3982,7 +3982,7 @@ namespace std::memcpy(&workSid, rdram + kRSlotWorkSid, 4); std::memcpy(&doneSid, rdram + kRSlotDoneSid, 4); - const uint64_t tok = ps2sched::current_fiber_token(); + const uint64_t tok = static_cast(ps2sched::current_fiber_token()); gRTokenLo.store(static_cast(tok & 0xFFFFFFFFu), std::memory_order_relaxed); gRTokenHi.store(static_cast(tok >> 32u), std::memory_order_release); @@ -4118,7 +4118,7 @@ void register_scheduler_race_tests() { while (!stopWakers.load(std::memory_order_acquire)) { - ps2sched::enqueue_external_wakeup_validated(tid, token); // safe no-op unless genuinely Blocked + ps2sched::enqueue_external_wakeup_validated(tid, static_cast(token)); // safe no-op unless genuinely Blocked } }); @@ -6048,7 +6048,7 @@ namespace // ----------------------------------------------------------------------- static void stepY4bPublishTokenThenExit(uint8_t *rdram, R5900Context *ctx, PS2Runtime * /*runtime*/) { - const uint64_t tok = ps2sched::current_fiber_token(); + const uint64_t tok = static_cast(ps2sched::current_fiber_token()); const uint32_t lo = static_cast(tok & 0xFFFFFFFFu); const uint32_t hi = static_cast(tok >> 32u); gY4bTokenLo.store(lo, std::memory_order_relaxed); @@ -6493,7 +6493,7 @@ void register_scheduler_sema_delete_tests() // The scheduler must detect the generation mismatch and drop it. std::thread foreignWaker([&]() { - ps2sched::enqueue_external_wakeup_validated(sleeperTid, wrongToken); + ps2sched::enqueue_external_wakeup_validated(sleeperTid, static_cast(wrongToken)); }); foreignWaker.join(); From 215caf86a6e34b86944185138bef5e92f8b1aeaa Mon Sep 17 00:00:00 2001 From: Shane Michael Mathews Date: Tue, 7 Jul 2026 19:17:50 -0400 Subject: [PATCH 08/15] simplify(sync): unify WaitEventFlag wait loop, publish/forceRelease helpers, markDeletedAndWake --- .../src/lib/Kernel/Syscalls/Helpers/Runtime.h | 104 ++++++ ps2xRuntime/src/lib/Kernel/Syscalls/Sync.cpp | 344 +++++------------- 2 files changed, 189 insertions(+), 259 deletions(-) diff --git a/ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/Runtime.h b/ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/Runtime.h index b0fe97c43..5d3c7b67c 100644 --- a/ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/Runtime.h +++ b/ps2xRuntime/src/lib/Kernel/Syscalls/Helpers/Runtime.h @@ -1,5 +1,6 @@ #include "ThreadExit.h" #include "ps2_scheduler.h" +#include #include #include #include @@ -24,6 +25,55 @@ static inline void wakeWaiters(std::mutex &m, std::vector> &waitList) +{ + waitList.emplace_back(g_currentThreadId, ps2sched::current_fiber_token()); +} + +// Removes the calling thread's entry from `waitList` if still present — a +// Signal/Set/Delete may already have popped it while we were parked. Must be +// called under the owning object's mutex, paired with publishWaiter above. +static inline void unpublishWaiter(std::vector> &waitList) +{ + auto it = std::find_if(waitList.begin(), waitList.end(), + [](const std::pair &e) { return e.first == g_currentThreadId; }); + if (it != waitList.end()) + { + waitList.erase(it); + } +} + +// The "deleting" sibling of wakeWaiters: also marks the object deleted under +// the same critical section as the swap (wakeWaiters deliberately does NOT do +// this — SetEventFlag/SignalSema's copy-based wake path, where waiters +// re-check and self-remove, must not touch `deleted` this way). Used by the +// two Delete* tails, which otherwise share this exact shape: lock, mark +// deleted, swap the wait-list out, unlock, drain with validated external +// wakeups, then yield once if anyone was actually woken. +template +static inline void markDeletedAndWake(WaitableObject &obj) +{ + std::vector> waiters; + { + std::lock_guard lk(obj.m); + obj.deleted = true; + waiters.swap(obj.waitList); + } + for (const auto &[tid, token] : waiters) + { + ps2sched::enqueue_external_wakeup_validated(tid, token); + } + if (!waiters.empty()) + { + ps2sched::maybe_yield(); + } +} + // Drops the guest token around `pause` if this worker actually holds it // (holds_guest_token()), reacquiring it afterward; otherwise just runs // `pause`. This is the one place that knows the async_guest_end/begin @@ -111,6 +161,60 @@ static void throwIfTerminated(const std::shared_ptr &info) } } +// Checks-and-clears info->forceRelease under info->m in one step, returning +// whether a release was actually pending. Null-safe: a borrowed worker +// (info == nullptr) never has forceRelease to consume, so this simply +// returns false — folding in the `if (!info) return false;` guard every +// call site otherwise had to repeat. +static inline bool consumeForceRelease(const std::shared_ptr &info) +{ + if (!info) + { + return false; + } + std::lock_guard lock(info->m); + if (info->forceRelease) + { + info->forceRelease = false; + return true; + } + return false; +} + +// Transitions `info` into THS_WAIT/THS_WAITSUSPEND for the given wait +// type/id and clears forceRelease. This is the ONE safe place to clear +// forceRelease unconditionally: ReleaseWaitThread only ever sets it while +// observing status == WAIT/WAITSUSPEND, and that transition (plus the clear) +// happens atomically under info->m here, so a stale true left over from a +// prior, unrelated wait cannot leak into this one. No-op for a borrowed +// worker (info == nullptr). Pairs with clearThreadWaiting below. +static inline void setThreadWaiting(const std::shared_ptr &info, int waitType, int waitId) +{ + if (!info) + { + return; + } + std::lock_guard lock(info->m); + info->status = (info->suspendCount > 0) ? THS_WAITSUSPEND : THS_WAIT; + info->waitType = waitType; + info->waitId = waitId; + info->forceRelease = false; +} + +// Reverses setThreadWaiting: returns `info` to THS_RUN/THS_SUSPEND and clears +// wait bookkeeping. No-op for a borrowed worker (info == nullptr). +static inline void clearThreadWaiting(const std::shared_ptr &info) +{ + if (!info) + { + return; + } + std::lock_guard lock(info->m); + info->status = (info->suspendCount > 0) ? THS_SUSPEND : THS_RUN; + info->waitType = TSW_NONE; + info->waitId = 0; +} + // Waiting is done via arm_park()/block_current(), which already // cooperatively yields to other fibers/borrowed host workers, so no explicit // guest-execution release/reacquire step is needed. diff --git a/ps2xRuntime/src/lib/Kernel/Syscalls/Sync.cpp b/ps2xRuntime/src/lib/Kernel/Syscalls/Sync.cpp index 792964c51..1c9705910 100644 --- a/ps2xRuntime/src/lib/Kernel/Syscalls/Sync.cpp +++ b/ps2xRuntime/src/lib/Kernel/Syscalls/Sync.cpp @@ -212,21 +212,9 @@ namespace ps2_syscalls g_semas.erase(it); } - // Collect all waiting threads, then wake each with token validation. - std::vector> waiters; - { - std::lock_guard lk(sema->m); - sema->deleted = true; - waiters.swap(sema->waitList); - } - for (const auto& [tid, token] : waiters) - { - ps2sched::enqueue_external_wakeup_validated(tid, token); - } - if (!waiters.empty()) - { - ps2sched::maybe_yield(); - } + // Mark deleted, drain the wait-list, and wake every waiter with a + // validated external wakeup. + markDeletedAndWake(*sema); // PS2 EE BIOS returns sid on success. setReturnS32(ctx, sid); @@ -335,14 +323,7 @@ namespace ps2_syscalls // here, so a stale true left over from a prior, unrelated wait on // this ThreadInfo cannot leak into this wait, and nothing can race // the clear itself. - if (info) - { - std::lock_guard tLock(info->m); - info->status = (info->suspendCount > 0) ? THS_WAITSUSPEND : THS_WAIT; - info->waitType = TSW_SEMA; - info->waitId = sid; - info->forceRelease = false; - } + setThreadWaiting(info, TSW_SEMA, sid); for (;;) { @@ -355,22 +336,10 @@ namespace ps2_syscalls // unconditionally here would silently clobber that pending // release, so consume it and take the same released-exit path // as the post-wake check. - if (info) + if (consumeForceRelease(info)) { - bool release = false; - { - std::lock_guard tLock(info->m); - if (info->forceRelease) - { - info->forceRelease = false; - release = true; - } - } - if (release) - { - ret = KE_RELEASE_WAIT; - break; - } + ret = KE_RELEASE_WAIT; + break; } // sema->waiters is a plain count of every thread genuinely @@ -407,7 +376,7 @@ namespace ps2_syscalls // object mutex. A SignalSema that fires in the publish/arm // window sees g_running_fiber == this fiber and records // wake_pending (consumed by block_current). - sema->waitList.emplace_back(g_currentThreadId, ps2sched::current_fiber_token()); + publishWaiter(sema->waitList); } // Drop sema->m BEFORE any scheduler operation. @@ -429,10 +398,7 @@ namespace ps2_syscalls // increment above. if (info) { - auto& wl = sema->waitList; - auto it = std::find_if(wl.begin(), wl.end(), - [](const std::pair& e){ return e.first == g_currentThreadId; }); - if (it != wl.end()) wl.erase(it); + unpublishWaiter(sema->waitList); } sema->waiters--; @@ -443,22 +409,10 @@ namespace ps2_syscalls break; } - if (info) + if (consumeForceRelease(info)) { - bool release = false; - { - std::lock_guard tLock(info->m); - if (info->forceRelease) - { - info->forceRelease = false; - release = true; - } - } - if (release) - { - ret = KE_RELEASE_WAIT; - break; - } + ret = KE_RELEASE_WAIT; + break; } if (info && info->terminated.load()) @@ -482,13 +436,7 @@ namespace ps2_syscalls // Reset thread status on all non-exception exit paths (fast path, slow path success, // and error breaks). The throw-ThreadExitException path unwinds without reaching here. - if (info) - { - std::lock_guard tLock(info->m); - info->status = (info->suspendCount > 0) ? THS_SUSPEND : THS_RUN; - info->waitType = TSW_NONE; - info->waitId = 0; - } + clearThreadWaiting(info); lock.unlock(); waitWhileSuspended(info); @@ -612,20 +560,9 @@ namespace ps2_syscalls return; } - std::vector> evfWaiters; - { - std::lock_guard lk(info->m); - info->deleted = true; - evfWaiters.swap(info->waitList); - } - for (const auto& [tid, token] : evfWaiters) - { - ps2sched::enqueue_external_wakeup_validated(tid, token); - } - if (!evfWaiters.empty()) - { - ps2sched::maybe_yield(); - } + // Mark deleted, drain the wait-list, and wake every waiter with a + // validated external wakeup. + markDeletedAndWake(*info); setReturnS32(ctx, 0); } @@ -760,203 +697,92 @@ namespace ps2_syscalls if (!satisfied()) { - if (!onFiber) + // Mesa-style re-block loop, unified for fiber and non-fiber + // (borrowed-worker) callers alike — mirrors WaitSema's slow path + // exactly (see WaitSema for the full publish-before-arm and + // forceRelease-clearing rationale). SetEventFlag wakes ALL + // waiters, including ones whose AND-mode bits are only partially + // satisfied, so we re-check satisfied() on every wake and + // re-publish to the wait-list if still unmet. + // + // A thread with a valid guest identity (tInfo != nullptr) also + // publishes itself to info->waitList and keeps its ThreadInfo + // status in THS_WAIT/THS_WAITSUSPEND for the duration, so + // ReferEventFlagStatus's numThreads count and ReleaseWaitThread's + // target lookup both see it, whether or not it is a real fiber. + // Off-fiber, current_fiber_token() is FiberToken{}; SetEventFlag's + // wake fan-out tolerates FiberToken{} (enqueue_external_wakeup_validated + // drops it): this waiter doesn't need the wake, it re-polls every + // backoff step. A true borrowed worker (tInfo == nullptr) never + // publishes and relies solely on satisfied() re-checks. + setThreadWaiting(tInfo, TSW_EVENT, eid); + + NonFiberBackoff nfBackoff; // unused for fibers; ramps for borrowed workers + + for (;;) { - // Non-fiber path (IRQ/alarm worker, or — since tInfo is now keyed - // off g_currentThreadId rather than onFiber — any host thread - // carrying a real guest tid, e.g. a raw std::thread standing in - // for a guest thread in tests): Mesa-style re-block loop, - // mirroring WaitSema's non-fiber handling. We cannot park a host - // worker on the fiber scheduler, so bounded exponential backoff - // (NonFiberBackoff) stands in for arm_park/block_current's real - // parking, re-checking satisfied() under info->m every iteration. - // - // A thread with a valid guest identity (tInfo != nullptr) also - // publishes itself to info->waitList and keeps its ThreadInfo - // status in THS_WAIT/THS_WAITSUSPEND for the duration, so - // ReferEventFlagStatus's numThreads count and ReleaseWaitThread's - // target lookup both see it — matching WaitSema. Off-fiber, - // current_fiber_token() is FiberToken{}; SetEventFlag's wake fan-out - // tolerates FiberToken{} (enqueue_external_wakeup_validated drops it): - // this waiter doesn't need the wake, it re-polls every backoff - // step. A true borrowed worker (tInfo == nullptr) never publishes - // and relies solely on satisfied() re-checks, same as before. - if (tInfo) + // Retry-check: consume a forceRelease that raced in since the + // last iteration decided to re-block (see WaitSema). + if (consumeForceRelease(tInfo)) { - std::lock_guard tLock(tInfo->m); - tInfo->status = (tInfo->suspendCount > 0) ? THS_WAITSUSPEND : THS_WAIT; - tInfo->waitType = TSW_EVENT; - tInfo->waitId = eid; - tInfo->forceRelease = false; + ret = KE_RELEASE_WAIT; + break; } - NonFiberBackoff nfBackoff; - for (;;) + // info->waiters counts every thread genuinely blocked here — + // fiber, non-fiber-with-identity, or fully borrowed + // (g_currentThreadId == -1) — so ReferEventFlagStatus's + // numThreads and the EA_MULTI exclusivity gate above are + // accurate for all of them (mirrors WaitSema's sema->waiters). + info->waiters++; + if (tInfo) { - if (tInfo) - { - bool release = false; - { - std::lock_guard tLock(tInfo->m); - if (tInfo->forceRelease) - { - tInfo->forceRelease = false; - release = true; - } - } - if (release) - { - ret = KE_RELEASE_WAIT; - break; - } - } - - // info->waiters counts every thread genuinely blocked here — - // fiber, non-fiber-with-identity, or fully borrowed - // (g_currentThreadId == -1) — so ReferEventFlagStatus's - // numThreads and the EA_MULTI exclusivity gate above are - // accurate for all of them. It carries no identity, so unlike - // waitList it has no tid-aliasing hazard and is safe to bump - // unconditionally (mirrors WaitSema's sema->waiters). - info->waiters++; - if (tInfo) - { - info->waitList.emplace_back(g_currentThreadId, ps2sched::current_fiber_token()); - } - - lock.unlock(); - const ps2sched::BlockResult br = nfBackoff.wait(false); - lock.lock(); - - if (tInfo) - { - auto &wl = info->waitList; - auto it = std::find_if(wl.begin(), wl.end(), - [](const std::pair& e){ return e.first == g_currentThreadId; }); - if (it != wl.end()) wl.erase(it); - } - info->waiters--; - - if (tInfo) - { - std::lock_guard tLock(tInfo->m); - if (tInfo->forceRelease) - { - tInfo->forceRelease = false; - ret = KE_RELEASE_WAIT; - } - } + // Publish under info->m; arm_park (fiber-only, inside + // nfBackoff.wait) runs AFTER we drop info->m below. + publishWaiter(info->waitList); + } - if (tInfo && tInfo->terminated.load()) - { - throw ThreadExitException(); - } + // Drop info->m BEFORE any scheduler operation. + lock.unlock(); + const ps2sched::BlockResult br = nfBackoff.wait(onFiber); - if (ret != KE_OK || info->deleted || satisfied()) - break; - } + // === Woke up here === + lock.lock(); if (tInfo) { - std::lock_guard tLock(tInfo->m); - tInfo->status = (tInfo->suspendCount > 0) ? THS_SUSPEND : THS_RUN; - tInfo->waitType = TSW_NONE; - tInfo->waitId = 0; + unpublishWaiter(info->waitList); } - } - else - { - // Mesa-style re-block loop for fibers. SetEventFlag wakes ALL - // waiters including those that waited AND bits that were only - // partially satisfied; re-check the condition on every wake and - // re-publish to the wait-list if still unsatisfied. - for (;;) - { - // Update thread wait state, unless a ReleaseWaitThread already - // raced in since our last check. Status flips back to RUN/SUSPEND - // at the bottom of every iteration (below) and only becomes WAIT - // again right here, so in principle no legitimate forceRelease - // can be pending at this exact point (see WaitSema for the - // general hazard this guards against). Check-and-consume first - // anyway, defensively: if it is somehow already set, take the - // same released-exit path as the post-wake check below instead - // of blindly clearing it and re-parking. - bool release = false; - if (tInfo) - { - std::lock_guard tLock(tInfo->m); - if (tInfo->forceRelease) - { - tInfo->forceRelease = false; - release = true; - } - else - { - tInfo->status = (tInfo->suspendCount > 0) ? THS_WAITSUSPEND : THS_WAIT; - tInfo->waitType = TSW_EVENT; - tInfo->waitId = eid; - } - } - if (release) - { - ret = KE_RELEASE_WAIT; - break; - } - - // Publish under info->m; arm_park after unlock (no nested locks). - info->waiters++; - info->waitList.emplace_back(g_currentThreadId, ps2sched::current_fiber_token()); - - // UNLOCK before any scheduler operation. - lock.unlock(); - ps2sched::arm_park(); - const ps2sched::BlockResult br = ps2sched::block_current(); - if (br != ps2sched::BlockResult::Parked && - br != ps2sched::BlockResult::WokenInWindow) - { - std::fprintf(stderr, - "FATAL [WaitEventFlag]: unexpected NonFiber result in fiber Mesa loop\n"); - std::terminate(); - } - - lock.lock(); - - // Remove self from wait-list (re-added at top of loop if not satisfied). - { - auto &wl = info->waitList; - auto it = std::find_if(wl.begin(), wl.end(), - [](const std::pair& e){ return e.first == g_currentThreadId; }); - if (it != wl.end()) wl.erase(it); - info->waiters--; - } + info->waiters--; - if (tInfo) - { - std::lock_guard tLock(tInfo->m); - tInfo->status = (tInfo->suspendCount > 0) ? THS_SUSPEND : THS_RUN; - tInfo->waitType = TSW_NONE; - tInfo->waitId = 0; - if (tInfo->forceRelease) - { - tInfo->forceRelease = false; - ret = KE_RELEASE_WAIT; - } - } + // Wake reasons that abort the wait outright: + if (info->deleted) + { + ret = KE_WAIT_DELETE; + break; + } - if (tInfo && tInfo->terminated.load()) - { - throw ThreadExitException(); - } + if (consumeForceRelease(tInfo)) + { + ret = KE_RELEASE_WAIT; + break; + } - // Exit if forceRelease/terminate/delete broke us out, OR - // if the condition is now satisfied. - if (ret != KE_OK || info->deleted || satisfied()) - break; + if (tInfo && tInfo->terminated.load()) + { + throw ThreadExitException(); + } - // Spurious wake (SetEventFlag set only a subset of AND bits): - // loop back to re-publish and re-block. + if (satisfied()) + { + ret = KE_OK; + break; } + // Spurious wake (SetEventFlag set only a subset of AND bits): + // loop back to re-publish and re-block. } + + clearThreadWaiting(tInfo); } if (ret == KE_OK && info->deleted) From 2c37267882d0fb83561e29f4577603ca3dd05ec1 Mon Sep 17 00:00:00 2001 From: Shane Michael Mathews Date: Tue, 7 Jul 2026 19:32:20 -0400 Subject: [PATCH 09/15] test(sched): ParkedHostWorker/SemaPair RAII + drain/poll helpers in the workload regression suite --- ps2xTest/include/SchedTestSupport.h | 132 ++++++++ ...s2_scheduler_workload_regression_tests.cpp | 306 +++++------------- 2 files changed, 218 insertions(+), 220 deletions(-) diff --git a/ps2xTest/include/SchedTestSupport.h b/ps2xTest/include/SchedTestSupport.h index a5f8acbc3..db5e3af35 100644 --- a/ps2xTest/include/SchedTestSupport.h +++ b/ps2xTest/include/SchedTestSupport.h @@ -11,6 +11,7 @@ #include "ps2_syscalls.h" #include "runtime/ps2_memory.h" +#include #include #include #include @@ -30,6 +31,16 @@ namespace ps2x_test return static_cast(::getRegU32(&ctx, reg)); } + // Atomic 32-bit read from guest RAM. Lives here (rather than duplicated + // per test file) because the poll helpers below need it, and any test + // that only ever reads a control word can use it directly too. + inline uint32_t rdramRead32(const std::vector &rdram, uint32_t addr) + { + return std::atomic_ref( + *reinterpret_cast(const_cast(rdram.data()) + addr)) + .load(); + } + // Polls `pred` at 1ms intervals until it is true or `timeout` elapses, // always giving `pred` one final check at the deadline. template @@ -47,6 +58,52 @@ namespace ps2x_test return pred(); } + // Waits for the guest executor to fully quiesce (every dispatched thread + // has exited and g_activeThreads has settled back to <= 0). This is the + // teardown/drain poll almost every scheduler workload test ends on; a + // thin wrap of waitUntil so call sites collapse to one line without + // changing the wait semantics. + inline bool drainedWithin(std::chrono::milliseconds timeout) + { + return waitUntil([] + { + return g_activeThreads.load(std::memory_order_acquire) <= 0; + }, timeout); + } + + // Waits for a guest-RAM control word to reach an exact value, or a floor + // (AtLeast) — the recurring "poll a flag/counter a fiber writes" shape. + // Also thin wraps of waitUntil; no new timing behavior. + inline bool waitForWord(const std::vector &rdram, uint32_t addr, + uint32_t want, std::chrono::milliseconds timeout) + { + return waitUntil([&] + { + return rdramRead32(rdram, addr) == want; + }, timeout); + } + + inline bool waitForWordAtLeast(const std::vector &rdram, uint32_t addr, + uint32_t want, std::chrono::milliseconds timeout) + { + return waitUntil([&] + { + return rdramRead32(rdram, addr) >= want; + }, timeout); + } + + // Hands out a fresh, disjoint worker-stack base of `size` bytes on every + // call, so scheduler tests never need to hand-pick non-overlapping + // addresses to avoid a sibling fiber's stack in the same test. Bumped + // monotonically for the life of the process; each returned range is + // exactly [base, base + size), so distinct calls can never overlap + // regardless of call order. + inline uint32_t nextWorkerStackBase(uint32_t size) + { + static std::atomic next{0x00510000u}; + return next.fetch_add(size, std::memory_order_relaxed); + } + // Owns the runtime + guest RAM a scheduler test dispatches fibers against. // Construction clears residual thread/sema/handler state from a previous // test (notifyRuntimeStop) and brings up a fresh scheduler epoch @@ -72,4 +129,79 @@ namespace ps2x_test runtime.requestStop(); } }; + + // RAII pair of scheduler semaphores: constructs both up front (exposed + // via .a()/.b()) and deletes both on scope exit on every path — normal + // fall-through, an assertion failure, or an early return — with no + // hand-written guard/delete boilerplate at the call site. `create` / + // `destroy` are passed in (rather than named directly here) because each + // test file owns its own thin createSchedSema/deleteSchedSema wrappers + // around the CreateSema/DeleteSema syscalls: file-local test scaffolding, + // not part of the shared harness. + struct SemaPair + { + using CreateFn = int32_t (*)(uint8_t *, PS2Runtime *, int, int); + using DeleteFn = void (*)(uint8_t *, PS2Runtime *, int32_t); + + uint8_t *rdram; + PS2Runtime *runtime; + DeleteFn destroy; + int32_t sidA; + int32_t sidB; + + SemaPair(uint8_t *rdram_, PS2Runtime *runtime_, CreateFn create, DeleteFn destroy_, + int initA, int maxA, int initB, int maxB) + : rdram(rdram_), runtime(runtime_), destroy(destroy_), + sidA(create(rdram_, runtime_, initA, maxA)), + sidB(create(rdram_, runtime_, initB, maxB)) + { + } + + int32_t a() const { return sidA; } + int32_t b() const { return sidB; } + + ~SemaPair() + { + destroy(rdram, runtime, sidA); + destroy(rdram, runtime, sidB); + } + }; + + // RAII non-fiber host worker that parks in async_guest_begin() for the + // guest token (the interrupt-worker shape: g_currentThreadId == -1, so it + // borrows the token rather than holding a fiber's own). Replaces the + // hand-rolled atomic pair + std::thread duplicated across the + // token-handoff regression cases: a caller just needs to know when the + // worker won the token and when it finished; the destructor joins + // unconditionally so callers never have to remember to. + struct ParkedHostWorker + { + std::atomic acquired{false}, done{false}; + std::thread th; + ParkedHostWorker() : th([this] + { + g_currentThreadId = -1; + ps2sched::async_guest_begin(); + acquired.store(true, std::memory_order_release); + ps2sched::async_guest_end(); + done.store(true, std::memory_order_release); + }) + { + } + bool wonTokenWithin(std::chrono::milliseconds t) + { + return waitUntil([&] { return acquired.load(std::memory_order_acquire); }, t); + } + bool finishedWithin(std::chrono::milliseconds t) + { + return waitUntil([&] { return done.load(std::memory_order_acquire); }, t); + } + ~ParkedHostWorker() + { + if (th.joinable()) + { + th.join(); + } + } + }; } diff --git a/ps2xTest/src/ps2_scheduler_workload_regression_tests.cpp b/ps2xTest/src/ps2_scheduler_workload_regression_tests.cpp index 07c3c8899..cda796acd 100644 --- a/ps2xTest/src/ps2_scheduler_workload_regression_tests.cpp +++ b/ps2xTest/src/ps2_scheduler_workload_regression_tests.cpp @@ -58,10 +58,8 @@ namespace rdramWrite32Raw(rdram.data(), addr, val); } - uint32_t rdramRead32(const std::vector &rdram, uint32_t addr) - { - return rdramRead32Raw(rdram.data(), addr); - } + // rdramRead32 (vector-based) now lives in SchedTestSupport.h, shared with + // the poll helpers there (waitForWord/waitForWordAtLeast/drainedWithin). // Create a semaphore via CreateSema (EE layout: count, max_count, init_count). int32_t createSchedSema(uint8_t *rdram, PS2Runtime *runtime, int initCount, int maxCount) @@ -596,80 +594,54 @@ void register_scheduler_token_handoff_tests() runtime.registerFunction(0x00700000u, &stepPingA); runtime.registerFunction(0x00700100u, &stepPingB); - const int32_t sidX = createSchedSema(rdram.data(), &runtime, 1, 1); // seeded permit - const int32_t sidY = createSchedSema(rdram.data(), &runtime, 0, 1); - t.IsTrue(sidX > 0 && sidY > 0, "H1: semas created"); - if (sidX <= 0 || sidY <= 0) + SemaPair semas(rdram.data(), &runtime, createSchedSema, deleteSchedSema, + 1, 1, 0, 1); // sidX seeded permit, sidY empty + t.IsTrue(semas.a() > 0 && semas.b() > 0, "H1: semas created"); + if (semas.a() <= 0 || semas.b() <= 0) { return; } + const int32_t sidX = semas.a(); + const int32_t sidY = semas.b(); rdramWrite32(rdram, kThStop, 0u); rdramWrite32(rdram, kThSidX, static_cast(sidX)); rdramWrite32(rdram, kThSidY, static_cast(sidY)); rdramWrite32(rdram, kThCount, 0u); const int32_t tidA = startSchedWorker(rdram.data(), &runtime, - 0x00700000u, 10, 0x00510000u, 0x2000u); + 0x00700000u, 10, nextWorkerStackBase(0x2000u), 0x2000u); const int32_t tidB = startSchedWorker(rdram.data(), &runtime, - 0x00700100u, 10, 0x00514000u, 0x2000u); + 0x00700100u, 10, nextWorkerStackBase(0x2000u), 0x2000u); t.IsTrue(tidA > 0 && tidB > 0, "H1: ping-pong fibers started"); if (tidA <= 0 || tidB <= 0) { rdramWrite32(rdram, kThStop, 1u); signalSchedSema(rdram.data(), &runtime, sidX); signalSchedSema(rdram.data(), &runtime, sidY); - deleteSchedSema(rdram.data(), &runtime, sidX); - deleteSchedSema(rdram.data(), &runtime, sidY); return; } // Wait until the ping-pong is demonstrably live. - const bool spinning = waitUntil([&]() - { - return rdramRead32(rdram, kThCount) >= 3u; - }, std::chrono::milliseconds(2000)); + const bool spinning = waitForWordAtLeast(rdram, kThCount, 3u, std::chrono::milliseconds(2000)); t.IsTrue(spinning, "H1: ping-pong fibers are running"); // Host worker (interrupt-worker shape): parks for the guest token. - std::atomic tokenAcquired{false}; - std::atomic workerDone{false}; - std::thread worker([&]() - { - g_currentThreadId = -1; // non-fiber host worker - ps2sched::async_guest_begin(); - tokenAcquired.store(true, std::memory_order_release); - ps2sched::async_guest_end(); - workerDone.store(true, std::memory_order_release); - }); + ParkedHostWorker worker; // THE regression assertion: the worker must win the token while the // fibers are still ping-ponging (bounded wait — a starved worker // fails here instead of hanging the binary). - const bool acquiredWhileBusy = waitUntil([&]() - { - return tokenAcquired.load(std::memory_order_acquire); - }, std::chrono::milliseconds(3000)); + const bool acquiredWhileBusy = worker.wonTokenWithin(std::chrono::milliseconds(3000)); // Escape hatch + orderly teardown (runs regardless of the verdict: // once the fibers exit, the executor idles and the worker's - // async_guest_begin proceeds, so join() below cannot hang). + // async_guest_begin proceeds, so the destructor's join cannot hang). rdramWrite32(rdram, kThStop, 1u); signalSchedSema(rdram.data(), &runtime, sidX); signalSchedSema(rdram.data(), &runtime, sidY); - const bool drained = waitUntil([&]() - { - return g_activeThreads.load(std::memory_order_acquire) <= 0; - }, std::chrono::milliseconds(3000)); - - const bool workerFinished = waitUntil([&]() - { - return workerDone.load(std::memory_order_acquire); - }, std::chrono::milliseconds(3000)); - if (worker.joinable()) - { - worker.join(); - } + const bool drained = drainedWithin(std::chrono::milliseconds(3000)); + const bool workerFinished = worker.finishedWithin(std::chrono::milliseconds(3000)); t.IsTrue(acquiredWhileBusy, "H1: parked host worker acquired the guest token while fibers were busy " @@ -679,9 +651,6 @@ void register_scheduler_token_handoff_tests() const uint32_t loops = rdramRead32(rdram, kThCount); t.IsTrue(loops >= 3u, "H1: guest made progress during the test"); - - deleteSchedSema(rdram.data(), &runtime, sidX); - deleteSchedSema(rdram.data(), &runtime, sidY); }); tc.Run("H2: spinning fiber yields at yield_point when a host worker is parked", [](TestCase &t) { @@ -694,54 +663,29 @@ void register_scheduler_token_handoff_tests() rdramWrite32(rdram, kThCount, 0u); const int32_t tid = startSchedWorker(rdram.data(), &runtime, - 0x00700200u, 10, 0x00518000u, 0x2000u); + 0x00700200u, 10, nextWorkerStackBase(0x2000u), 0x2000u); t.IsTrue(tid > 0, "H2: spin fiber started"); if (tid <= 0) { return; } - const bool spinning = waitUntil([&]() - { - return rdramRead32(rdram, kThCount) >= 1000u; - }, std::chrono::milliseconds(2000)); + const bool spinning = waitForWordAtLeast(rdram, kThCount, 1000u, std::chrono::milliseconds(2000)); t.IsTrue(spinning, "H2: fiber is spinning through yield_point samples"); - std::atomic tokenAcquired{false}; - std::atomic workerDone{false}; - std::thread worker([&]() - { - g_currentThreadId = -1; // non-fiber host worker - ps2sched::async_guest_begin(); - tokenAcquired.store(true, std::memory_order_release); - ps2sched::async_guest_end(); - workerDone.store(true, std::memory_order_release); - }); + ParkedHostWorker worker; // THE regression assertion: without yield_point step 4 the fiber // never leaves ps2fiber_resume, so the worker cannot win the token // while the spin is live (bounded wait; escape hatch below). - const bool acquiredWhileSpinning = waitUntil([&]() - { - return tokenAcquired.load(std::memory_order_acquire); - }, std::chrono::milliseconds(3000)); + const bool acquiredWhileSpinning = worker.wonTokenWithin(std::chrono::milliseconds(3000)); // Escape hatch + teardown: end the spin; the fiber exits, the // executor idles, and the parked worker (if still parked) proceeds. rdramWrite32(rdram, kThStop, 1u); - const bool drained = waitUntil([&]() - { - return g_activeThreads.load(std::memory_order_acquire) <= 0; - }, std::chrono::milliseconds(3000)); - const bool workerFinished = waitUntil([&]() - { - return workerDone.load(std::memory_order_acquire); - }, std::chrono::milliseconds(3000)); - if (worker.joinable()) - { - worker.join(); - } + const bool drained = drainedWithin(std::chrono::milliseconds(3000)); + const bool workerFinished = worker.finishedWithin(std::chrono::milliseconds(3000)); t.IsTrue(acquiredWhileSpinning, "H2: parked host worker acquired the token while the fiber was spinning " @@ -761,52 +705,27 @@ void register_scheduler_token_handoff_tests() rdramWrite32(rdram, kThCount, 0u); const int32_t tid = startSchedWorker(rdram.data(), &runtime, - 0x00700300u, 10, 0x0051C000u, 0x2000u); + 0x00700300u, 10, nextWorkerStackBase(0x2000u), 0x2000u); t.IsTrue(tid > 0, "H3: cross-dispatch fiber started"); if (tid <= 0) { return; } - const bool spinning = waitUntil([&]() - { - return rdramRead32(rdram, kThCount) >= 1000u; - }, std::chrono::milliseconds(2000)); + const bool spinning = waitForWordAtLeast(rdram, kThCount, 1000u, std::chrono::milliseconds(2000)); t.IsTrue(spinning, "H3: fiber is spinning across function dispatches"); - std::atomic tokenAcquired{false}; - std::atomic workerDone{false}; - std::thread worker([&]() - { - g_currentThreadId = -1; // non-fiber host worker - ps2sched::async_guest_begin(); - tokenAcquired.store(true, std::memory_order_release); - ps2sched::async_guest_end(); - workerDone.store(true, std::memory_order_release); - }); + ParkedHostWorker worker; // THE regression assertion: the function bodies never call the // back-edge hook, so only the dispatch-loop preempt can yield. - const bool acquiredWhileSpinning = waitUntil([&]() - { - return tokenAcquired.load(std::memory_order_acquire); - }, std::chrono::milliseconds(3000)); + const bool acquiredWhileSpinning = worker.wonTokenWithin(std::chrono::milliseconds(3000)); // Escape hatch + teardown. rdramWrite32(rdram, kThStop, 1u); - const bool drained = waitUntil([&]() - { - return g_activeThreads.load(std::memory_order_acquire) <= 0; - }, std::chrono::milliseconds(3000)); - const bool workerFinished = waitUntil([&]() - { - return workerDone.load(std::memory_order_acquire); - }, std::chrono::milliseconds(3000)); - if (worker.joinable()) - { - worker.join(); - } + const bool drained = drainedWithin(std::chrono::milliseconds(3000)); + const bool workerFinished = worker.finishedWithin(std::chrono::milliseconds(3000)); t.IsTrue(acquiredWhileSpinning, "H3: parked host worker acquired the token during a cross-dispatch spin " @@ -841,11 +760,11 @@ void register_scheduler_rpc_loop_park_tests() // RPC server first (it runs first and, unfixed, never lets go). const int32_t serverTid = startSchedWorker(rdram.data(), &runtime, - 0x00700500u, 10, 0x00520000u, 0x2000u); + 0x00700500u, 10, nextWorkerStackBase(0x2000u), 0x2000u); // Probe fiber at the SAME priority: only a genuine park (not a // priority preempt) can let it run. const int32_t probeTid = startSchedWorker(rdram.data(), &runtime, - 0x00700600u, 10, 0x00524000u, 0x2000u); + 0x00700600u, 10, nextWorkerStackBase(0x2000u), 0x2000u); t.IsTrue(serverTid > 0 && probeTid > 0, "R1: both fibers started"); if (serverTid <= 0 || probeTid <= 0) { @@ -855,10 +774,7 @@ void register_scheduler_rpc_loop_park_tests() // THE regression assertion: the probe fiber runs within bounded // time, i.e. the RPC server fiber parked in SleepThread instead of // re-entering the stub forever. - const bool probeRan = waitUntil([&]() - { - return rdramRead32(rdram, kThFlag) == 1u; - }, std::chrono::milliseconds(3000)); + const bool probeRan = waitForWord(rdram, kThFlag, 1u, std::chrono::milliseconds(3000)); t.IsTrue(probeRan, "R1: equal-priority fiber ran while sceSifRpcLoop was active " @@ -906,7 +822,7 @@ void register_scheduler_guest_context_stop_tests() EnsureVSyncWorkerRunning(rdram.data(), &runtime); const int32_t tid = startSchedWorker(rdram.data(), &runtime, - 0x00700700u, 10, 0x00528000u, 0x2000u); + 0x00700700u, 10, nextWorkerStackBase(0x2000u), 0x2000u); t.IsTrue(tid > 0, "G1: fiber started"); if (tid <= 0) { @@ -917,10 +833,7 @@ void register_scheduler_guest_context_stop_tests() // calls requestStop() from guest context, and RETURNS (writing the // flag). Unfixed, notifyRuntimeStop joins the parked worker from // the fiber and wedges before the flag write. - const bool stopReturned = waitUntil([&]() - { - return rdramRead32(rdram, kThFlag) == 1u; - }, std::chrono::milliseconds(5000)); + const bool stopReturned = waitForWord(rdram, kThFlag, 1u, std::chrono::milliseconds(5000)); t.IsTrue(stopReturned, "G1: requestStop() invoked on the guest executor returned " @@ -936,10 +849,7 @@ void register_scheduler_guest_context_stop_tests() return; } - const bool drained = waitUntil([&]() - { - return g_activeThreads.load(std::memory_order_acquire) <= 0; - }, std::chrono::milliseconds(3000)); + const bool drained = drainedWithin(std::chrono::milliseconds(3000)); t.IsTrue(drained, "G1: fiber drained after guest-context stop"); // Main-thread shutdown performs the real joins (idempotent stops). @@ -1044,25 +954,19 @@ void register_scheduler_dmac_guest_dispatch_tests() // unconditional AsyncGuestScope this std::terminate()s the // process - a loud bounded failure, never a hang. const int32_t tid = startSchedWorker(rdram.data(), &runtime, - 0x00700900u, 10, 0x0052C000u, 0x2000u); + 0x00700900u, 10, nextWorkerStackBase(0x2000u), 0x2000u); t.IsTrue(tid > 0, "M1: dispatching fiber started"); if (tid <= 0) { return; } - const bool done = waitUntil([&]() - { - return rdramRead32(rdram, kThFlag) == 1u; - }, std::chrono::milliseconds(3000)); + const bool done = waitForWord(rdram, kThFlag, 1u, std::chrono::milliseconds(3000)); t.IsTrue(done, "M1: guest-context dispatch returned"); t.Equals(rdramRead32(rdram, kThSent), 1u, "M1: handler ran exactly once from guest context"); - const bool drained = waitUntil([&]() - { - return g_activeThreads.load(std::memory_order_acquire) <= 0; - }, std::chrono::milliseconds(3000)); + const bool drained = drainedWithin(std::chrono::milliseconds(3000)); t.IsTrue(drained, "M1: fiber drained"); // Host-thread path must still borrow the token and work: dispatch @@ -1123,13 +1027,15 @@ void register_scheduler_recovery_isolation_tests() // Both semas start at 0: A signals B then waits on A; B waits on B // then signals A - a one-shot ping-pong handoff, not a loop. - const int32_t sidA = createSchedSema(rdram.data(), &runtime, 0, 1); - const int32_t sidB = createSchedSema(rdram.data(), &runtime, 0, 1); - t.IsTrue(sidA > 0 && sidB > 0, "R1: semas created"); - if (sidA <= 0 || sidB <= 0) + SemaPair semas(rdram.data(), &runtime, createSchedSema, deleteSchedSema, + 0, 1, 0, 1); + t.IsTrue(semas.a() > 0 && semas.b() > 0, "R1: semas created"); + if (semas.a() <= 0 || semas.b() <= 0) { return; } + const int32_t sidA = semas.a(); + const int32_t sidB = semas.b(); rdramWrite32(rdram, kRiSidA, static_cast(sidA)); rdramWrite32(rdram, kRiSidB, static_cast(sidB)); rdramWrite32(rdram, kRiAForeign, 0u); @@ -1138,21 +1044,16 @@ void register_scheduler_recovery_isolation_tests() rdramWrite32(rdram, kRiBOwn, 0u); const int32_t tidA = startSchedWorker(rdram.data(), &runtime, - kEntryA, 10, 0x00530000u, 0x2000u); + kEntryA, 10, nextWorkerStackBase(0x2000u), 0x2000u); const int32_t tidB = startSchedWorker(rdram.data(), &runtime, - kEntryB, 10, 0x00534000u, 0x2000u); + kEntryB, 10, nextWorkerStackBase(0x2000u), 0x2000u); t.IsTrue(tidA > 0 && tidB > 0, "R1: both fibers started"); if (tidA <= 0 || tidB <= 0) { - deleteSchedSema(rdram.data(), &runtime, sidA); - deleteSchedSema(rdram.data(), &runtime, sidB); return; } - const bool drained = waitUntil([&]() - { - return g_activeThreads.load(std::memory_order_acquire) <= 0; - }, std::chrono::milliseconds(3000)); + const bool drained = drainedWithin(std::chrono::milliseconds(3000)); t.IsTrue(drained, "R1: both fibers completed the handoff and exited"); // THE regression assertions: under the shared thread_local, A and @@ -1166,9 +1067,6 @@ void register_scheduler_recovery_isolation_tests() "R1: fiber A's trace must contain fiber A's own marker"); t.Equals(rdramRead32(rdram, kRiBOwn), 1u, "R1: fiber B's trace must contain fiber B's own marker"); - - deleteSchedSema(rdram.data(), &runtime, sidA); - deleteSchedSema(rdram.data(), &runtime, sidB); }); tc.Run("R2: dispatch-trace ring is fresh after genuine tid reuse", [](TestCase &t) @@ -1199,10 +1097,7 @@ void register_scheduler_recovery_isolation_tests() return; } - const bool drained1 = waitUntil([&]() - { - return g_activeThreads.load(std::memory_order_acquire) <= 0; - }, std::chrono::milliseconds(3000)); + const bool drained1 = drainedWithin(std::chrono::milliseconds(3000)); t.IsTrue(drained1, "R2: fiber 1 fully torn down (FiberContext destroyed)"); // Force GENUINE tid reuse: erase T1 from the kernel thread map @@ -1225,10 +1120,7 @@ void register_scheduler_recovery_isolation_tests() kEntryReuse, 10, 0x0053C000u, 0x2000u); t.Equals(T2, T1, "R2: tid genuinely recycled (not merely a fresh, different tid)"); - const bool drained2 = waitUntil([&]() - { - return g_activeThreads.load(std::memory_order_acquire) <= 0; - }, std::chrono::milliseconds(3000)); + const bool drained2 = drainedWithin(std::chrono::milliseconds(3000)); t.IsTrue(drained2, "R2: fiber 2 (reused tid) completed"); // THE regression assertion: under the shared thread_local, fiber 2 @@ -1307,22 +1199,21 @@ void register_scheduler_stack_isolation_tests() { R5900Context s{}; setRegU32(s,4,kSysB); setRegU32(s,5,kInvB); ps2_syscalls::SetSyscall(rdram.data(), &s, &runtime); } - const int32_t sidA = createSchedSema(rdram.data(), &runtime, 0, 1); - const int32_t sidB = createSchedSema(rdram.data(), &runtime, 0, 1); - t.IsTrue(sidA > 0 && sidB > 0, "I1: semas created"); - rdramWrite32(rdram, kSiSemA, static_cast(sidA)); - rdramWrite32(rdram, kSiSemB, static_cast(sidB)); + SemaPair semas(rdram.data(), &runtime, createSchedSema, deleteSchedSema, + 0, 1, 0, 1); + t.IsTrue(semas.a() > 0 && semas.b() > 0, "I1: semas created"); + rdramWrite32(rdram, kSiSemA, static_cast(semas.a())); + rdramWrite32(rdram, kSiSemB, static_cast(semas.b())); rdramWrite32(rdram, kSiSysA, kSysA); rdramWrite32(rdram, kSiSysB, kSysB); rdramWrite32(rdram, kSiTopA, 0u); rdramWrite32(rdram, kSiTopB, 0u); - const int32_t tidA = startSchedWorker(rdram.data(), &runtime, kEntryA, 10, 0x00540000u, 0x2000u); - const int32_t tidB = startSchedWorker(rdram.data(), &runtime, kEntryB, 10, 0x00544000u, 0x2000u); + const int32_t tidA = startSchedWorker(rdram.data(), &runtime, kEntryA, 10, nextWorkerStackBase(0x2000u), 0x2000u); + const int32_t tidB = startSchedWorker(rdram.data(), &runtime, kEntryB, 10, nextWorkerStackBase(0x2000u), 0x2000u); t.IsTrue(tidA > 0 && tidB > 0, "I1: both fibers started"); - const bool drained = waitUntil([&]{ return g_activeThreads.load(std::memory_order_acquire) <= 0; }, - std::chrono::milliseconds(3000)); + const bool drained = drainedWithin(std::chrono::milliseconds(3000)); t.IsTrue(drained, "I1: both fibers completed the handoff and exited"); const uint32_t topA = rdramRead32(rdram, kSiTopA); @@ -1336,8 +1227,6 @@ void register_scheduler_stack_isolation_tests() ps2_syscalls::SetSyscall(rdram.data(), &s, &runtime); } { R5900Context s{}; setRegU32(s,4,kSysB); setRegU32(s,5,0u); ps2_syscalls::SetSyscall(rdram.data(), &s, &runtime); } - deleteSchedSema(rdram.data(), &runtime, sidA); - deleteSchedSema(rdram.data(), &runtime, sidB); }); // I2 --- B3: two fibers interleaving through inline DMAC dispatch must @@ -1364,19 +1253,18 @@ void register_scheduler_stack_isolation_tests() ps2_syscalls::AddDmacHandler(rdram.data(), &a, &runtime); hidB = getRegS32(a, 2); } t.IsTrue(hidA > 0 && hidB > 0, "I2: DMAC handlers registered"); - const int32_t sidA = createSchedSema(rdram.data(), &runtime, 0, 1); - const int32_t sidB = createSchedSema(rdram.data(), &runtime, 0, 1); - rdramWrite32(rdram, kSiSemA, static_cast(sidA)); - rdramWrite32(rdram, kSiSemB, static_cast(sidB)); + SemaPair semas(rdram.data(), &runtime, createSchedSema, deleteSchedSema, + 0, 1, 0, 1); + rdramWrite32(rdram, kSiSemA, static_cast(semas.a())); + rdramWrite32(rdram, kSiSemB, static_cast(semas.b())); rdramWrite32(rdram, kSiDmacTopA, 0u); rdramWrite32(rdram, kSiDmacTopB, 0u); - const int32_t tidA = startSchedWorker(rdram.data(), &runtime, kEntryA, 10, 0x00548000u, 0x2000u); - const int32_t tidB = startSchedWorker(rdram.data(), &runtime, kEntryB, 10, 0x0054C000u, 0x2000u); + const int32_t tidA = startSchedWorker(rdram.data(), &runtime, kEntryA, 10, nextWorkerStackBase(0x2000u), 0x2000u); + const int32_t tidB = startSchedWorker(rdram.data(), &runtime, kEntryB, 10, nextWorkerStackBase(0x2000u), 0x2000u); t.IsTrue(tidA > 0 && tidB > 0, "I2: both fibers started"); - const bool drained = waitUntil([&]{ return g_activeThreads.load(std::memory_order_acquire) <= 0; }, - std::chrono::milliseconds(3000)); + const bool drained = drainedWithin(std::chrono::milliseconds(3000)); t.IsTrue(drained, "I2: both fibers completed dispatch and exited"); const uint32_t topA = rdramRead32(rdram, kSiDmacTopA); @@ -1390,8 +1278,6 @@ void register_scheduler_stack_isolation_tests() ps2_syscalls::RemoveDmacHandler(rdram.data(), &r, &runtime); } { R5900Context r{}; setRegU32(r,4,6u); setRegU32(r,5,static_cast(hidB)); ps2_syscalls::RemoveDmacHandler(rdram.data(), &r, &runtime); } - deleteSchedSema(rdram.data(), &runtime, sidA); - deleteSchedSema(rdram.data(), &runtime, sidB); }); // I3 --- shared-mechanism contract (stands in for B4). dispatchGuest- @@ -1449,14 +1335,16 @@ void register_scheduler_override_isolation_tests() g_syscall_overrides[kOvSyscall] = kOvHandler; } - const int32_t sidStartB = createSchedSema(rdram.data(), &runtime, 0, 1); - const int32_t sidResumeA = createSchedSema(rdram.data(), &runtime, 0, 1); - t.IsTrue(sidStartB > 0 && sidResumeA > 0, "O1: semas created"); - if (sidStartB <= 0 || sidResumeA <= 0) + SemaPair semas(rdram.data(), &runtime, createSchedSema, deleteSchedSema, + 0, 1, 0, 1); + t.IsTrue(semas.a() > 0 && semas.b() > 0, "O1: semas created"); + if (semas.a() <= 0 || semas.b() <= 0) { { std::lock_guard lk(g_syscall_override_mutex); g_syscall_overrides.erase(kOvSyscall); } return; } + const int32_t sidStartB = semas.a(); + const int32_t sidResumeA = semas.b(); rdramWrite32(rdram, kOvSidStartB, static_cast(sidStartB)); rdramWrite32(rdram, kOvSidResumeA, static_cast(sidResumeA)); rdramWrite32(rdram, kOvAEntered, 0u); @@ -1464,22 +1352,17 @@ void register_scheduler_override_isolation_tests() rdramWrite32(rdram, kOvBRan, 0u); const int32_t tidA = startSchedWorker(rdram.data(), &runtime, - kOvEntryA, 10, 0x00550000u, 0x2000u); + kOvEntryA, 10, nextWorkerStackBase(0x2000u), 0x2000u); const int32_t tidB = startSchedWorker(rdram.data(), &runtime, - kOvEntryB, 10, 0x00554000u, 0x2000u); + kOvEntryB, 10, nextWorkerStackBase(0x2000u), 0x2000u); t.IsTrue(tidA > 0 && tidB > 0, "O1: both fibers started"); if (tidA <= 0 || tidB <= 0) { - deleteSchedSema(rdram.data(), &runtime, sidStartB); - deleteSchedSema(rdram.data(), &runtime, sidResumeA); { std::lock_guard lk(g_syscall_override_mutex); g_syscall_overrides.erase(kOvSyscall); } return; } - const bool drained = waitUntil([&]() - { - return g_activeThreads.load(std::memory_order_acquire) <= 0; - }, std::chrono::milliseconds(3000)); + const bool drained = drainedWithin(std::chrono::milliseconds(3000)); // A timeout here means the PARK mechanism failed (WaitSema nested in // the override invoke) — NOT the B2 guard. Debug the park, not the fix. t.IsTrue(drained, "O1: both fibers completed the handoff and exited"); @@ -1496,8 +1379,6 @@ void register_scheduler_override_isolation_tests() t.Equals(rdramRead32(rdram, kOvBRan), 1u, "O1: fiber B's own override must run while fiber A is parked inside the same syscall's override"); - deleteSchedSema(rdram.data(), &runtime, sidStartB); - deleteSchedSema(rdram.data(), &runtime, sidResumeA); { std::lock_guard lk(g_syscall_override_mutex); g_syscall_overrides.erase(kOvSyscall); } }); }); // MiniTest::Case("SchedulerOverrideIsolation") @@ -1556,15 +1437,15 @@ void register_scheduler_join_starvation_tests() runtime.registerFunction(kEntryJoiner, &stepJoinStarveJoiner); // One permit circulates X -> Y -> X, keeping the prio-10 level hot. - const int32_t sidX = createSchedSema(rdram.data(), &runtime, 1, 1); - const int32_t sidY = createSchedSema(rdram.data(), &runtime, 0, 1); - t.IsTrue(sidX > 0 && sidY > 0, "J1: ping-pong semas created"); - if (sidX <= 0 || sidY <= 0) + SemaPair semas(rdram.data(), &runtime, createSchedSema, deleteSchedSema, + 1, 1, 0, 1); // sidX seeded permit, sidY empty + t.IsTrue(semas.a() > 0 && semas.b() > 0, "J1: ping-pong semas created"); + if (semas.a() <= 0 || semas.b() <= 0) { - deleteSchedSema(rdram.data(), &runtime, sidX); - deleteSchedSema(rdram.data(), &runtime, sidY); return; } + const int32_t sidX = semas.a(); + const int32_t sidY = semas.b(); rdramWrite32(rdram, kThSidX, static_cast(sidX)); rdramWrite32(rdram, kThSidY, static_cast(sidY)); rdramWrite32(rdram, kThStop, 0u); @@ -1574,38 +1455,29 @@ void register_scheduler_join_starvation_tests() rdramWrite32(rdram, kJsTargetTid, 0u); // Pingers first (prio 10), then confirm the level is circulating. - const int32_t tidS1 = startSchedWorker(rdram.data(), &runtime, kEntryPingA, 10, 0x00560000u, 0x2000u); - const int32_t tidS2 = startSchedWorker(rdram.data(), &runtime, kEntryPingB, 10, 0x00562000u, 0x2000u); + const int32_t tidS1 = startSchedWorker(rdram.data(), &runtime, kEntryPingA, 10, nextWorkerStackBase(0x2000u), 0x2000u); + const int32_t tidS2 = startSchedWorker(rdram.data(), &runtime, kEntryPingB, 10, nextWorkerStackBase(0x2000u), 0x2000u); t.IsTrue(tidS1 > 0 && tidS2 > 0, "J1: ping-pong pair started"); - const bool pingLive = waitUntil([&]() - { - return rdramRead32(rdram, kThCount) >= 3u; - }, std::chrono::milliseconds(1000)); + const bool pingLive = waitForWordAtLeast(rdram, kThCount, 3u, std::chrono::milliseconds(1000)); t.IsTrue(pingLive, "J1: prio-10 ping-pong is circulating"); // Target B (prio 10) — never self-exits; killed by A's Terminate. - const int32_t tidB = startSchedWorker(rdram.data(), &runtime, kEntryTarget, 10, 0x00564000u, 0x2000u); + const int32_t tidB = startSchedWorker(rdram.data(), &runtime, kEntryTarget, 10, nextWorkerStackBase(0x2000u), 0x2000u); t.IsTrue(tidB > 0, "J1: target fiber started"); - const bool bRunning = waitUntil([&]() - { - return rdramRead32(rdram, kJsBStarted) == 1u; - }, std::chrono::milliseconds(1000)); + const bool bRunning = waitForWord(rdram, kJsBStarted, 1u, std::chrono::milliseconds(1000)); t.IsTrue(bRunning, "J1: target fiber is running"); // Joiner A (prio 5) terminates B; join_fiber floors A to B's level. rdramWrite32(rdram, kJsTargetTid, static_cast(tidB)); - const int32_t tidA = startSchedWorker(rdram.data(), &runtime, kEntryJoiner, 5, 0x00566000u, 0x2000u); + const int32_t tidA = startSchedWorker(rdram.data(), &runtime, kEntryJoiner, 5, nextWorkerStackBase(0x2000u), 0x2000u); t.IsTrue(tidA > 0, "J1: joiner fiber started"); // THE regression assertion: TerminateThread(B) returns within a // bounded number of scheduler rounds. Buggy (+1 floor): A is // stranded below the pingers and this never flips -> timeout -> // FAIL (binary still exits; teardown below quiesces everything). - const bool joinReturned = waitUntil([&]() - { - return rdramRead32(rdram, kJsJoinReturned) == 1u; - }, std::chrono::milliseconds(3000)); + const bool joinReturned = waitForWord(rdram, kJsJoinReturned, 1u, std::chrono::milliseconds(3000)); t.IsTrue(joinReturned, "J1: TerminateThread(target) returned — join floor must be the " "target's own priority level, not target+1"); @@ -1622,14 +1494,8 @@ void register_scheduler_join_starvation_tests() signalSchedSema(rdram.data(), &runtime, sidX); signalSchedSema(rdram.data(), &runtime, sidY); } - const bool drained = waitUntil([&]() - { - return g_activeThreads.load(std::memory_order_acquire) <= 0; - }, std::chrono::milliseconds(3000)); + const bool drained = drainedWithin(std::chrono::milliseconds(3000)); t.IsTrue(drained, "J1: all fibers quiesced after teardown"); - - deleteSchedSema(rdram.data(), &runtime, sidX); - deleteSchedSema(rdram.data(), &runtime, sidY); }); }); // MiniTest::Case("SchedulerJoinStarvation") } From f48242208ff90ee58c93b83a69fc27da2f67d6eb Mon Sep 17 00:00:00 2001 From: Shane Michael Mathews Date: Tue, 7 Jul 2026 19:58:38 -0400 Subject: [PATCH 10/15] test(sched): callSyscall marshalling helper + shared drain/rdram helpers in the expansion suite --- ps2xTest/include/SchedTestSupport.h | 25 + ps2xTest/src/ps2_runtime_expansion_tests.cpp | 694 +++++-------------- 2 files changed, 211 insertions(+), 508 deletions(-) diff --git a/ps2xTest/include/SchedTestSupport.h b/ps2xTest/include/SchedTestSupport.h index db5e3af35..22b109260 100644 --- a/ps2xTest/include/SchedTestSupport.h +++ b/ps2xTest/include/SchedTestSupport.h @@ -14,6 +14,7 @@ #include #include #include +#include #include #include @@ -92,6 +93,30 @@ namespace ps2x_test }, timeout); } + // Note: no shared rdramWrite32 is added here. ps2_runtime_expansion_tests.cpp + // already has a file-local rdramWrite32 (same signature/behavior), and + // ps2_scheduler_workload_regression_tests.cpp independently defines its own + // too; both already do `using namespace ps2x_test;`, so promoting either + // one to this header would make `rdramWrite32(...)` ambiguous at every + // call site outside its defining anonymous namespace. Each file keeps + // reusing its own existing local helper instead of duplicating a new one. + + // Marshals a 0-3-arg / reg4..6-in, reg2-out syscall call: binds up to + // three MIPS-ABI argument registers, invokes the syscall, and reads back + // its return value, collapsing the ctx/setRegU32/call/getRegS32 ritual + // duplicated across scheduler test bodies into a single call. + using SyscallFn = void (*)(uint8_t *, R5900Context *, PS2Runtime *); + inline int32_t callSyscall(PS2Runtime &rt, std::vector &ram, SyscallFn fn, + uint32_t a0, uint32_t a1 = 0, uint32_t a2 = 0) + { + R5900Context ctx{}; + setRegU32(ctx, 4, a0); + setRegU32(ctx, 5, a1); + setRegU32(ctx, 6, a2); + fn(ram.data(), &ctx, &rt); + return getRegS32(ctx, 2); + } + // Hands out a fresh, disjoint worker-stack base of `size` bytes on every // call, so scheduler tests never need to hand-pick non-overlapping // addresses to avoid a sibling fiber's stack in the same test. Bumped diff --git a/ps2xTest/src/ps2_runtime_expansion_tests.cpp b/ps2xTest/src/ps2_runtime_expansion_tests.cpp index 51c73548c..577f68698 100644 --- a/ps2xTest/src/ps2_runtime_expansion_tests.cpp +++ b/ps2xTest/src/ps2_runtime_expansion_tests.cpp @@ -496,10 +496,7 @@ void register_ps2_runtime_expansion_tests() t.IsTrue(tid > 0, "serialized-guest worker should start"); } - const bool allDone = waitUntil([&]() - { - return g_activeThreads.load(std::memory_order_relaxed) == 0; - }, std::chrono::milliseconds(2000)); + const bool allDone = drainedWithin(std::chrono::milliseconds(2000)); t.IsTrue(allDone, "all serialized-guest workers should finish"); t.Equals(gSerializedGuestActive.load(std::memory_order_acquire), 0, @@ -1612,17 +1609,10 @@ void register_ps2_runtime_expansion_tests() runtime.registerFunction(kEntry, &testRuntimeWorkerLoop); std::memcpy(rdram.data() + kThreadParamAddr, threadParam, sizeof(threadParam)); - R5900Context createCtx{}; - setRegU32(createCtx, 4, kThreadParamAddr); - CreateThread(rdram.data(), &createCtx, &runtime); - const int32_t tid = getRegS32(createCtx, 2); + const int32_t tid = callSyscall(runtime, rdram, ps2_syscalls::CreateThread, kThreadParamAddr); t.IsTrue(tid > 0, "CreateThread should succeed for teardown-join test"); - R5900Context startCtx{}; - setRegU32(startCtx, 4, static_cast(tid)); - setRegU32(startCtx, 5, 0u); - StartThread(rdram.data(), &startCtx, &runtime); - t.Equals(getRegS32(startCtx, 2), KE_OK, "StartThread should launch worker"); + t.Equals(callSyscall(runtime, rdram, ps2_syscalls::StartThread, static_cast(tid), 0u), KE_OK, "StartThread should launch worker"); const bool started = waitUntil([&]() { @@ -1650,10 +1640,7 @@ void register_ps2_runtime_expansion_tests() // not by anything under test. Generous headroom avoids false failures // when the host machine is under heavy load (e.g. running this whole // suite back-to-back many times in a stress loop). - const bool drained = waitUntil([&]() - { - return g_activeThreads.load(std::memory_order_acquire) == 0; - }, std::chrono::milliseconds(5000)); + const bool drained = drainedWithin(std::chrono::milliseconds(5000)); t.IsTrue(drained, "requestStop should drain all guest worker threads"); ps2sched::scheduler_shutdown(); @@ -1677,10 +1664,7 @@ void register_ps2_runtime_expansion_tests() }; std::memcpy(rdram.data() + kParamAddr, semaParam, sizeof(semaParam)); - R5900Context createCtx{}; - setRegU32(createCtx, 4, kParamAddr); - CreateSema(rdram.data(), &createCtx, &runtime); - const int32_t sid = getRegS32(createCtx, 2); + const int32_t sid = callSyscall(runtime, rdram, ps2_syscalls::CreateSema, kParamAddr); t.IsTrue(sid > 0, "CreateSema should return a valid sid"); std::atomic pollOkCount{0}; @@ -1694,10 +1678,7 @@ void register_ps2_runtime_expansion_tests() { for (int i = 0; i < 64; ++i) { - R5900Context pollCtx{}; - setRegU32(pollCtx, 4, static_cast(sid)); - PollSema(rdram.data(), &pollCtx, &runtime); - if (getRegS32(pollCtx, 2) == sid) + if (callSyscall(runtime, rdram, ps2_syscalls::PollSema, static_cast(sid)) == sid) { pollOkCount.fetch_add(1, std::memory_order_relaxed); } @@ -1715,10 +1696,7 @@ void register_ps2_runtime_expansion_tests() { for (int i = 0; i < 64; ++i) { - R5900Context signalCtx{}; - setRegU32(signalCtx, 4, static_cast(sid)); - SignalSema(rdram.data(), &signalCtx, &runtime); - if (getRegS32(signalCtx, 2) == sid) + if (callSyscall(runtime, rdram, ps2_syscalls::SignalSema, static_cast(sid)) == sid) { signalOkCount.fetch_add(1, std::memory_order_relaxed); } @@ -1749,11 +1727,7 @@ void register_ps2_runtime_expansion_tests() "contended SignalSema should observe successful releases"); constexpr uint32_t kStatusAddr = 0x2100u; - R5900Context referCtx{}; - setRegU32(referCtx, 4, static_cast(sid)); - setRegU32(referCtx, 5, kStatusAddr); - ReferSemaStatus(rdram.data(), &referCtx, &runtime); - t.Equals(getRegS32(referCtx, 2), KE_OK, "ReferSemaStatus should succeed after contention"); + t.Equals(callSyscall(runtime, rdram, ps2_syscalls::ReferSemaStatus, static_cast(sid), kStatusAddr), KE_OK, "ReferSemaStatus should succeed after contention"); int32_t finalCount = 0; std::memcpy(&finalCount, rdram.data() + kStatusAddr + 0u, sizeof(finalCount)); @@ -1773,10 +1747,7 @@ void register_ps2_runtime_expansion_tests() const uint32_t eventParam[3] = {0u, 0u, 0u}; std::memcpy(rdram.data() + kEventParamAddr, eventParam, sizeof(eventParam)); - R5900Context createCtx{}; - setRegU32(createCtx, 4, kEventParamAddr); - CreateEventFlag(rdram.data(), &createCtx, &runtime); - const int32_t eid = getRegS32(createCtx, 2); + const int32_t eid = callSyscall(runtime, rdram, ps2_syscalls::CreateEventFlag, kEventParamAddr); t.IsTrue(eid > 0, "CreateEventFlag should return a valid id"); std::atomic waiterDone{false}; @@ -1813,10 +1784,7 @@ void register_ps2_runtime_expansion_tests() try { std::this_thread::sleep_for(std::chrono::milliseconds(10)); - R5900Context setCtx{}; - setRegU32(setCtx, 4, static_cast(eid)); - setRegU32(setCtx, 5, 0x1u); - SetEventFlag(rdram.data(), &setCtx, &runtime); + callSyscall(runtime, rdram, ps2_syscalls::SetEventFlag, static_cast(eid), 0x1u); } catch (...) { @@ -1829,10 +1797,7 @@ void register_ps2_runtime_expansion_tests() try { std::this_thread::sleep_for(std::chrono::milliseconds(15)); - R5900Context setCtx{}; - setRegU32(setCtx, 4, static_cast(eid)); - setRegU32(setCtx, 5, 0x2u); - SetEventFlag(rdram.data(), &setCtx, &runtime); + callSyscall(runtime, rdram, ps2_syscalls::SetEventFlag, static_cast(eid), 0x2u); } catch (...) { @@ -1869,9 +1834,7 @@ void register_ps2_runtime_expansion_tests() t.IsTrue((waiterBits.load(std::memory_order_relaxed) & 0x3u) == 0x3u, "WaitEventFlag result bits should include both concurrently-set bits"); - R5900Context deleteCtx{}; - setRegU32(deleteCtx, 4, static_cast(eid)); - DeleteEventFlag(rdram.data(), &deleteCtx, &runtime); + callSyscall(runtime, rdram, ps2_syscalls::DeleteEventFlag, static_cast(eid)); runtime.requestStop(); }); @@ -2088,10 +2051,7 @@ void register_scheduler_tests() const bool workerBlocked = waitUntil([&]() { constexpr uint32_t kStatusAddr = 0x2C00u; - R5900Context refCtx{}; - setRegU32(refCtx, 4, static_cast(sid)); - setRegU32(refCtx, 5, kStatusAddr); - ps2_syscalls::ReferSemaStatus(rdram.data(), &refCtx, &runtime); + callSyscall(runtime, rdram, ps2_syscalls::ReferSemaStatus, static_cast(sid), kStatusAddr); int32_t waitThreads = 0; std::memcpy(&waitThreads, rdram.data() + kStatusAddr + 12u, sizeof(waitThreads)); return waitThreads >= 1; @@ -2101,20 +2061,14 @@ void register_scheduler_tests() // Signal sid from the host thread — the pool will pick up the worker fiber. { - R5900Context sigCtx{}; - setRegU32(sigCtx, 4, static_cast(sid)); - ps2_syscalls::SignalSema(rdram.data(), &sigCtx, &runtime); - t.Equals(getRegS32(sigCtx, 2), sid, "WaitSema test: SignalSema(sid) should return sid"); + t.Equals(callSyscall(runtime, rdram, ps2_syscalls::SignalSema, static_cast(sid)), sid, "WaitSema test: SignalSema(sid) should return sid"); } // Wait until the worker signals done_sid (i.e., its WaitSema returned). const bool workerDone = waitUntil([&]() { constexpr uint32_t kStatusAddr = 0x2C80u; - R5900Context refCtx{}; - setRegU32(refCtx, 4, static_cast(doneSid)); - setRegU32(refCtx, 5, kStatusAddr); - ps2_syscalls::ReferSemaStatus(rdram.data(), &refCtx, &runtime); + callSyscall(runtime, rdram, ps2_syscalls::ReferSemaStatus, static_cast(doneSid), kStatusAddr); int32_t count = 0; std::memcpy(&count, rdram.data() + kStatusAddr + 0u, sizeof(count)); // current count field (offset 0 in ee_sema_t) return count >= 1; @@ -2130,10 +2084,7 @@ void register_scheduler_tests() } // Wait for the worker fiber to finish. - const bool finished = waitUntil([&]() - { - return g_activeThreads.load(std::memory_order_acquire) <= 0; - }, std::chrono::milliseconds(1000)); + const bool finished = drainedWithin(std::chrono::milliseconds(1000)); t.IsTrue(finished, "WaitSema test: worker fiber should finish within 1s"); deleteSchedSema(rdram.data(), &runtime, sid); @@ -2178,10 +2129,7 @@ void register_scheduler_tests() const bool workerBlocked = waitUntil([&]() { constexpr uint32_t kStatusAddr = 0x2C00u; - R5900Context refCtx{}; - setRegU32(refCtx, 4, static_cast(blockSid)); - setRegU32(refCtx, 5, kStatusAddr); - ps2_syscalls::ReferSemaStatus(rdram.data(), &refCtx, &runtime); + callSyscall(runtime, rdram, ps2_syscalls::ReferSemaStatus, static_cast(blockSid), kStatusAddr); int32_t waitThreads = 0; std::memcpy(&waitThreads, rdram.data() + kStatusAddr + 12u, sizeof(waitThreads)); return waitThreads >= 1; @@ -2193,10 +2141,7 @@ void register_scheduler_tests() // Terminate the worker fiber. { - R5900Context termCtx{}; - setRegU32(termCtx, 4, static_cast(workerTid)); - ps2_syscalls::TerminateThread(rdram.data(), &termCtx, &runtime); - t.Equals(getRegS32(termCtx, 2), KE_OK, "TerminateThread should return KE_OK"); + t.Equals(callSyscall(runtime, rdram, ps2_syscalls::TerminateThread, static_cast(workerTid)), KE_OK, "TerminateThread should return KE_OK"); } // g_activeThreads should decrement as the fiber unwinds. @@ -2262,10 +2207,7 @@ void register_scheduler_tests() const bool allBlocked = waitUntil([&]() { constexpr uint32_t kStatusAddr = 0x2D00u; - R5900Context refCtx{}; - setRegU32(refCtx, 4, static_cast(sid)); - setRegU32(refCtx, 5, kStatusAddr); - ps2_syscalls::ReferSemaStatus(rdram.data(), &refCtx, &runtime); + callSyscall(runtime, rdram, ps2_syscalls::ReferSemaStatus, static_cast(sid), kStatusAddr); int32_t waiters = 0; std::memcpy(&waiters, rdram.data() + kStatusAddr + 12u, sizeof(waiters)); return waiters >= kNumWorkers; @@ -2275,10 +2217,7 @@ void register_scheduler_tests() // DeleteSema: marks deleted, wakes all N blocked fibers via make_ready. { - R5900Context delCtx{}; - setRegU32(delCtx, 4, static_cast(sid)); - ps2_syscalls::DeleteSema(rdram.data(), &delCtx, &runtime); - t.Equals(getRegS32(delCtx, 2), sid, "DeleteSema should return sid"); + t.Equals(callSyscall(runtime, rdram, ps2_syscalls::DeleteSema, static_cast(sid)), sid, "DeleteSema should return sid"); } // Wait until all N workers have signalled done_sid (count == N). @@ -2303,18 +2242,12 @@ void register_scheduler_tests() // PollSema on the deleted id should return KE_UNKNOWN_SEMID. { - R5900Context laterCtx{}; - setRegU32(laterCtx, 4, static_cast(sid)); - ps2_syscalls::PollSema(rdram.data(), &laterCtx, &runtime); - t.Equals(getRegS32(laterCtx, 2), KE_UNKNOWN_SEMID, + t.Equals(callSyscall(runtime, rdram, ps2_syscalls::PollSema, static_cast(sid)), KE_UNKNOWN_SEMID, "PollSema on a deleted sema id should return KE_UNKNOWN_SEMID"); } // Wait for all fibers to finish and clean up. - const bool finished = waitUntil([&]() - { - return g_activeThreads.load(std::memory_order_acquire) <= 0; - }, std::chrono::milliseconds(2000)); + const bool finished = drainedWithin(std::chrono::milliseconds(2000)); t.IsTrue(finished, "DeleteSema test: all fibers should finish within 2s"); deleteSchedSema(rdram.data(), &runtime, doneSid); @@ -2734,10 +2667,7 @@ void register_scheduler_protocol_tests() return; } - { - int32_t v = goSid; - std::memcpy(rdram.data() + kSlotGoSid, &v, 4); - } + rdramWrite32(rdram, kSlotGoSid, static_cast(goSid)); rdramSeqReset(rdram); std::memset(rdram.data() + kRunLog, 0, 32u); @@ -2767,8 +2697,7 @@ void register_scheduler_protocol_tests() ps2sched::async_guest_begin(); for (int i = 0; i < 3; ++i) { - R5900Context sc{}; setRegU32(sc, 4, static_cast(goSid)); - ps2_syscalls::SignalSema(rdram.data(), &sc, &runtime); + callSyscall(runtime, rdram, ps2_syscalls::SignalSema, static_cast(goSid)); } ps2sched::async_guest_end(); @@ -2781,7 +2710,7 @@ void register_scheduler_protocol_tests() t.Equals(log[1], tidB, "T1: prio 10 runs second"); t.Equals(log[2], tidC, "T1: prio 20 runs last"); - waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + drainedWithin(std::chrono::milliseconds(1000)); deleteSchedSema(rdram.data(), &runtime, goSid); }); @@ -2803,10 +2732,7 @@ void register_scheduler_protocol_tests() return; } - { - int32_t v = goSid; - std::memcpy(rdram.data() + kSlotGoSid, &v, 4); - } + rdramWrite32(rdram, kSlotGoSid, static_cast(goSid)); rdramSeqReset(rdram); std::memset(rdram.data() + kRunLog, 0, 16u); @@ -2826,8 +2752,7 @@ void register_scheduler_protocol_tests() ps2sched::async_guest_begin(); for (int i = 0; i < 2; ++i) { - R5900Context sc{}; setRegU32(sc, 4, static_cast(goSid)); - ps2_syscalls::SignalSema(rdram.data(), &sc, &runtime); + callSyscall(runtime, rdram, ps2_syscalls::SignalSema, static_cast(goSid)); } ps2sched::async_guest_end(); @@ -2839,7 +2764,7 @@ void register_scheduler_protocol_tests() t.Equals(log[0], tidA, "T2: A (created first) runs first"); t.Equals(log[1], tidB, "T2: B (created second) runs second"); - waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + drainedWithin(std::chrono::milliseconds(1000)); deleteSchedSema(rdram.data(), &runtime, goSid); }); @@ -2867,8 +2792,7 @@ void register_scheduler_protocol_tests() // Rotate [A,B,C] -> [B,C,A] while executor is locked out { - R5900Context sc{}; setRegU32(sc, 4, 10u); - ps2_syscalls::RotateThreadReadyQueue(rdram.data(), &sc, &runtime); + callSyscall(runtime, rdram, ps2_syscalls::RotateThreadReadyQueue, 10u); } // Release executor @@ -2883,7 +2807,7 @@ void register_scheduler_protocol_tests() t.Equals(log[1], tidC, "T3: C runs second after rotate"); t.Equals(log[2], tidA, "T3: A runs last (moved to tail)"); - waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + drainedWithin(std::chrono::milliseconds(1000)); }); // ------------------------------------------------------------------ @@ -2910,10 +2834,7 @@ void register_scheduler_protocol_tests() const int32_t tidY = startSchedWorker(rdram.data(), &runtime, 0x00630000u, 10, 0x00530000u, 0x2000u); t.IsTrue(tidX > 0, "T4: X started"); t.IsTrue(tidY > 0, "T4: Y started"); - { - int32_t v = tidX; - std::memcpy(rdram.data() + kSlotTidParam, &v, 4); - } + rdramWrite32(rdram, kSlotTidParam, static_cast(tidX)); // Release executor: Y runs first (prio 10), calls ChangeThreadPriority(X, 5), // X bumped to prio=5 (higher than Y=10), Y yields 500x -> X preempts @@ -2924,7 +2845,7 @@ void register_scheduler_protocol_tests() t.IsTrue(allDone, "T4: both fibers completed"); // Wait for both fibers to exit naturally. - waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + drainedWithin(std::chrono::milliseconds(1000)); int32_t log[2] = {}; std::memcpy(log, rdram.data() + kRunLog, 8); @@ -2963,9 +2884,7 @@ void register_scheduler_protocol_tests() t.IsTrue(started, "T5: fiber started"); { - R5900Context sc{}; setRegU32(sc, 4, static_cast(tid)); - ps2_syscalls::SuspendThread(rdram.data(), &sc, &runtime); - t.Equals(getRegS32(sc, 2), KE_OK, "T5: SuspendThread returned KE_OK"); + t.Equals(callSyscall(runtime, rdram, ps2_syscalls::SuspendThread, static_cast(tid)), KE_OK, "T5: SuspendThread returned KE_OK"); } // Wait for THS_SUSPEND status AND for the progress counter to stop @@ -2992,9 +2911,7 @@ void register_scheduler_protocol_tests() t.Equals(cBefore, cAfter, "T5: counter not advancing while suspended"); { - R5900Context sc{}; setRegU32(sc, 4, static_cast(tid)); - ps2_syscalls::ResumeThread(rdram.data(), &sc, &runtime); - t.Equals(getRegS32(sc, 2), KE_OK, "T5: ResumeThread returned KE_OK"); + t.Equals(callSyscall(runtime, rdram, ps2_syscalls::ResumeThread, static_cast(tid)), KE_OK, "T5: ResumeThread returned KE_OK"); } const bool advanced = waitUntil([&]() @@ -3004,7 +2921,7 @@ void register_scheduler_protocol_tests() t.IsTrue(advanced, "T5: progress resumes after ResumeThread"); gStopFlag.store(1u, std::memory_order_release); - waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + drainedWithin(std::chrono::milliseconds(1000)); }); // ------------------------------------------------------------------ @@ -3029,15 +2946,11 @@ void register_scheduler_protocol_tests() } { - int32_t w = workSid, d = doneSid; - std::memcpy(rdram.data() + kSlotWorkSid, &w, 4); - std::memcpy(rdram.data() + kSlotDoneSid, &d, 4); + rdramWrite32(rdram, kSlotWorkSid, static_cast(workSid)); + rdramWrite32(rdram, kSlotDoneSid, static_cast(doneSid)); } rdramSeqReset(rdram); - { - int32_t bad = -9999; - std::memcpy(rdram.data() + kResultBase, &bad, 4); - } + rdramWrite32(rdram, kResultBase, static_cast(-9999)); const int32_t tid = startSchedWorker(rdram.data(), &runtime, 0x00650000u, 10, 0x00550000u, 0x2000u); t.IsTrue(tid > 0, "T6: fiber started"); @@ -3049,9 +2962,7 @@ void register_scheduler_protocol_tests() t.IsTrue(blocked, "T6: fiber blocked on workSid"); { - R5900Context sc{}; setRegU32(sc, 4, static_cast(tid)); - ps2_syscalls::SuspendThread(rdram.data(), &sc, &runtime); - t.Equals(getRegS32(sc, 2), KE_OK, "T6: SuspendThread returned KE_OK"); + t.Equals(callSyscall(runtime, rdram, ps2_syscalls::SuspendThread, static_cast(tid)), KE_OK, "T6: SuspendThread returned KE_OK"); } const bool isSuspended = waitUntil([&]() @@ -3063,9 +2974,7 @@ void register_scheduler_protocol_tests() t.IsTrue(isSuspended, "T6: fiber reached THS_WAITSUSPEND"); { - R5900Context sc{}; setRegU32(sc, 4, static_cast(workSid)); - ps2_syscalls::SignalSema(rdram.data(), &sc, &runtime); - t.Equals(getRegS32(sc, 2), workSid, "T6: SignalSema returned workSid"); + t.Equals(callSyscall(runtime, rdram, ps2_syscalls::SignalSema, static_cast(workSid)), workSid, "T6: SignalSema returned workSid"); } // Wait to confirm fiber did NOT consume it @@ -3074,9 +2983,7 @@ void register_scheduler_protocol_tests() t.Equals(getSemaCount(rdram, &runtime, workSid), 1, "T6: permit not consumed while suspended"); { - R5900Context sc{}; setRegU32(sc, 4, static_cast(tid)); - ps2_syscalls::ResumeThread(rdram.data(), &sc, &runtime); - t.Equals(getRegS32(sc, 2), KE_OK, "T6: ResumeThread returned KE_OK"); + t.Equals(callSyscall(runtime, rdram, ps2_syscalls::ResumeThread, static_cast(tid)), KE_OK, "T6: ResumeThread returned KE_OK"); } const bool woke = waitUntil([&](){ return rdramSeq(rdram) >= 1u; }, std::chrono::milliseconds(1000)); @@ -3087,7 +2994,7 @@ void register_scheduler_protocol_tests() t.Equals(ret, workSid, "T6: WaitSema returned workSid after resume"); } - waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + drainedWithin(std::chrono::milliseconds(1000)); deleteSchedSema(rdram.data(), &runtime, workSid); deleteSchedSema(rdram.data(), &runtime, doneSid); }); @@ -3114,15 +3021,11 @@ void register_scheduler_protocol_tests() } { - int32_t w = workSid, d = doneSid; - std::memcpy(rdram.data() + kSlotWorkSid, &w, 4); - std::memcpy(rdram.data() + kSlotDoneSid, &d, 4); + rdramWrite32(rdram, kSlotWorkSid, static_cast(workSid)); + rdramWrite32(rdram, kSlotDoneSid, static_cast(doneSid)); } rdramSeqReset(rdram); - { - int32_t bad = -9999; - std::memcpy(rdram.data() + kResultBase, &bad, 4); - } + rdramWrite32(rdram, kResultBase, static_cast(-9999)); const int32_t tid = startSchedWorker(rdram.data(), &runtime, 0x00660000u, 10, 0x00560000u, 0x2000u); t.IsTrue(tid > 0, "T7: waiter fiber started"); @@ -3143,16 +3046,13 @@ void register_scheduler_protocol_tests() { for (int i = 0; i < 200 && pollOK.load() == 0; ++i) { - R5900Context sc{}; setRegU32(sc, 4, static_cast(workSid)); - ps2_syscalls::PollSema(rdram.data(), &sc, &runtime); - if (getRegS32(sc, 2) == workSid) + if (callSyscall(runtime, rdram, ps2_syscalls::PollSema, static_cast(workSid)) == workSid) pollOK.store(1); } }); { - R5900Context sc{}; setRegU32(sc, 4, static_cast(workSid)); - ps2_syscalls::SignalSema(rdram.data(), &sc, &runtime); + callSyscall(runtime, rdram, ps2_syscalls::SignalSema, static_cast(workSid)); } poller.join(); @@ -3177,9 +3077,7 @@ void register_scheduler_protocol_tests() while (releaseRet != KE_OK && std::chrono::steady_clock::now() < releaseDeadline) { - R5900Context sc{}; setRegU32(sc, 4, static_cast(tid)); - ps2_syscalls::ReleaseWaitThread(rdram.data(), &sc, &runtime); - releaseRet = getRegS32(sc, 2); + releaseRet = callSyscall(runtime, rdram, ps2_syscalls::ReleaseWaitThread, static_cast(tid)); if (releaseRet != KE_OK) std::this_thread::sleep_for(std::chrono::milliseconds(1)); } @@ -3199,7 +3097,7 @@ void register_scheduler_protocol_tests() t.IsTrue(getSemaCount(rdram, &runtime, workSid) >= 0, "T7: sema count never negative"); t.Equals(getSemaCount(rdram, &runtime, workSid), 0, "T7: permit fully consumed, none left over"); - waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + drainedWithin(std::chrono::milliseconds(1000)); deleteSchedSema(rdram.data(), &runtime, workSid); deleteSchedSema(rdram.data(), &runtime, doneSid); }); @@ -3237,7 +3135,7 @@ void register_scheduler_protocol_tests() t.IsTrue(started, "T8: fiber starts after async_guest_end"); gStopFlag.store(1u, std::memory_order_release); - waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + drainedWithin(std::chrono::milliseconds(1000)); }); // ------------------------------------------------------------------ @@ -3275,7 +3173,7 @@ void register_scheduler_protocol_tests() t.IsTrue(started, "T9: fiber starts after RAII scope released token on exception"); gStopFlag.store(1u, std::memory_order_release); - waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + drainedWithin(std::chrono::milliseconds(1000)); }); // ------------------------------------------------------------------ @@ -3301,10 +3199,7 @@ void register_scheduler_protocol_tests() return; } - { - int32_t d = doneSid; - std::memcpy(rdram.data() + kSlotDoneSid, &d, 4); - } + rdramWrite32(rdram, kSlotDoneSid, static_cast(doneSid)); rdramSeqReset(rdram); const int32_t tid = startSchedWorker(rdram.data(), &runtime, 0x00690000u, 10, 0x00590000u, 0x2000u); @@ -3322,9 +3217,7 @@ void register_scheduler_protocol_tests() // from an ISR). This sets wakeupCount++ and unblocks the fiber. std::thread foreign([&]() { - R5900Context sc{}; - setRegU32(sc, 4, static_cast(tid)); - ps2_syscalls::WakeupThread(rdram.data(), &sc, &runtime); + callSyscall(runtime, rdram, ps2_syscalls::WakeupThread, static_cast(tid)); }); foreign.join(); @@ -3336,7 +3229,7 @@ void register_scheduler_protocol_tests() t.Equals(log0, tid, "T10: correct fiber woke"); } - waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + drainedWithin(std::chrono::milliseconds(1000)); deleteSchedSema(rdram.data(), &runtime, doneSid); }); @@ -3358,10 +3251,7 @@ void register_scheduler_protocol_tests() return; } - { - int32_t d = doneSid; - std::memcpy(rdram.data() + kSlotDoneSid, &d, 4); - } + rdramWrite32(rdram, kSlotDoneSid, static_cast(doneSid)); rdramSeqReset(rdram); gT11TokenLo.store(0u, std::memory_order_release); gT11TokenHi.store(0u, std::memory_order_release); @@ -3387,9 +3277,7 @@ void register_scheduler_protocol_tests() static_cast(gT11TokenLo.load(std::memory_order_acquire)); { - R5900Context sc{}; setRegU32(sc, 4, static_cast(tid)); - ps2_syscalls::SuspendThread(rdram.data(), &sc, &runtime); - t.Equals(getRegS32(sc, 2), KE_OK, "T11: SuspendThread returned KE_OK"); + t.Equals(callSyscall(runtime, rdram, ps2_syscalls::SuspendThread, static_cast(tid)), KE_OK, "T11: SuspendThread returned KE_OK"); } const bool isSuspended = waitUntil([&]() @@ -3413,9 +3301,7 @@ void register_scheduler_protocol_tests() } { - R5900Context sc{}; setRegU32(sc, 4, static_cast(tid)); - ps2_syscalls::ResumeThread(rdram.data(), &sc, &runtime); - t.Equals(getRegS32(sc, 2), KE_OK, "T11: ResumeThread returned KE_OK"); + t.Equals(callSyscall(runtime, rdram, ps2_syscalls::ResumeThread, static_cast(tid)), KE_OK, "T11: ResumeThread returned KE_OK"); } // After ResumeThread the fiber should RE-PARK inside SleepThread @@ -3432,15 +3318,13 @@ void register_scheduler_protocol_tests() // A genuine WakeupThread must now release the fiber. { - R5900Context sc{}; setRegU32(sc, 4, static_cast(tid)); - ps2_syscalls::WakeupThread(rdram.data(), &sc, &runtime); - t.Equals(getRegS32(sc, 2), KE_OK, "T11: WakeupThread returned KE_OK"); + t.Equals(callSyscall(runtime, rdram, ps2_syscalls::WakeupThread, static_cast(tid)), KE_OK, "T11: WakeupThread returned KE_OK"); } const bool woke = waitUntil([&](){ return rdramSeq(rdram) >= 1u; }, std::chrono::milliseconds(1000)); t.IsTrue(woke, "T11: fiber woke after WakeupThread"); - waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + drainedWithin(std::chrono::milliseconds(1000)); deleteSchedSema(rdram.data(), &runtime, doneSid); }); @@ -3466,15 +3350,11 @@ void register_scheduler_protocol_tests() } { - int32_t w = workSid, d = doneSid; - std::memcpy(rdram.data() + kSlotWorkSid, &w, 4); - std::memcpy(rdram.data() + kSlotDoneSid, &d, 4); + rdramWrite32(rdram, kSlotWorkSid, static_cast(workSid)); + rdramWrite32(rdram, kSlotDoneSid, static_cast(doneSid)); } rdramSeqReset(rdram); - { - int32_t bad = -9999; - std::memcpy(rdram.data() + kResultBase, &bad, 4); - } + rdramWrite32(rdram, kResultBase, static_cast(-9999)); const int32_t tid = startSchedWorker(rdram.data(), &runtime, 0x006B0000u, 10, 0x005B0000u, 0x2000u); t.IsTrue(tid > 0, "T12: fiber started"); @@ -3486,8 +3366,7 @@ void register_scheduler_protocol_tests() t.IsTrue(blocked, "T12: fiber blocked on workSid"); { - R5900Context sc{}; setRegU32(sc, 4, static_cast(workSid)); - ps2_syscalls::SignalSema(rdram.data(), &sc, &runtime); + callSyscall(runtime, rdram, ps2_syscalls::SignalSema, static_cast(workSid)); } const bool done = waitUntil([&](){ return rdramSeq(rdram) >= 1u; }, std::chrono::milliseconds(1000)); @@ -3500,7 +3379,7 @@ void register_scheduler_protocol_tests() } t.Equals(getSemaCount(rdram, &runtime, workSid), 0, "T12: permit consumed"); - waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + drainedWithin(std::chrono::milliseconds(1000)); deleteSchedSema(rdram.data(), &runtime, workSid); deleteSchedSema(rdram.data(), &runtime, doneSid); }); @@ -3523,10 +3402,7 @@ void register_scheduler_protocol_tests() return; } - { - int32_t d = doneSid; - std::memcpy(rdram.data() + kSlotDoneSid, &d, 4); - } + rdramWrite32(rdram, kSlotDoneSid, static_cast(doneSid)); rdramSeqReset(rdram); const int32_t tid = startSchedWorker(rdram.data(), &runtime, 0x006C0000u, 10, 0x005C0000u, 0x2000u); @@ -3550,9 +3426,7 @@ void register_scheduler_protocol_tests() t.IsTrue(sleeping, "T13: fiber is in SleepThread"); { - R5900Context sc{}; setRegU32(sc, 4, static_cast(tid)); - ps2_syscalls::WakeupThread(rdram.data(), &sc, &runtime); - t.Equals(getRegS32(sc, 2), KE_OK, "T13: WakeupThread returned KE_OK"); + t.Equals(callSyscall(runtime, rdram, ps2_syscalls::WakeupThread, static_cast(tid)), KE_OK, "T13: WakeupThread returned KE_OK"); } const bool woke = waitUntil([&](){ return rdramSeq(rdram) >= 1u; }, std::chrono::milliseconds(1000)); @@ -3563,7 +3437,7 @@ void register_scheduler_protocol_tests() t.Equals(log0, tid, "T13: correct fiber woke"); } - waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + drainedWithin(std::chrono::milliseconds(1000)); deleteSchedSema(rdram.data(), &runtime, doneSid); }); @@ -3586,10 +3460,7 @@ void register_scheduler_protocol_tests() return; } - { - int32_t w = workSid; - std::memcpy(rdram.data() + kSlotWorkSid, &w, 4); - } + rdramWrite32(rdram, kSlotWorkSid, static_cast(workSid)); rdramSeqReset(rdram); const int32_t workerTid = startSchedWorker(rdram.data(), &runtime, 0x006D0000u, 10, 0x005D0000u, 0x2000u); @@ -3601,15 +3472,12 @@ void register_scheduler_protocol_tests() }, std::chrono::milliseconds(500)); t.IsTrue(blocked, "T14: worker blocked on workSid"); - { - int32_t v = workerTid; - std::memcpy(rdram.data() + kSlotTidParam, &v, 4); - } + rdramWrite32(rdram, kSlotTidParam, static_cast(workerTid)); const int32_t killerTid = startSchedWorker(rdram.data(), &runtime, 0x006D8000u, 5, 0x005D2000u, 0x2000u); t.IsTrue(killerTid > 0, "T14: killer started"); - const bool allDone = waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(3000)); + const bool allDone = drainedWithin(std::chrono::milliseconds(3000)); t.IsTrue(allDone, "T14: both worker and killer finished"); t.IsTrue(rdramSeq(rdram) >= 1u, "T14: killer logged after join_fiber returned"); @@ -3634,10 +3502,7 @@ void register_scheduler_protocol_tests() return; } - { - int32_t w = workSid; - std::memcpy(rdram.data() + kSlotWorkSid, &w, 4); - } + rdramWrite32(rdram, kSlotWorkSid, static_cast(workSid)); const int32_t tid1 = startSchedWorker(rdram.data(), &runtime, 0x006E0000u, 10, 0x005E0000u, 0x2000u); const int32_t tid2 = startSchedWorker(rdram.data(), &runtime, 0x006E0000u, 10, 0x005E2000u, 0x2000u); @@ -3685,15 +3550,11 @@ void register_scheduler_protocol_tests() } { - int32_t e = eid, d = doneSid; - std::memcpy(rdram.data() + kSlotEid, &e, 4); - std::memcpy(rdram.data() + kSlotDoneSid, &d, 4); + rdramWrite32(rdram, kSlotEid, static_cast(eid)); + rdramWrite32(rdram, kSlotDoneSid, static_cast(doneSid)); } rdramSeqReset(rdram); - { - int32_t bad = -9999; - std::memcpy(rdram.data() + kResultBase, &bad, 4); - } + rdramWrite32(rdram, kResultBase, static_cast(-9999)); rdramWrite32(rdram, kSlotResBits, 0u); const int32_t tid = startSchedWorker(rdram.data(), &runtime, 0x006F0000u, 10, 0x005F0000u, 0x2000u); @@ -3707,8 +3568,7 @@ void register_scheduler_protocol_tests() // Partial set — AND not satisfied { - R5900Context sc{}; setRegU32(sc, 4, static_cast(eid)); setRegU32(sc, 5, 0x1u); - ps2_syscalls::SetEventFlag(rdram.data(), &sc, &runtime); + callSyscall(runtime, rdram, ps2_syscalls::SetEventFlag, static_cast(eid), 0x1u); } // Fiber may briefly dequeue and re-block; wait for it to re-block @@ -3722,8 +3582,7 @@ void register_scheduler_protocol_tests() // Complete set { - R5900Context sc{}; setRegU32(sc, 4, static_cast(eid)); setRegU32(sc, 5, 0x2u); - ps2_syscalls::SetEventFlag(rdram.data(), &sc, &runtime); + callSyscall(runtime, rdram, ps2_syscalls::SetEventFlag, static_cast(eid), 0x2u); } const bool completed = waitUntil([&](){ return rdramSeq(rdram) >= 1u; }, std::chrono::milliseconds(1000)); @@ -3740,7 +3599,7 @@ void register_scheduler_protocol_tests() t.IsTrue((resBits & 0x3u) == 0x3u, "T16: result bits include all waited bits"); } - waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + drainedWithin(std::chrono::milliseconds(1000)); deleteSchedEvf(rdram, &runtime, eid); deleteSchedSema(rdram.data(), &runtime, doneSid); }); @@ -3768,9 +3627,8 @@ void register_scheduler_protocol_tests() } { - int32_t e = eid, d = doneSid; - std::memcpy(rdram.data() + kSlotEid, &e, 4); - std::memcpy(rdram.data() + kSlotDoneSid, &d, 4); + rdramWrite32(rdram, kSlotEid, static_cast(eid)); + rdramWrite32(rdram, kSlotDoneSid, static_cast(doneSid)); } rdramSeqReset(rdram); for (int i = 0; i < 3; ++i) @@ -3805,12 +3663,10 @@ void register_scheduler_protocol_tests() // eid is now invalid { - R5900Context sc{}; setRegU32(sc, 4, static_cast(eid)); setRegU32(sc, 5, kReferScratch); - ps2_syscalls::ReferEventFlagStatus(rdram.data(), &sc, &runtime); - t.IsTrue(getRegS32(sc, 2) < 0, "T17: deleted evf returns error on Refer"); + t.IsTrue(callSyscall(runtime, rdram, ps2_syscalls::ReferEventFlagStatus, static_cast(eid), kReferScratch) < 0, "T17: deleted evf returns error on Refer"); } - waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(2000)); + drainedWithin(std::chrono::milliseconds(2000)); deleteSchedSema(rdram.data(), &runtime, doneSid); }); @@ -3836,9 +3692,8 @@ void register_scheduler_protocol_tests() } { - int32_t w = workSid, d = doneSid; - std::memcpy(rdram.data() + kSlotWorkSid, &w, 4); - std::memcpy(rdram.data() + kSlotDoneSid, &d, 4); + rdramWrite32(rdram, kSlotWorkSid, static_cast(workSid)); + rdramWrite32(rdram, kSlotDoneSid, static_cast(doneSid)); } rdramSeqReset(rdram); for (int i = 0; i < 3; ++i) @@ -3859,9 +3714,7 @@ void register_scheduler_protocol_tests() t.IsTrue(allBlocked, "T18: all 3 blocked on workSid"); { - R5900Context sc{}; setRegU32(sc, 4, static_cast(workSid)); - ps2_syscalls::DeleteSema(rdram.data(), &sc, &runtime); - t.Equals(getRegS32(sc, 2), workSid, "T18: DeleteSema returned workSid"); + t.Equals(callSyscall(runtime, rdram, ps2_syscalls::DeleteSema, static_cast(workSid)), workSid, "T18: DeleteSema returned workSid"); } const bool allDone = waitUntil([&](){ return rdramSeq(rdram) >= 3u; }, std::chrono::milliseconds(2000)); @@ -3877,12 +3730,10 @@ void register_scheduler_protocol_tests() // workSid is now invalid { - R5900Context sc{}; setRegU32(sc, 4, static_cast(workSid)); - ps2_syscalls::PollSema(rdram.data(), &sc, &runtime); - t.IsTrue(getRegS32(sc, 2) < 0, "T18: deleted sema returns error on Poll"); + t.IsTrue(callSyscall(runtime, rdram, ps2_syscalls::PollSema, static_cast(workSid)) < 0, "T18: deleted sema returns error on Poll"); } - waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(2000)); + drainedWithin(std::chrono::milliseconds(2000)); deleteSchedSema(rdram.data(), &runtime, doneSid); }); @@ -3909,7 +3760,7 @@ void register_scheduler_protocol_tests() const int32_t tid = startSchedWorker(rdram.data(), &runtime, 0x00720000u, 10, stackAddr, 0x2000u); t.IsTrue(tid > 0, std::string("T19: fiber ") + std::to_string(i) + " started"); - const bool drained = waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + const bool drained = drainedWithin(std::chrono::milliseconds(1000)); t.IsTrue(drained, std::string("T19: fiber ") + std::to_string(i) + " exited and g_activeThreads returned to 0"); } @@ -4084,8 +3935,8 @@ void register_scheduler_race_tests() return; } - std::memcpy(rdram.data() + kRSlotWorkSid, &workSid, 4); - std::memcpy(rdram.data() + kRSlotDoneSid, &doneSid, 4); + rdramWrite32(rdram, kRSlotWorkSid, static_cast(workSid)); + rdramWrite32(rdram, kRSlotDoneSid, static_cast(doneSid)); gRProgress.store(0u, std::memory_order_release); gRTokenLo.store(0u, std::memory_order_release); gRTokenHi.store(0u, std::memory_order_release); @@ -4128,16 +3979,11 @@ void register_scheduler_race_tests() { for (int i = 0; i < kRounds; ++i) { - R5900Context sc{}; - setRegU32(sc, 4, static_cast(workSid)); - ps2_syscalls::SignalSema(rdram.data(), &sc, &runtime); - while (getRegS32(sc, 2) != workSid) + int32_t sigRet = callSyscall(runtime, rdram, ps2_syscalls::SignalSema, static_cast(workSid)); + while (sigRet != workSid) { std::this_thread::yield(); - R5900Context sc2{}; - setRegU32(sc2, 4, static_cast(workSid)); - ps2_syscalls::SignalSema(rdram.data(), &sc2, &runtime); - sc = sc2; + sigRet = callSyscall(runtime, rdram, ps2_syscalls::SignalSema, static_cast(workSid)); } } }); @@ -4155,10 +4001,7 @@ void register_scheduler_race_tests() t.IsTrue(reached, "R1: fiber completed all 500 park/wake rounds (no lost wakeup)"); // Drain: fiber signals doneSid and exits. - const bool finished = waitUntil([&]() - { - return g_activeThreads.load(std::memory_order_acquire) == 0; - }, std::chrono::milliseconds(2000)); + const bool finished = drainedWithin(std::chrono::milliseconds(2000)); t.IsTrue(finished, "R1: worker fiber exits cleanly (g_activeThreads==0)"); deleteSchedSema(rdram.data(), &runtime, workSid); @@ -4191,10 +4034,7 @@ void register_scheduler_race_tests() try { ps2sched::async_guest_begin(); - R5900Context sc{}; - setRegU32(sc, 4, static_cast(blockSid)); - ps2_syscalls::WaitSema(rdram.data(), &sc, &runtime); - waitRet.store(getRegS32(sc, 2), std::memory_order_release); + waitRet.store(callSyscall(runtime, rdram, ps2_syscalls::WaitSema, static_cast(blockSid)), std::memory_order_release); ps2sched::async_guest_end(); } catch (...) @@ -4209,9 +4049,7 @@ void register_scheduler_race_tests() // then feed it a permit so the Mesa re-check returns blockSid. std::this_thread::sleep_for(std::chrono::milliseconds(20)); { - R5900Context sc{}; - setRegU32(sc, 4, static_cast(blockSid)); - ps2_syscalls::SignalSema(rdram.data(), &sc, &runtime); + callSyscall(runtime, rdram, ps2_syscalls::SignalSema, static_cast(blockSid)); } const bool returned = waitUntil([&]() @@ -4239,7 +4077,7 @@ void register_scheduler_race_tests() }, std::chrono::milliseconds(2000)); t.IsTrue(ran, "R2: scheduler still healthy — fiber runs after the non-fiber block"); - waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(2000)); + drainedWithin(std::chrono::milliseconds(2000)); deleteSchedSema(rdram.data(), &runtime, blockSid); }); @@ -4270,12 +4108,7 @@ void register_scheduler_race_tests() // thread with no intervening sleep, so stop is requested within a // handful of microseconds -- many orders of magnitude below 3.84s // even under heavy TSan instrumentation. - R5900Context sc{}; - setRegU32(sc, 4, 60000u); // ticks = 60000 (~3.84s); see rationale above - setRegU32(sc, 5, 0x00732000u); // handler entry - setRegU32(sc, 6, 0u); // arg - ps2_syscalls::SetAlarm(rdram.data(), &sc, &runtime); - const int32_t alarmId = getRegS32(sc, 2); + const int32_t alarmId = callSyscall(runtime, rdram, ps2_syscalls::SetAlarm, 60000u, 0x00732000u, 0u); t.IsTrue(alarmId > 0, "R3: SetAlarm queued an alarm and started the worker"); // Immediately stop the worker — must return promptly (joins, does not spin). @@ -4327,9 +4160,7 @@ void register_scheduler_race_tests() // Wake the sleeping exit handler from the host. { - R5900Context sc{}; - setRegU32(sc, 4, static_cast(tid)); - ps2_syscalls::WakeupThread(rdram.data(), &sc, &runtime); + callSyscall(runtime, rdram, ps2_syscalls::WakeupThread, static_cast(tid)); } // Handler should complete and the fiber should finish. @@ -4339,10 +4170,7 @@ void register_scheduler_race_tests() }, std::chrono::milliseconds(2000)); t.IsTrue(handlerDone, "R4: blocking exit handler resumed and ran to completion"); - const bool drained = waitUntil([&]() - { - return g_activeThreads.load(std::memory_order_acquire) == 0; - }, std::chrono::milliseconds(2000)); + const bool drained = drainedWithin(std::chrono::milliseconds(2000)); t.IsTrue(drained, "R4: fiber freed cleanly after exit handler finished (no double-free/leak)"); }); @@ -4384,17 +4212,11 @@ void register_scheduler_race_tests() }; auto suspend = [&]() { - R5900Context sc{}; - setRegU32(sc, 4, static_cast(tid)); - ps2_syscalls::SuspendThread(rdram.data(), &sc, &runtime); - return getRegS32(sc, 2); + return callSyscall(runtime, rdram, ps2_syscalls::SuspendThread, static_cast(tid)); }; auto resume = [&]() { - R5900Context sc{}; - setRegU32(sc, 4, static_cast(tid)); - ps2_syscalls::ResumeThread(rdram.data(), &sc, &runtime); - return getRegS32(sc, 2); + return callSyscall(runtime, rdram, ps2_syscalls::ResumeThread, static_cast(tid)); }; // Suspend twice (nested). @@ -4423,7 +4245,7 @@ void register_scheduler_race_tests() { gStopFlag.store(1u, std::memory_order_release); } - waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(2000)); + drainedWithin(std::chrono::milliseconds(2000)); }); @@ -4528,9 +4350,7 @@ void register_scheduler_stress_tests() { if (gR6Done.load(std::memory_order_acquire) != 0u) break; - R5900Context wc{}; - setRegU32(wc, 4, static_cast(tid)); - ps2_syscalls::WakeupThread(rdram.data(), &wc, &runtime); + callSyscall(runtime, rdram, ps2_syscalls::WakeupThread, static_cast(tid)); // No sleep, no THS_WAIT confirmation — keep the loop as tight as possible so // wakeups race against the fiber's arm_park -> block_current transition. } @@ -4547,10 +4367,7 @@ void register_scheduler_stress_tests() t.Equals(counter, 500u, "R6: fiber should have returned from SleepThread exactly 500 times"); // Fiber should have exited and decremented g_activeThreads. - const bool drained = waitUntil([&]() - { - return g_activeThreads.load(std::memory_order_acquire) <= 0; - }, std::chrono::milliseconds(1000)); + const bool drained = drainedWithin(std::chrono::milliseconds(1000)); t.IsTrue(drained, "R6: sleep-loop fiber should exit and drain g_activeThreads"); }); @@ -4603,10 +4420,7 @@ void register_scheduler_stress_tests() { g_currentThreadId = -1; // non-fiber host worker (matches IRQ/alarm workers) ps2sched::async_guest_begin(); // acquire the guest token (AsyncGuestScope-equivalent) - R5900Context wc{}; - setRegU32(wc, 4, static_cast(workSid)); - ps2_syscalls::WaitSema(rdram.data(), &wc, &runtime); // count=0 -> borrowed-worker block/retry path - workerRet.store(getRegS32(wc, 2), std::memory_order_release); + workerRet.store(callSyscall(runtime, rdram, ps2_syscalls::WaitSema, static_cast(workSid)), std::memory_order_release); // count=0 -> borrowed-worker block/retry path ps2sched::async_guest_end(); // release the guest token } catch (...) @@ -4632,9 +4446,7 @@ void register_scheduler_stress_tests() // Deadlock escape hatch so we don't hang the whole test binary on failure: // signal the sema from this (main) thread to release the borrowed worker, // then join. The assertion below still records the failure. - R5900Context sc{}; - setRegU32(sc, 4, static_cast(workSid)); - ps2_syscalls::SignalSema(rdram.data(), &sc, &runtime); + callSyscall(runtime, rdram, ps2_syscalls::SignalSema, static_cast(workSid)); } if (borrowedWorker.joinable()) @@ -4651,10 +4463,7 @@ void register_scheduler_stress_tests() t.Equals(signalled, 1u, "R7: the fiber (not the main thread) should have produced the permit"); // Everything should drain. - const bool drained = waitUntil([&]() - { - return g_activeThreads.load(std::memory_order_acquire) <= 0; - }, std::chrono::milliseconds(1000)); + const bool drained = drainedWithin(std::chrono::milliseconds(1000)); t.IsTrue(drained, "R7: signalling fiber should exit and drain g_activeThreads"); deleteSchedSema(rdram.data(), &runtime, workSid); @@ -4741,10 +4550,7 @@ void register_scheduler_vsync_priority_tests() { return; } - { - int32_t v = workSid; - std::memcpy(rdram.data() + kS1SlotWorkSid, &v, 4); - } + rdramWrite32(rdram, kS1SlotWorkSid, static_cast(workSid)); // Start fiber A (the vsync waiter). WaitForNextVSyncTick calls // EnsureVSyncWorkerRunning internally. @@ -4780,18 +4586,12 @@ void register_scheduler_vsync_priority_tests() // it (join_fiber). A unwinds out of WaitForNextVSyncTick; the wake path // removes A's tid from g_vsync_waitList before signaling B. { - R5900Context termCtx{}; - setRegU32(termCtx, 4, static_cast(tidA)); - ps2_syscalls::TerminateThread(rdram.data(), &termCtx, &runtime); - t.Equals(getRegS32(termCtx, 2), KE_OK, "S1: TerminateThread(A) returns KE_OK"); + t.Equals(callSyscall(runtime, rdram, ps2_syscalls::TerminateThread, static_cast(tidA)), KE_OK, "S1: TerminateThread(A) returns KE_OK"); } // Wait for A to finish (TerminateThread joined it, but g_activeThreads // decrement may trail slightly). - const bool aGone = waitUntil([&]() - { - return g_activeThreads.load(std::memory_order_acquire) == 0; - }, std::chrono::milliseconds(1000)); + const bool aGone = drainedWithin(std::chrono::milliseconds(1000)); t.IsTrue(aGone, "S1: fiber A finished after TerminateThread"); // A's tid must no longer be in g_vsync_waitList. @@ -4832,11 +4632,9 @@ void register_scheduler_vsync_priority_tests() // Cleanup: signal the sema so B can unwind cleanly, then shut down. { - R5900Context sigCtx{}; - setRegU32(sigCtx, 4, static_cast(workSid)); - ps2_syscalls::SignalSema(rdram.data(), &sigCtx, &runtime); + callSyscall(runtime, rdram, ps2_syscalls::SignalSema, static_cast(workSid)); } - waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + drainedWithin(std::chrono::milliseconds(1000)); deleteSchedSema(rdram.data(), &runtime, workSid); ps2sched::scheduler_shutdown(); // joins the vsync worker via stopInterruptWorker() @@ -4873,11 +4671,7 @@ void register_scheduler_vsync_priority_tests() // Change A's priority from 20 to 5 while A is queued Ready. // Expected new order: [A(5), C(10), B(15)]. { - R5900Context sc{}; - setRegU32(sc, 4, static_cast(tidA)); - setRegU32(sc, 5, 5u); - ps2_syscalls::ChangeThreadPriority(rdram.data(), &sc, &runtime); - t.Equals(getRegS32(sc, 2), KE_OK, "S2: ChangeThreadPriority(A,5) returns KE_OK"); + t.Equals(callSyscall(runtime, rdram, ps2_syscalls::ChangeThreadPriority, static_cast(tidA), 5u), KE_OK, "S2: ChangeThreadPriority(A,5) returns KE_OK"); } // Release the executor; fibers now run in the re-sorted priority order. @@ -4893,7 +4687,7 @@ void register_scheduler_vsync_priority_tests() t.Equals(log[1], tidC, "S2: C (prio 10) runs second"); t.Equals(log[2], tidB, "S2: B (prio 15) runs last"); - waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + drainedWithin(std::chrono::milliseconds(1000)); }); }); // MiniTest::Case("SchedulerVSyncAndPriority") @@ -5062,14 +4856,13 @@ void register_scheduler_lifecycle_tests() // Sentinels: -1 means "not written yet" so we can tell the fiber actually ran. gU1FiberOnExec.store(-1, std::memory_order_release); - { int32_t init = -1; std::memcpy(rdram.data() + kU1SlotHostOnExec, &init, 4); } + rdramWrite32(rdram, kU1SlotHostOnExec, static_cast(-1)); // Host-thread side: a plain std::thread is NOT the guest executor thread, // so ps2fiber_on_executor_thread() must return false there. std::thread hostProbe([&]() { - const int32_t r = ps2fiber_on_executor_thread() ? 1 : 0; - std::memcpy(rdram.data() + kU1SlotHostOnExec, &r, 4); + rdramWrite32(rdram, kU1SlotHostOnExec, ps2fiber_on_executor_thread() ? 1u : 0u); }); hostProbe.join(); @@ -5084,7 +4877,7 @@ void register_scheduler_lifecycle_tests() }, std::chrono::milliseconds(2000)); t.IsTrue(fiberWrote, "U1: probe fiber recorded its on-executor result"); - waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + drainedWithin(std::chrono::milliseconds(1000)); const int32_t fiberSaw = gU1FiberOnExec.load(std::memory_order_acquire); int32_t hostSaw = -1; @@ -5119,7 +4912,7 @@ void register_scheduler_lifecycle_tests() runtime.registerFunction(0x00770000u, &stepU2TargetYieldLoop); gU2Spinning.store(0u, std::memory_order_release); - { int32_t neg = -1; std::memcpy(rdram.data() + kU2SlotJoinerPrio, &neg, 4); } + rdramWrite32(rdram, kU2SlotJoinerPrio, static_cast(-1)); // Lock the executor so no fibers run until we are ready. ps2sched::async_guest_begin(); @@ -5130,12 +4923,12 @@ void register_scheduler_lifecycle_tests() // (prio 61) so that B (prio 60) runs while A waits. const int32_t tidB = startSchedWorker(rdram.data(), &runtime, 0x00770000u, 60, 0x004E0000u, 0x2000u); t.IsTrue(tidB > 0, "U2: target B started"); - { int32_t v = tidB; std::memcpy(rdram.data() + kU2SlotTargetTid, &v, 4); } + rdramWrite32(rdram, kU2SlotTargetTid, static_cast(tidB)); // Start joiner A (prio 50). A immediately calls TerminateThread(B) -> join_fiber(B). const int32_t tidA = startSchedWorker(rdram.data(), &runtime, 0x00768000u, 50, 0x004DC000u, 0x2000u); t.IsTrue(tidA > 0, "U2: joiner A started"); - { int32_t v = tidA; std::memcpy(rdram.data() + kU2SlotJoinerTid, &v, 4); } + rdramWrite32(rdram, kU2SlotJoinerTid, static_cast(tidA)); // Start reprio thread BEFORE releasing the executor so it is already // spinning when B sets kU2SlotSpinning=1. Use a tight spin (no sleep) @@ -5152,10 +4945,7 @@ void register_scheduler_lifecycle_tests() if (s == 1u) break; } if (s != 1u) return; - R5900Context sc{}; - setRegU32(sc, 4, static_cast(tidA)); - setRegU32(sc, 5, 10u); - ps2_syscalls::ChangeThreadPriority(rdram.data(), &sc, &runtime); + callSyscall(runtime, rdram, ps2_syscalls::ChangeThreadPriority, static_cast(tidA), 10u); }); // Wait until reprio is spinning before releasing the executor. @@ -5167,10 +4957,7 @@ void register_scheduler_lifecycle_tests() ps2sched::async_guest_end(); // Wait for both fibers to drain (A finishes after the join returns and it logs prio). - const bool drained = waitUntil([&]() - { - return g_activeThreads.load(std::memory_order_acquire) == 0; - }, std::chrono::milliseconds(4000)); + const bool drained = drainedWithin(std::chrono::milliseconds(4000)); reprio.join(); t.IsTrue(drained, "U2: both fibers finished"); @@ -5201,8 +4988,8 @@ void register_scheduler_lifecycle_tests() { return; } - { int32_t v = workSid; std::memcpy(rdram.data() + kU3SlotWorkSid, &v, 4); } - { uint32_t z = 0u; std::memcpy(rdram.data() + kU3SlotExitRan, &z, 4); } + rdramWrite32(rdram, kU3SlotWorkSid, static_cast(workSid)); + rdramWrite32(rdram, kU3SlotExitRan, static_cast(0u)); const int32_t workerTid = startSchedWorker(rdram.data(), &runtime, 0x00778000u, 10, 0x004E4000u, 0x2000u); t.IsTrue(workerTid > 0, "U3: worker started"); @@ -5216,16 +5003,10 @@ void register_scheduler_lifecycle_tests() // Terminate from the host: request_terminate wakes the worker, it unwinds via // ThreadExitException, fiber_trampoline runs the exit hook -> our handler. { - R5900Context termCtx{}; - setRegU32(termCtx, 4, static_cast(workerTid)); - ps2_syscalls::TerminateThread(rdram.data(), &termCtx, &runtime); - t.Equals(getRegS32(termCtx, 2), KE_OK, "U3: TerminateThread returns KE_OK"); + t.Equals(callSyscall(runtime, rdram, ps2_syscalls::TerminateThread, static_cast(workerTid)), KE_OK, "U3: TerminateThread returns KE_OK"); } - const bool drained = waitUntil([&]() - { - return g_activeThreads.load(std::memory_order_acquire) == 0; - }, std::chrono::milliseconds(3000)); + const bool drained = drainedWithin(std::chrono::milliseconds(3000)); t.IsTrue(drained, "U3: g_activeThreads reached 0 after termination"); uint32_t exitRan = 0u; @@ -5253,7 +5034,7 @@ void register_scheduler_lifecycle_tests() { return; } - { int32_t v = workSid; std::memcpy(rdram.data() + kU4SlotWorkSid, &v, 4); } + rdramWrite32(rdram, kU4SlotWorkSid, static_cast(workSid)); std::memset(rdram.data() + kU4BodyBase, 0, kU4FiberCount * 4); std::memset(rdram.data() + kU4HookBase, 0, kU4FiberCount * 4); @@ -5463,10 +5244,7 @@ void register_scheduler_borrowed_worker_tests() { g_currentThreadId = -1; // non-fiber host worker ps2sched::async_guest_begin(); - R5900Context wc{}; - setRegU32(wc, 4, static_cast(workSid)); - ps2_syscalls::WaitSema(rdram.data(), &wc, &runtime); - workerRet.store(getRegS32(wc, 2), std::memory_order_release); + workerRet.store(callSyscall(runtime, rdram, ps2_syscalls::WaitSema, static_cast(workSid)), std::memory_order_release); ps2sched::async_guest_end(); } catch (...) @@ -5486,9 +5264,7 @@ void register_scheduler_borrowed_worker_tests() { // Escape hatch so a regression does not hang the whole binary; the // assertions below still record the failure. - R5900Context sc{}; - setRegU32(sc, 4, static_cast(workSid)); - ps2_syscalls::SignalSema(rdram.data(), &sc, &runtime); + callSyscall(runtime, rdram, ps2_syscalls::SignalSema, static_cast(workSid)); } if (borrowedWorker.joinable()) borrowedWorker.join(); @@ -5508,10 +5284,7 @@ void register_scheduler_borrowed_worker_tests() "W1: borrowed worker created no g_threads[-1] entry"); } - const bool drained = waitUntil([&]() - { - return g_activeThreads.load(std::memory_order_acquire) <= 0; - }, std::chrono::milliseconds(1000)); + const bool drained = drainedWithin(std::chrono::milliseconds(1000)); t.IsTrue(drained, "W1: producer fiber drained g_activeThreads"); deleteSchedSema(rdram.data(), &runtime, workSid); @@ -5543,10 +5316,7 @@ void register_scheduler_borrowed_worker_tests() // The fiber runs its body (registers the handler, returns), then the trampoline // drives on_fiber_exit -> our handler -> ExitThread (throws). The exit hook // is wrapped in try/catch, so a throwing handler must not crash the process. - const bool drained = waitUntil([&]() - { - return g_activeThreads.load(std::memory_order_acquire) == 0; - }, std::chrono::milliseconds(3000)); + const bool drained = drainedWithin(std::chrono::milliseconds(3000)); t.IsTrue(drained, "W2: fiber cleaned up despite ExitThread thrown from its exit handler"); uint32_t bodyRan = 0u, sentinel = 0u; @@ -5766,19 +5536,14 @@ void register_scheduler_window_tests() uint32_t iters = 0u; while (std::chrono::steady_clock::now() < deadline && iters < 200000u) { - R5900Context sc{}; - setRegU32(sc, 4, static_cast(sid)); - ps2_syscalls::SignalSema(rdram.data(), &sc, &runtime); - const int32_t sret = getRegS32(sc, 2); + const int32_t sret = callSyscall(runtime, rdram, ps2_syscalls::SignalSema, static_cast(sid)); signalsSent.fetch_add(1, std::memory_order_relaxed); // If the permit is still outstanding (waiter has not consumed it), // drain it so the next SignalSema is a fresh 0->1 transition. if (sret == sid) { - R5900Context pc{}; - setRegU32(pc, 4, static_cast(sid)); - ps2_syscalls::PollSema(rdram.data(), &pc, &runtime); + callSyscall(runtime, rdram, ps2_syscalls::PollSema, static_cast(sid)); } ++iters; } @@ -5809,9 +5574,7 @@ void register_scheduler_window_tests() // waiter parked between iterations wakes, sees the stop flag, and returns. gX1StopFlag.store(1u, std::memory_order_release); { - R5900Context sc{}; - setRegU32(sc, 4, static_cast(sid)); - ps2_syscalls::SignalSema(rdram.data(), &sc, &runtime); + callSyscall(runtime, rdram, ps2_syscalls::SignalSema, static_cast(sid)); } const bool exited = waitUntil([&]() @@ -5820,10 +5583,7 @@ void register_scheduler_window_tests() }, std::chrono::milliseconds(3000)); t.IsTrue(exited, "X1: waiter fiber observed stop flag and exited"); - const bool drained = waitUntil([&]() - { - return g_activeThreads.load(std::memory_order_acquire) <= 0; - }, std::chrono::milliseconds(3000)); + const bool drained = drainedWithin(std::chrono::milliseconds(3000)); t.IsTrue(drained, "X1: waiter fiber drained g_activeThreads"); // Reported for visibility; not asserted as a hard number (cooperative @@ -5910,9 +5670,7 @@ void register_scheduler_window_tests() // Each successful wake must make it across the CV to the executor. // Wait (bounded) until the fiber's wake count catches up so we // exercise one notify at a time against live CV contention. - R5900Context sc{}; - setRegU32(sc, 4, static_cast(sid)); - ps2_syscalls::SignalSema(rdram.data(), &sc, &runtime); + callSyscall(runtime, rdram, ps2_syscalls::SignalSema, static_cast(sid)); const uint32_t target = i + 1u; const bool advanced = waitUntil([&]() @@ -5963,10 +5721,7 @@ void register_scheduler_window_tests() }, std::chrono::milliseconds(3000)); t.IsTrue(exited, "X2: executor fiber completed its 64-wake loop and exited"); - const bool drained = waitUntil([&]() - { - return g_activeThreads.load(std::memory_order_acquire) <= 0; - }, std::chrono::milliseconds(3000)); + const bool drained = drainedWithin(std::chrono::milliseconds(3000)); t.IsTrue(drained, "X2: executor fiber drained g_activeThreads"); deleteSchedSema(rdram.data(), &runtime, sid); @@ -6116,10 +5871,7 @@ void register_scheduler_sleep_resume_tests() // Step 2: suspend the sleeping fiber; it must transition to THS_WAITSUSPEND. { - R5900Context sc{}; - setRegU32(sc, 4, static_cast(tid)); - ps2_syscalls::SuspendThread(rdram.data(), &sc, &runtime); - t.Equals(getRegS32(sc, 2), KE_OK, "Y1: SuspendThread returned KE_OK"); + t.Equals(callSyscall(runtime, rdram, ps2_syscalls::SuspendThread, static_cast(tid)), KE_OK, "Y1: SuspendThread returned KE_OK"); } const bool suspended = waitUntil([&]() @@ -6133,10 +5885,7 @@ void register_scheduler_sleep_resume_tests() // Step 3: ResumeThread clears the suspend gate but provides NO wakeupCount permit. // The Mesa loop inside SleepThread must re-park. { - R5900Context sc{}; - setRegU32(sc, 4, static_cast(tid)); - ps2_syscalls::ResumeThread(rdram.data(), &sc, &runtime); - t.Equals(getRegS32(sc, 2), KE_OK, "Y1: ResumeThread returned KE_OK"); + t.Equals(callSyscall(runtime, rdram, ps2_syscalls::ResumeThread, static_cast(tid)), KE_OK, "Y1: ResumeThread returned KE_OK"); } // Step 4: wait generously, then assert the fiber did NOT exit SleepThread. @@ -6158,10 +5907,7 @@ void register_scheduler_sleep_resume_tests() // Step 5: a genuine WakeupThread must now release the fiber from SleepThread. { - R5900Context sc{}; - setRegU32(sc, 4, static_cast(tid)); - ps2_syscalls::WakeupThread(rdram.data(), &sc, &runtime); - t.Equals(getRegS32(sc, 2), KE_OK, "Y1: WakeupThread returned KE_OK"); + t.Equals(callSyscall(runtime, rdram, ps2_syscalls::WakeupThread, static_cast(tid)), KE_OK, "Y1: WakeupThread returned KE_OK"); } const bool woke = waitUntil([&]() @@ -6175,7 +5921,7 @@ void register_scheduler_sleep_resume_tests() t.Equals(sleepRet, KE_OK, "Y1: SleepThread returned KE_OK on genuine wakeup"); } - waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + drainedWithin(std::chrono::milliseconds(1000)); }); }); // MiniTest::Case("SchedulerSleepResume") @@ -6292,11 +6038,8 @@ void register_scheduler_borrowed_guard_tests() { g_currentThreadId = -1; // borrowed host worker: no PS2 thread identity ps2sched::async_guest_begin(); - R5900Context ctx{}; - setRegU32(ctx, 4, c.a0); // $a0 = 0 (TH_SELF / self-target) - setRegU32(ctx, 5, c.a1); // $a1 = extra arg (e.g. new priority) - c.fn(rdram.data(), &ctx, &runtime); - retVal.store(getRegS32(ctx, 2), std::memory_order_release); + // c.a0 = $a0 = 0 (TH_SELF / self-target); c.a1 = $a1 = extra arg (e.g. new priority) + retVal.store(callSyscall(runtime, rdram, c.fn, c.a0, c.a1), std::memory_order_release); ps2sched::async_guest_end(); } catch (...) @@ -6352,14 +6095,10 @@ void register_scheduler_sema_delete_tests() } // Publish the sema id and reset result slots before starting fibers. - { - int32_t v = workSid; - std::memcpy(rdram.data() + kY4WorkSid, &v, 4); - } + rdramWrite32(rdram, kY4WorkSid, static_cast(workSid)); for (int i = 0; i < kN; ++i) { - int32_t bad = -9999; - std::memcpy(rdram.data() + kY4RetBase + static_cast(i) * 4u, &bad, 4); + rdramWrite32(rdram, kY4RetBase + static_cast(i) * 4u, static_cast(-9999)); } gY4SlotCounter.store(0, std::memory_order_relaxed); @@ -6388,17 +6127,11 @@ void register_scheduler_sema_delete_tests() // DeleteSema: must drain the pair-based waitList and wake all N with KE_WAIT_DELETE. { - R5900Context dc{}; - setRegU32(dc, 4, static_cast(workSid)); - ps2_syscalls::DeleteSema(rdram.data(), &dc, &runtime); - t.Equals(getRegS32(dc, 2), workSid, "Y4a: DeleteSema returned workSid"); + t.Equals(callSyscall(runtime, rdram, ps2_syscalls::DeleteSema, static_cast(workSid)), workSid, "Y4a: DeleteSema returned workSid"); } // All N fibers must finish. - const bool drained = waitUntil([&]() - { - return g_activeThreads.load(std::memory_order_acquire) == 0; - }, std::chrono::milliseconds(2000)); + const bool drained = drainedWithin(std::chrono::milliseconds(2000)); t.IsTrue(drained, "Y4a: all N fibers exited after DeleteSema"); // Every recorded return value must be KE_WAIT_DELETE. @@ -6445,8 +6178,7 @@ void register_scheduler_sema_delete_tests() }, std::chrono::milliseconds(1000)); t.IsTrue(tokenWritten, "Y4b: token fiber published its token"); - waitUntil([&](){ return g_activeThreads.load() == 0; }, - std::chrono::milliseconds(1000)); + drainedWithin(std::chrono::milliseconds(1000)); const uint32_t tokenLo = gY4bTokenLo.load(std::memory_order_acquire); const uint32_t tokenHi = gY4bTokenHi.load(std::memory_order_acquire); @@ -6477,11 +6209,8 @@ void register_scheduler_sema_delete_tests() if (!sleeperSleeping) { // Kick the sleeper out so we can clean up. - R5900Context sc{}; - setRegU32(sc, 4, static_cast(sleeperTid)); - ps2_syscalls::WakeupThread(rdram.data(), &sc, &runtime); - waitUntil([&](){ return g_activeThreads.load() == 0; }, - std::chrono::milliseconds(1000)); + callSyscall(runtime, rdram, ps2_syscalls::WakeupThread, static_cast(sleeperTid)); + drainedWithin(std::chrono::milliseconds(1000)); return; } @@ -6514,10 +6243,7 @@ void register_scheduler_sema_delete_tests() // Sanity: a valid WakeupThread DOES reach the sleeper, proving it was genuinely // reachable and the stale wake was specifically rejected (not an inert no-op). { - R5900Context sc{}; - setRegU32(sc, 4, static_cast(sleeperTid)); - ps2_syscalls::WakeupThread(rdram.data(), &sc, &runtime); - t.Equals(getRegS32(sc, 2), KE_OK, "Y4b: WakeupThread returned KE_OK"); + t.Equals(callSyscall(runtime, rdram, ps2_syscalls::WakeupThread, static_cast(sleeperTid)), KE_OK, "Y4b: WakeupThread returned KE_OK"); } const bool woke = waitUntil([&]() @@ -6526,7 +6252,7 @@ void register_scheduler_sema_delete_tests() }, std::chrono::milliseconds(1000)); t.IsTrue(woke, "Y4b: sleeper woke normally after a valid WakeupThread"); - waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + drainedWithin(std::chrono::milliseconds(1000)); }); }); // MiniTest::Case("SchedulerSemaDelete") @@ -6587,9 +6313,7 @@ void register_scheduler_tid_reuse_tests() // Wait for the executor to run the fiber to completion (outside // the guest scope so the executor can hold the token). - const bool exited = waitUntil( - [&](){ return g_activeThreads.load() == 0; }, - std::chrono::milliseconds(500)); + const bool exited = drainedWithin(std::chrono::milliseconds(500)); if (!exited) { break; // timed out — assertion below will catch it @@ -6598,9 +6322,7 @@ void register_scheduler_tid_reuse_tests() // Delete the dormant thread so CreateThread can reuse the tid // on the next iteration, exercising the same-tid recycle path. ps2sched::async_guest_begin(); - R5900Context dc{}; - setRegU32(dc, 4, static_cast(tid)); - ps2_syscalls::DeleteThread(rdram.data(), &dc, &runtime); + callSyscall(runtime, rdram, ps2_syscalls::DeleteThread, static_cast(tid)); ps2sched::async_guest_end(); } } @@ -6626,8 +6348,7 @@ void register_scheduler_tid_reuse_tests() t.Equals(static_cast(completions), kCycles, "Z1: all fibers ran to completion (kZ1Done == kCycles)"); - waitUntil([&](){ return g_activeThreads.load() == 0; }, - std::chrono::milliseconds(1000)); + drainedWithin(std::chrono::milliseconds(1000)); }); }); // MiniTest::Case("SchedulerTidReuse") @@ -6936,15 +6657,10 @@ void register_scheduler_evf_mode_tests() return; } - { - int32_t e = eid; - std::memcpy(rdram.data() + kAASlotEid, &e, 4); - } - uint32_t zero = 0u; + rdramWrite32(rdram, kAASlotEid, static_cast(eid)); gAASeq.store(0u, std::memory_order_release); - std::memcpy(rdram.data() + kAASlotResBits, &zero, 4); - int32_t bad = -9999; - std::memcpy(rdram.data() + kAASlotResult, &bad, 4); + rdramWrite32(rdram, kAASlotResBits, 0u); + rdramWrite32(rdram, kAASlotResult, static_cast(-9999)); const int32_t tid = startSchedWorker(rdram.data(), &runtime, 0x00808000u, 10, @@ -6965,10 +6681,7 @@ void register_scheduler_evf_mode_tests() // Set ONLY bit 0x1 — OR-mode should be satisfied immediately { - R5900Context sc{}; - setRegU32(sc, 4, static_cast(eid)); - setRegU32(sc, 5, 0x1u); - ps2_syscalls::SetEventFlag(rdram.data(), &sc, &runtime); + callSyscall(runtime, rdram, ps2_syscalls::SetEventFlag, static_cast(eid), 0x1u); } // Fiber must complete without re-blocking @@ -6996,7 +6709,7 @@ void register_scheduler_evf_mode_tests() t.Equals(seq, 1u, "AA1: sequence incremented exactly once (no re-block)"); } - waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + drainedWithin(std::chrono::milliseconds(1000)); deleteSchedEvf(rdram, &runtime, eid); }); @@ -7019,15 +6732,10 @@ void register_scheduler_evf_mode_tests() return; } - { - int32_t e = eid; - std::memcpy(rdram.data() + kAASlotEid, &e, 4); - } - uint32_t zero = 0u; + rdramWrite32(rdram, kAASlotEid, static_cast(eid)); gAASeq.store(0u, std::memory_order_release); - std::memcpy(rdram.data() + kAASlotResBits, &zero, 4); - int32_t bad = -9999; - std::memcpy(rdram.data() + kAASlotResult, &bad, 4); + rdramWrite32(rdram, kAASlotResBits, 0u); + rdramWrite32(rdram, kAASlotResult, static_cast(-9999)); // The initBits=0xF already has bits 0x3 set, so the fiber will satisfy // the AND condition immediately on entry and WEF_CLEAR_ALL must clear all bits. @@ -7059,7 +6767,7 @@ void register_scheduler_evf_mode_tests() t.Equals(currBits, 0u, "AA2: WEF_CLEAR_ALL cleared all bits to zero"); } - waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + drainedWithin(std::chrono::milliseconds(1000)); deleteSchedEvf(rdram, &runtime, eid); }); @@ -7088,9 +6796,8 @@ void register_scheduler_evf_mode_tests() } { - int32_t e = eid, d = doneSid; - std::memcpy(rdram.data() + kSlotEid, &e, 4); - std::memcpy(rdram.data() + kSlotDoneSid, &d, 4); + rdramWrite32(rdram, kSlotEid, static_cast(eid)); + rdramWrite32(rdram, kSlotDoneSid, static_cast(doneSid)); } rdramSeqReset(rdram); for (int i = 0; i < 2; ++i) @@ -7124,7 +6831,7 @@ void register_scheduler_evf_mode_tests() std::string("AA4: fiber ") + std::to_string(i) + " received KE_WAIT_DELETE"); } - waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(2000)); + drainedWithin(std::chrono::milliseconds(2000)); // Now probe the deleted EVF id: WaitEventFlag must return KE_UNKNOWN_EVFID immediately { @@ -7159,15 +6866,10 @@ void register_scheduler_evf_mode_tests() return; } - { - int32_t e = eid; - std::memcpy(rdram.data() + kAASlotEid, &e, 4); - } - uint32_t zero = 0u; + rdramWrite32(rdram, kAASlotEid, static_cast(eid)); gAASeq.store(0u, std::memory_order_release); - std::memcpy(rdram.data() + kAASlotResBits, &zero, 4); - int32_t bad = -9999; - std::memcpy(rdram.data() + kAASlotResult, &bad, 4); + rdramWrite32(rdram, kAASlotResBits, 0u); + rdramWrite32(rdram, kAASlotResult, static_cast(-9999)); const int32_t tid = startSchedWorker(rdram.data(), &runtime, 0x00840000u, 10, @@ -7187,10 +6889,7 @@ void register_scheduler_evf_mode_tests() // Set only bit 0x1 — OR-mode is satisfied, WEF_CLEAR should clear only bit 0x1 { - R5900Context sc{}; - setRegU32(sc, 4, static_cast(eid)); - setRegU32(sc, 5, 0x1u); - ps2_syscalls::SetEventFlag(rdram.data(), &sc, &runtime); + callSyscall(runtime, rdram, ps2_syscalls::SetEventFlag, static_cast(eid), 0x1u); } const bool completed = waitUntil([&]() @@ -7214,7 +6913,7 @@ void register_scheduler_evf_mode_tests() t.IsTrue((currBits & 0x1u) == 0u, "AA13: WEF_CLEAR cleared the matched bit 0x1"); } - waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + drainedWithin(std::chrono::milliseconds(1000)); deleteSchedEvf(rdram, &runtime, eid); }); @@ -7249,10 +6948,7 @@ void register_scheduler_shutdown_fiber_tests() return; } - { - int32_t e = eid; - std::memcpy(rdram.data() + kAASlotEid, &e, 4); - } + rdramWrite32(rdram, kAASlotEid, static_cast(eid)); uint32_t zero = 0u; gAASeq.store(0u, std::memory_order_release); @@ -7346,9 +7042,7 @@ void register_scheduler_join_host_tests() // Signal the fiber to stop spinning first so it can exit gAAWoken.store(1u, std::memory_order_release); - R5900Context tc{}; - setRegU32(tc, 4, static_cast(tid)); - ps2_syscalls::TerminateThread(rdram.data(), &tc, &runtime); + callSyscall(runtime, rdram, ps2_syscalls::TerminateThread, static_cast(tid)); } catch (...) { @@ -7454,10 +7148,7 @@ void register_scheduler_reinit_tests() }, std::chrono::milliseconds(1000)); t.IsTrue(cycle1Done, "AA8: cycle1: both fibers exited"); - const bool cycle1Drained = waitUntil([&]() - { - return g_activeThreads.load(std::memory_order_acquire) == 0; - }, std::chrono::milliseconds(1000)); + const bool cycle1Drained = drainedWithin(std::chrono::milliseconds(1000)); t.IsTrue(cycle1Drained, "AA8: cycle1: g_activeThreads drained to 0"); ps2sched::scheduler_shutdown(); @@ -7489,10 +7180,7 @@ void register_scheduler_reinit_tests() }, std::chrono::milliseconds(1000)); t.IsTrue(cycle2Done, "AA8: cycle2: fiber exited"); - const bool cycle2Drained = waitUntil([&]() - { - return g_activeThreads.load(std::memory_order_acquire) == 0; - }, std::chrono::milliseconds(1000)); + const bool cycle2Drained = drainedWithin(std::chrono::milliseconds(1000)); t.IsTrue(cycle2Drained, "AA8: cycle2: g_activeThreads drained to 0"); ps2sched::scheduler_shutdown(); @@ -7524,8 +7212,7 @@ void register_scheduler_join_priority_tests() uint32_t zero = 0u; gAAStarted.store(0u, std::memory_order_release); gAASeq.store(0u, std::memory_order_release); - int32_t bad = -9999; - std::memcpy(rdram.data() + kAASlotResult, &bad, 4); + rdramWrite32(rdram, kAASlotResult, static_cast(-9999)); // Start target first (prio 50) const int32_t targetTid = startSchedWorker(rdram.data(), &runtime, @@ -7538,10 +7225,7 @@ void register_scheduler_join_priority_tests() } // Store target tid in kAASlotEid so stepAA9Joiner can read it - { - int32_t tv = targetTid; - std::memcpy(rdram.data() + kAASlotEid, &tv, 4); - } + rdramWrite32(rdram, kAASlotEid, static_cast(targetTid)); // Wait for target to signal it started const bool targetStarted = waitUntil([&]() @@ -7574,7 +7258,7 @@ void register_scheduler_join_priority_tests() t.Equals(finalPrio, 10, "AA9: joiner priority restored to original 10 after join"); } - waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + drainedWithin(std::chrono::milliseconds(1000)); }); }); // MiniTest::Case("SchedulerJoinPriority") @@ -7892,7 +7576,7 @@ void register_scheduler_park_window_tests() t.IsTrue(woken != 0u, "AA10: fiber observed WokenInWindow result from block_current"); } - waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + drainedWithin(std::chrono::milliseconds(1000)); }); // ------------------------------------------------------------------ @@ -7942,9 +7626,7 @@ void register_scheduler_park_window_tests() { try { - R5900Context tcx{}; - setRegU32(tcx, 4, static_cast(tid)); - ps2_syscalls::TerminateThread(rdram.data(), &tcx, &runtime); + callSyscall(runtime, rdram, ps2_syscalls::TerminateThread, static_cast(tid)); } catch (...) { hostThrew.store(true, std::memory_order_release); } hostDone.store(true, std::memory_order_release); @@ -8042,10 +7724,7 @@ void register_scheduler_park_window_tests() const auto retryDeadline = std::chrono::steady_clock::now() + std::chrono::milliseconds(1000); for (;;) { - R5900Context rc{}; - setRegU32(rc, 4, static_cast(tid)); - ps2_syscalls::ResumeThread(rdram.data(), &rc, &runtime); - if (getRegS32(rc, 2) == KE_OK) break; + if (callSyscall(runtime, rdram, ps2_syscalls::ResumeThread, static_cast(tid)) == KE_OK) break; if (std::chrono::steady_clock::now() > retryDeadline) break; } } @@ -8061,7 +7740,7 @@ void register_scheduler_park_window_tests() "T-DEF2a: every SuspendThread/ResumeThread pair completed promptly"); waitUntil([&]() { return gDef2aDone.load(std::memory_order_acquire) != 0u; }, std::chrono::milliseconds(2000)); - waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + drainedWithin(std::chrono::milliseconds(1000)); }); // ------------------------------------------------------------------ @@ -8108,7 +7787,7 @@ void register_scheduler_park_window_tests() }, std::chrono::milliseconds(2000)); t.IsTrue(returned, "T-DEF2b: suspend_self() returned promptly instead of parking forever"); - waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + drainedWithin(std::chrono::milliseconds(1000)); }); // ------------------------------------------------------------------ @@ -8237,9 +7916,8 @@ void register_scheduler_fiber_ptr_tests() runtime.registerFunction(0x00838000u, &stepRecordFiberPtr); - uint32_t zero = 0u; gAAStarted.store(0u, std::memory_order_release); - std::memcpy(rdram.data() + kAAFiberPtrSlot, &zero, 4); + rdramWrite32(rdram, kAAFiberPtrSlot, 0u); gAAWoken.store(0u, std::memory_order_release); gAASeq.store(0u, std::memory_order_release); @@ -8278,7 +7956,7 @@ void register_scheduler_fiber_ptr_tests() t.IsTrue(ptrOnResume != 0u, "AA11: ps2fiber_current() was non-null after resume"); t.Equals(ptrOnEntry, ptrOnResume, "AA11: ps2fiber_current() is the same pointer before and after yield"); - waitUntil([&](){ return g_activeThreads.load() == 0; }, std::chrono::milliseconds(1000)); + drainedWithin(std::chrono::milliseconds(1000)); }); }); // MiniTest::Case("SchedulerFiberPtr") From 6c6d714ab725b5d5344dc1728619e3179bc7b1a2 Mon Sep 17 00:00:00 2001 From: Shane Michael Mathews Date: Wed, 15 Jul 2026 15:12:03 -0400 Subject: [PATCH 11/15] fix(merge): resolve post-merge build/link fallout from upstream IOP refactor - elf_parser.h: add missing include; elfio's headers rely on transitive uint16_t/uint32_t visibility that newer libstdc++ no longer provides implicitly, which broke every elfio symbol in elf_parser.cpp. - ps2_syscalls.h / ps2_runtime.cpp: drop the joinAllGuestHostThreads()/ detachAllGuestHostThreads() declaration and destructor call that the merge resolution reintroduced from origin/main. This branch already replaced the g_hostThreads std::thread-per-guest-thread model with the fiber scheduler, so the underlying joinAllHostThreads()/ detachAllHostThreads() helpers no longer exist; the destructor comment already correctly notes that scheduler_shutdown() in run() owns fiber pool teardown now. --- ps2xRecomp/include/ps2recomp/elf_parser.h | 1 + ps2xRuntime/include/ps2_syscalls.h | 2 -- ps2xRuntime/src/lib/ps2_runtime.cpp | 5 +++-- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/ps2xRecomp/include/ps2recomp/elf_parser.h b/ps2xRecomp/include/ps2recomp/elf_parser.h index c8cafe5ee..10f9e4ce8 100644 --- a/ps2xRecomp/include/ps2recomp/elf_parser.h +++ b/ps2xRecomp/include/ps2recomp/elf_parser.h @@ -1,6 +1,7 @@ #ifndef PS2RECOMP_ELF_PARSER_H #define PS2RECOMP_ELF_PARSER_H +#include #include #include #include diff --git a/ps2xRuntime/include/ps2_syscalls.h b/ps2xRuntime/include/ps2_syscalls.h index e23795bb1..d635a2962 100644 --- a/ps2xRuntime/include/ps2_syscalls.h +++ b/ps2xRuntime/include/ps2_syscalls.h @@ -32,8 +32,6 @@ namespace ps2_syscalls void initializeGuestKernelState(uint8_t *rdram); void TODO(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime, uint32_t encodedSyscallId); void notifyRuntimeStop(); - void joinAllGuestHostThreads(); - void detachAllGuestHostThreads(); void EnsureVSyncWorkerRunning(uint8_t *rdram, PS2Runtime *runtime); uint64_t GetCurrentVSyncTick(); uint64_t WaitForNextVSyncTick(uint8_t *rdram, PS2Runtime *runtime); diff --git a/ps2xRuntime/src/lib/ps2_runtime.cpp b/ps2xRuntime/src/lib/ps2_runtime.cpp index 06419550b..6d14f6695 100644 --- a/ps2xRuntime/src/lib/ps2_runtime.cpp +++ b/ps2xRuntime/src/lib/ps2_runtime.cpp @@ -565,8 +565,9 @@ PS2Runtime::~PS2Runtime() try { requestStop(); - // Fiber pool is cleaned up by scheduler_shutdown() in run(). - ps2_syscalls::detachAllGuestHostThreads(); + // Fiber pool is cleaned up by scheduler_shutdown() in run(); there is + // no longer a g_hostThreads map to detach (superseded by the fiber + // scheduler), so no detachAllGuestHostThreads() call is needed here. m_iopSubsystem.reset(); m_iopHost.reset(); #if defined(PLATFORM_VITA) From 69a2154384dbf4936c6e6e94d4b0a6ba2f05e2bd Mon Sep 17 00:00:00 2001 From: Shane Michael Mathews Date: Wed, 15 Jul 2026 16:12:30 -0400 Subject: [PATCH 12/15] fix(test): remove vsync-tick re-sample race in sceGsSyncV interlaced field-parity test The sceGsSyncV interlaced field-parity test reconstructed the tick each sceGsSyncV call consumed by calling GetCurrentVSyncTick() separately after the call returned. The vsync worker free-runs on a wall-clock timer, so the global tick counter can advance between sceGsSyncV returning and that re-sample, and the slop's parity is not stable across the two calls. On a loaded windows-msvc CI runner the test thread was descheduled between return and re-sample with differing parity, breaking the field0 ^ (delta & 1) cross-check even though sceGsSyncV's field logic was correct. Fast, unloaded linux-clang/linux-gcc runners never hit the window, so it passed there. Record the exact consumed tick inside sceGsSyncV (atomically with, and from the same value as, the field it computes) and expose it via lastGsSyncVConsumedTick() for tests to read, instead of re-sampling the racy free-running counter. This removes the non-determinism at its source rather than loosening a tolerance; the XOR-by-parity cross-check still guards the field-derivation logic against real regressions. --- ps2xRuntime/src/lib/Kernel/Stubs/GS.cpp | 24 ++++++++++++++++ ps2xRuntime/src/lib/Kernel/Stubs/GS.h | 1 + ps2xTest/src/ps2_gs_tests.cpp | 38 ++++++++++++++----------- 3 files changed, 47 insertions(+), 16 deletions(-) diff --git a/ps2xRuntime/src/lib/Kernel/Stubs/GS.cpp b/ps2xRuntime/src/lib/Kernel/Stubs/GS.cpp index edb5a421d..90c45a8d9 100644 --- a/ps2xRuntime/src/lib/Kernel/Stubs/GS.cpp +++ b/ps2xRuntime/src/lib/Kernel/Stubs/GS.cpp @@ -10,6 +10,7 @@ namespace ps2_stubs { std::mutex g_gs_sync_v_mutex; uint64_t g_gs_sync_v_base_tick = 0u; + uint64_t g_gs_sync_v_last_tick = 0u; std::mutex g_gs_sync_v_callback_mutex; uint32_t g_gs_sync_v_callback_func = 0u; uint32_t g_gs_sync_v_callback_gp = 0u; @@ -602,6 +603,20 @@ namespace ps2_stubs { std::lock_guard lock(g_gs_sync_v_mutex); g_gs_sync_v_base_tick = ps2_syscalls::GetCurrentVSyncTick(); + g_gs_sync_v_last_tick = 0u; + } + + // Test-observability accessor: returns the exact vsync tick that the most + // recent sceGsSyncV call consumed (the value WaitForNextVSyncTick returned + // inside that call, recorded atomically with the field computation). Tests + // must use this instead of re-sampling GetCurrentVSyncTick() after the call + // returns: the vsync worker free-runs on a wall clock, so a post-call + // re-sample can observe a later tick than the one the call actually waited + // on, and the slop's parity is not stable across calls on a loaded runner. + uint64_t lastGsSyncVConsumedTick() + { + std::lock_guard lock(g_gs_sync_v_mutex); + return g_gs_sync_v_last_tick; } static int32_t getGsSyncVFieldForTick(uint64_t tick) @@ -1454,6 +1469,15 @@ namespace ps2_stubs void sceGsSyncV(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) { const uint64_t tick = ps2_syscalls::WaitForNextVSyncTick(rdram, runtime); + // Record the exact tick this call consumed, atomically with (and from + // the same value as) the field computed below, so tests can cross-check + // the reported field against the real consumed tick without racing the + // free-running vsync worker. Released before getGsSyncVFieldForTick, + // which takes the same (non-recursive) mutex. + { + std::lock_guard lock(g_gs_sync_v_mutex); + g_gs_sync_v_last_tick = tick; + } if (g_gparam.interlace != 0u) { setReturnS32(ctx, getGsSyncVFieldForTick(tick)); diff --git a/ps2xRuntime/src/lib/Kernel/Stubs/GS.h b/ps2xRuntime/src/lib/Kernel/Stubs/GS.h index 8cc2e377f..b34b4a821 100644 --- a/ps2xRuntime/src/lib/Kernel/Stubs/GS.h +++ b/ps2xRuntime/src/lib/Kernel/Stubs/GS.h @@ -5,6 +5,7 @@ namespace ps2_stubs { void resetGsSyncVCallbackState(); + uint64_t lastGsSyncVConsumedTick(); void dispatchGsSyncVCallback(uint8_t *rdram, PS2Runtime *runtime, uint64_t tick); void sceGifPkAddGsAD(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime); void sceGifPkAddGsData(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime); diff --git a/ps2xTest/src/ps2_gs_tests.cpp b/ps2xTest/src/ps2_gs_tests.cpp index 7cb150dc8..bfff59022 100644 --- a/ps2xTest/src/ps2_gs_tests.cpp +++ b/ps2xTest/src/ps2_gs_tests.cpp @@ -3373,21 +3373,27 @@ void register_ps2_gs_tests() R5900Context sync0{}; ps2_stubs::sceGsSyncV(rdram.data(), &sync0, &runtime); - // Sample the vsync tick this call consumed as close to the call as - // possible. The vsync worker (ps2xRuntime/.../Interrupt.cpp, - // interruptWorkerMain) free-runs on a wall-clock timer - // (kVblankPeriod, ~16.6ms) independent of this thread, and - // WaitForNextVSyncTick's contract is only "returns a tick strictly - // after the one you started on" -- not "exactly one tick later". - // On a loaded/virtualized CI runner (observed on windows-msvc) THIS - // test thread -- not the worker -- can be descheduled for longer - // than one vsync period between the two sceGsSyncV calls below, in - // which case the second call legitimately consumes tick N+2 (or - // later) instead of N+1, and a hardcoded "must be odd" expectation - // is simply wrong. Deriving the expected field from the observed - // tick keeps this assertion correct under that jitter while still - // catching real regressions in the field-parity logic. - const uint64_t tick0 = ps2_syscalls::GetCurrentVSyncTick(); + // Read the EXACT vsync tick this call consumed. sceGsSyncV records + // it (GS.cpp, g_gs_sync_v_last_tick) atomically with the field it + // computes, from the same tick WaitForNextVSyncTick returned. We + // must NOT re-sample GetCurrentVSyncTick() here: the vsync worker + // (ps2xRuntime/.../Interrupt.cpp, interruptWorkerMain) free-runs on + // a wall-clock timer (kVblankPeriod, ~16.6ms) independent of this + // thread, so a post-call re-sample can observe a LATER tick than the + // one the call actually waited on. On a loaded/virtualized CI runner + // (observed on windows-msvc) that slop differed in parity between the + // two calls, which broke the XOR cross-check below even though the + // field logic was correct. Reading the recorded consumed tick removes + // that race entirely. + // + // Note WaitForNextVSyncTick's contract is only "returns a tick + // strictly after the one you started on" -- not "exactly one tick + // later". If this thread is descheduled for more than one vsync + // period between the two sceGsSyncV calls, the second legitimately + // consumes tick N+2 (or later); deriving the expected field from the + // actual consumed-tick delta below keeps the assertion correct under + // that jitter while still catching real field-parity regressions. + const uint64_t tick0 = ps2_stubs::lastGsSyncVConsumedTick(); const int32_t field0 = static_cast(getRegU32Test(sync0, 2)); if (baseUnambiguous) { @@ -3396,7 +3402,7 @@ void register_ps2_gs_tests() R5900Context sync1{}; ps2_stubs::sceGsSyncV(rdram.data(), &sync1, &runtime); - const uint64_t tick1 = ps2_syscalls::GetCurrentVSyncTick(); + const uint64_t tick1 = ps2_stubs::lastGsSyncVConsumedTick(); const int32_t field1 = static_cast(getRegU32Test(sync1, 2)); // Liveness: the second call must have actually waited for a new From 07e33238c112c9fbfb1534971e9be1f5d0cfb170 Mon Sep 17 00:00:00 2001 From: Shane Michael Mathews Date: Tue, 21 Jul 2026 07:01:26 -0400 Subject: [PATCH 13/15] fix(sched): RotateThreadReadyQueue must rotate the caller into its own group rotate_ready_queue() only rotated nodes already in g_run_queue, but the currently-running fiber is popped off g_run_queue by the executor before it resumes -- so the caller, which real EE hardware treats as the head of its own priority ring, was invisible to the rotation and was never moved. Combined with maybe_yield()/yield_point() preempting only for a strictly higher priority, the one syscall whose entire purpose is round-robining equal-priority threads could not do so: a guest thread looping on RotateThreadReadyQueue(myOwnPriority) -- the standard PS2 idle/yield-thread idiom -- monopolised the N=1 cooperative executor forever and starved every other Ready fiber at that priority. When the caller's priority matches the requested priority, treat it as the ring head: enqueue it at the tail of its own priority group via enqueue_locked() and yield, gated on another fiber at that priority or better actually being runnable so a lone thread does not burn a pointless context switch. Rotating a group the caller is not in keeps the previous head-to-tail behaviour unchanged, now in an else branch. --- ps2xRuntime/include/ps2_scheduler.h | 8 ++ ps2xRuntime/src/lib/ps2_scheduler.cpp | 102 ++++++++++++++++++++++---- 2 files changed, 97 insertions(+), 13 deletions(-) diff --git a/ps2xRuntime/include/ps2_scheduler.h b/ps2xRuntime/include/ps2_scheduler.h index f75e4df23..2837208d1 100644 --- a/ps2xRuntime/include/ps2_scheduler.h +++ b/ps2xRuntime/include/ps2_scheduler.h @@ -142,6 +142,14 @@ namespace ps2sched void clear_suspend(int tid); // Rotate the equal-priority group in the run queue (RotateThreadReadyQueue). + // rotate_ready_queue() is the primitive the RotateThreadReadyQueue syscall + // wrapper calls before its own trailing maybe_yield() (Thread.cpp). A fiber + // caller at its own priority is the head of that ready ring: it moves to the + // tail of its group and yields, but only when another fiber at that priority + // or better is runnable. A fiber caller rotating a different priority, or a + // host-borrowed worker with no current fiber, rotates the requested group + // without yielding here. See the definition in ps2_scheduler.cpp for the + // full case analysis, including the interrupt-context / EE-fidelity notes. void rotate_ready_queue(int priority); // --- Async guest-code borrow --- diff --git a/ps2xRuntime/src/lib/ps2_scheduler.cpp b/ps2xRuntime/src/lib/ps2_scheduler.cpp index b5a09a853..d2ebb3d65 100644 --- a/ps2xRuntime/src/lib/ps2_scheduler.cpp +++ b/ps2xRuntime/src/lib/ps2_scheduler.cpp @@ -1044,19 +1044,95 @@ void ps2sched::clear_suspend(int tid) void ps2sched::rotate_ready_queue(int priority) { - std::lock_guard lk(g_sched_mutex); - FiberContext** pp = &g_run_queue; - while (*pp && (*pp)->priority != priority) - pp = &(*pp)->next; - if (!*pp) return; - FiberContext* victim = *pp; // first node at this priority - *pp = victim->next; - victim->next = nullptr; - FiberContext** ins = pp; // re-insert after the last node at this priority - while (*ins && (*ins)->priority == priority) - ins = &(*ins)->next; - victim->next = *ins; - *ins = victim; + // RotateThreadReadyQueue primitive. The RUNNING fiber is not in g_run_queue + // (the executor pops it before resuming), yet on real hardware the running + // thread IS the head of its priority's ready queue. Three cases: + // + // 1. A fiber caller whose own priority == `priority` is the head of that + // ready ring: it moves to the tail of its group and YIELDS, but only if + // another fiber at that priority or better is runnable; if nothing at + // this priority or better is runnable it returns WITHOUT yielding (no + // pointless context switch). This self-enqueue is required, not + // incidental: maybe_yield() preempts only for a STRICTLY higher-priority + // fiber, so nothing else ever round-robins the caller against its + // equals. This is the whole point of the syscall -- the standard PS2 + // idle/yield-thread idiom, a guest "yield to my equals" primitive. + // 2. A fiber caller rotating a DIFFERENT priority than its own moves that + // group's head node to the tail and does NOT yield HERE. The syscall can + // still yield: the wrapper's trailing maybe_yield() (Thread.cpp) preempts + // this caller for a STRICTLY higher-priority fiber -- via the wrapper, + // never through this function. + // 3. A caller with NO current fiber (tls_current_fiber == nullptr) -- a + // host-borrowed worker running under AsyncGuestScope -- cannot match the + // caller-is-head case, so it just rotates the requested group WITHOUT + // yielding. It is the ONLY caller guaranteed not to yield regardless of + // run-queue state: the wrapper's maybe_yield() is likewise a no-op when + // tls_current_fiber == nullptr (its !fc early return). The branch below + // turns solely on tls_current_fiber, not on interrupt-vs-thread context. + // + // Interrupt-context / EE-fidelity note: an interrupt/DMAC handler can run + // INLINE on a fiber (see dispatchDmacHandlersForCause), and + // iRotateThreadReadyQueue is a direct alias of the base syscall, so an + // in-handler caller at its own priority with a runnable peer takes case 1 + // and YIELDS mid-handler. This IS an EE-fidelity deviation: real EE + // i-prefixed handlers defer any reschedule to interrupt exit and never + // context-switch mid-handler, whereas this inline-on-fiber path reschedules + // to an equal-priority peer before the handler returns. It is pinned rather + // than merely asserted: the SchedulerProtocol suite drives an + // inline-DMAC-dispatched handler that calls RotateThreadReadyQueue at its own + // priority with a queued equal-priority peer and checks the switch is + // crash/deadlock-safe and deterministic -- the peer runs to completion, then + // the handler resumes and returns on its still-live per-invocation scratch + // stack. Inline handler dispatch reserves that fresh scratch stack precisely + // to tolerate a handler body yielding at a back-edge, the same mechanism the + // other back-edge yield paths rely on; the mid-handler switch is a bounded + // extension of it, now covered by test. + FiberContext* self = tls_current_fiber; + bool yieldSelf = false; + { + std::lock_guard lk(g_sched_mutex); + + if (self != nullptr && self->priority == priority) + { + // Caller is the head of this group. Only actually hand the CPU over + // if somebody else can run at this priority or better; otherwise a + // requeue+switch would just resume us and burn fiber switches. + // self is not in g_run_queue (the executor pops the running fiber + // before resuming it), and the queue is priority-ordered, so a + // runnable peer at this priority or better exists iff the head is. + const bool peerAtPriorityOrBetterRunnable = + (g_run_queue != nullptr && g_run_queue->priority <= priority); + if (peerAtPriorityOrBetterRunnable) + { + enqueue_locked(self); // tail of our priority group, state=Ready + yieldSelf = true; + } + // The remaining same-priority nodes keep their relative order, which + // is exactly head->tail rotation with the caller as head. + } + else + { + // Rotating a group we are not part of: move that group's head node + // to the tail. No reschedule of the caller. + FiberContext** pp = &g_run_queue; + while (*pp && (*pp)->priority != priority) + pp = &(*pp)->next; + if (!*pp) return; + FiberContext* victim = *pp; // first node at this priority + *pp = victim->next; + victim->next = nullptr; + FiberContext** ins = pp; // re-insert after the last node at this priority + while (*ins && (*ins)->priority == priority) + ins = &(*ins)->next; + victim->next = *ins; + *ins = victim; + } + } + + if (yieldSelf) + { + ps2fiber_yield(); // executor sees Ready+queued and reschedules us later + } } void ps2sched::async_guest_begin() From 10c03dd801a0dfdf9887832aa53a90f43b4fe997 Mon Sep 17 00:00:00 2001 From: Shane Michael Mathews Date: Tue, 21 Jul 2026 07:01:36 -0400 Subject: [PATCH 14/15] test(sched): cover RotateThreadReadyQueue yield/no-yield for all callers Adds four SchedulerProtocol cases that drive RotateThreadReadyQueue from inside a running fiber (tls_current_fiber == self). The first three call it with the caller's own priority, exercising the new caller-is-head path rather than the existing [A,B,C]->[B,C,A] test's group-you-are-not-in fallback; the fourth rotates a foreign priority group and pins that the caller-is-head yield stays gated on the caller actually being at the rotated priority: - yield-to-peer: A rotates its own priority group with an equal-priority peer B Ready; asserts B logs first and A last, so the caller yields its ring head. Reintroducing the rotate livelock makes A log first and fails the case. - lone-caller no-yield: A is alone at its priority (empty run queue) with a host worker parked; asserts A logs before the worker, pinning the peerAtPriorityOrBetterRunnable gate's first conjunct (g_run_queue != nullptr) -- a caller with nobody else runnable must not requeue+yield. - lower-priority-peer no-yield: A is alone at its priority but a strictly-lower-priority fiber B is Ready; asserts A logs first, then the parked worker, then B, pinning the gate's second conjunct (g_run_queue->priority <= priority) -- the caller must not yield to a worse-priority fiber. - foreign-priority rotate: A (prio 10) rotates a foreign prio-20 group [C,D] while a host worker is parked; asserts A logs first (no yield), then the worker, then D and C -- the target group rotated head-to-tail while the caller kept running. Pins the entry guard's priority-equality conjunct (self->priority == priority): only a caller at its own priority takes the yield path, so a foreign-priority caller must not requeue+yield. Dropping that conjunct lets A yield and fails the case. A fifth case drives RotateThreadReadyQueue(ownPriority) from inside an inline-dispatched DMAC handler with an equal-priority peer queued and pins the inline-handler case-1 path crash/deadlock-safe and deterministic: the peer runs first, then the handler resumes on its still-live scratch stack and returns. It is the only case exercising the is_guest_thread() inline-dispatch path, and reverting the caller-is-head rotation fails it just as it fails the thread-context yield case. --- ps2xTest/src/ps2_runtime_expansion_tests.cpp | 425 ++++++++++++++++++- 1 file changed, 422 insertions(+), 3 deletions(-) diff --git a/ps2xTest/src/ps2_runtime_expansion_tests.cpp b/ps2xTest/src/ps2_runtime_expansion_tests.cpp index 397fbde0a..6b46461ed 100644 --- a/ps2xTest/src/ps2_runtime_expansion_tests.cpp +++ b/ps2xTest/src/ps2_runtime_expansion_tests.cpp @@ -2370,7 +2370,7 @@ void register_scheduler_tests() } // --------------------------------------------------------------------------- -// Scheduler protocol tests — 19-test suite +// Scheduler protocol tests // --------------------------------------------------------------------------- namespace @@ -2532,7 +2532,7 @@ namespace } // ----------------------------------------------------------------------- - // Step functions for the 19 protocol tests + // Step functions for the protocol tests // All use the atomic gSeq counter (release/acquire) to publish per-step // ----------------------------------------------------------------------- @@ -2754,10 +2754,111 @@ namespace ctx->pc = 0u; } + // stepRotateOwnPrioThenLog: yield to equal-priority peers via + // RotateThreadReadyQueue(0) (0 => caller's own priority), then log self. + // Pre-fix the rotate never moved the caller, so the caller kept the + // executor and logged before any equal-priority peer ran. + static void stepRotateOwnPrioThenLog(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + R5900Context sc{}; + setRegU32(sc, 4, 0u); // prio 0 => use caller's own current priority + ps2_syscalls::RotateThreadReadyQueue(rdram, &sc, runtime); + const uint32_t seq = gSeq.load(std::memory_order_relaxed); + int32_t tid = g_currentThreadId; + std::memcpy(rdram + kRunLog + seq * 4, &tid, 4); + gSeq.fetch_add(1u, std::memory_order_release); // RMW: preserves release-sequence chaining + ctx->pc = 0u; + } + + // stepSpinForHostWorkerThenRotate: exercise the peerAtPriorityOrBetterRunnable==false + // short-circuit. This fiber is the ONLY fiber at its priority, so + // RotateThreadReadyQueue(ownPriority) must NOT requeue+yield -- it returns and + // the caller keeps running. To make the (absent) yield observable, a host + // worker is parked in async_guest_begin() first: if the rotate wrongly + // yielded, the executor's fairness deferral would hand the guest token to that + // worker, which would then log BEFORE this caller. With the short-circuit + // intact the caller logs first and only then does the worker get the token. + static void stepSpinForHostWorkerThenRotate(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + gStartedFlag.store(1u, std::memory_order_release); + // Wait until the host worker has entered async_guest_begin() and parked, + // so it is a candidate for the executor's fairness handoff the instant we + // yield. host_token_waiters() is a lock-free atomic load; spinning here is + // safe because this fiber owns the single executor slot. + while (ps2sched::host_token_waiters() == 0) + { + std::this_thread::yield(); + } + R5900Context sc{}; + setRegU32(sc, 4, 0u); // prio 0 => caller's own current priority + ps2_syscalls::RotateThreadReadyQueue(rdram, &sc, runtime); + const uint32_t seq = gSeq.load(std::memory_order_relaxed); + int32_t tid = g_currentThreadId; + std::memcpy(rdram + kRunLog + seq * 4, &tid, 4); + gSeq.fetch_add(1u, std::memory_order_release); // RMW: preserves release-sequence chaining + ctx->pc = 0u; + } + + // stepSpinForHostWorkerThenRotateForeign: case-2 pin. This fiber (prio 10) + // rotates a DIFFERENT priority group (20) than its own. rotate_ready_queue's + // case 2 must NOT requeue+yield the caller -- only a caller at its OWN + // priority takes the yield path. Same host-worker lever as T3c/T3d: if the + // caller wrongly yielded, the parked worker would get the guest token and log + // first. + static void stepSpinForHostWorkerThenRotateForeign(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + gStartedFlag.store(1u, std::memory_order_release); + while (ps2sched::host_token_waiters() == 0) + { + std::this_thread::yield(); + } + R5900Context sc{}; + setRegU32(sc, 4, 20u); // rotate the prio-20 group -- foreign to this prio-10 caller + ps2_syscalls::RotateThreadReadyQueue(rdram, &sc, runtime); + const uint32_t seq = gSeq.load(std::memory_order_relaxed); + int32_t tid = g_currentThreadId; + std::memcpy(rdram + kRunLog + seq * 4, &tid, 4); + gSeq.fetch_add(1u, std::memory_order_release); // RMW: preserves release-sequence chaining + ctx->pc = 0u; + } + + // T3f: the inline-interrupt-handler case-1 pin. A DMAC handler dispatched + // INLINE on a fiber (is_guest_thread() true) calls RotateThreadReadyQueue at + // its own priority with an equal-priority peer queued. Case 1 requeues the + // caller and yields MID-HANDLER; the peer runs to completion, then the + // handler resumes on its still-live per-invocation scratch stack and returns. + // This is the EE-fidelity deviation the review flagged: exercised here so it + // is a tested behaviour, not an untested assertion. Private DMAC cause so it + // cannot collide with the workload suite's cause-5/6 handlers. + static constexpr uint32_t kT3fDmacCause = 3u; + + // Fiber A entry: dispatch the DMAC handler inline on this fiber, then exit. + static void stepDispatchInlineDmac(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + ps2_syscalls::dispatchDmacHandlersForCause(rdram, runtime, kT3fDmacCause); + ctx->pc = 0u; + } + + // Runs AS the inline DMAC handler on fiber A. Rotate A's own priority group + // (prio 0 => caller's current priority) with equal-priority peer B queued: + // case 1 requeues A and yields mid-handler. B runs and exits, then A resumes + // HERE (inside runHandlers, scratch stack still live) and logs its own tid. + static void stepHandlerRotateOwnPrioThenLog(uint8_t *rdram, R5900Context *ctx, PS2Runtime *runtime) + { + R5900Context sc{}; + setRegU32(sc, 4, 0u); // prio 0 => caller's own current priority (A = 10) + ps2_syscalls::RotateThreadReadyQueue(rdram, &sc, runtime); + const uint32_t seq = gSeq.load(std::memory_order_relaxed); + int32_t tid = g_currentThreadId; // still A's tid: same fiber, inline dispatch + std::memcpy(rdram + kRunLog + seq * 4, &tid, 4); + gSeq.fetch_add(1u, std::memory_order_release); // RMW: preserves release-sequence chaining + ctx->pc = 0u; // handler returns + } + } // anonymous namespace // --------------------------------------------------------------------------- -// 19-test scheduler protocol suite +// scheduler protocol suite // --------------------------------------------------------------------------- void register_scheduler_protocol_tests() { @@ -2924,6 +3025,324 @@ void register_scheduler_protocol_tests() drainedWithin(std::chrono::milliseconds(1000)); }); + // ------------------------------------------------------------------ + // Test 3b: RotateThreadReadyQueue(ownPriority) yields to an + // equal-priority peer (the caller is the head of its own ring). + // Pre-fix the caller never moved and monopolised the N=1 executor; + // it would log first here (livelock), failing this test. + // ------------------------------------------------------------------ + tc.Run("RotateThreadReadyQueue(ownPriority) yields to an equal-priority peer", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + runtime.registerFunction(0x00622000u, &stepRotateOwnPrioThenLog); + runtime.registerFunction(0x00620000u, &stepLogAndExit); + + rdramSeqReset(rdram); + std::memset(rdram.data() + kRunLog, 0, 32u); + + // Lock out the executor so both fibers are Ready before either runs. + ps2sched::async_guest_begin(); + + // A (head, prio 10) rotates its own priority group then logs. + // B (prio 10) just logs. Both created at the same priority; A first + // so A is the ready-queue head. + const int32_t tidA = startSchedWorker(rdram.data(), &runtime, 0x00622000u, 10, 0x00520000u, 0x2000u); + const int32_t tidB = startSchedWorker(rdram.data(), &runtime, 0x00620000u, 10, 0x00522000u, 0x2000u); + t.IsTrue(tidA > 0, "T3b: A started"); + t.IsTrue(tidB > 0, "T3b: B started"); + + // Release executor: A runs first, RotateThreadReadyQueue(own prio) + // must move A to the tail and yield to B. + ps2sched::async_guest_end(); + + const bool allDone = waitUntil([&](){ return rdramSeq(rdram) >= 2u; }, std::chrono::milliseconds(2000)); + t.IsTrue(allDone, "T3b: both fibers logged"); + + int32_t log[2] = {}; + std::memcpy(log, rdram.data() + kRunLog, 8); + t.Equals(log[0], tidB, "T3b: B runs first (A yielded its group head)"); + t.Equals(log[1], tidA, "T3b: A runs last (rotated to tail)"); + + drainedWithin(std::chrono::milliseconds(1000)); + }); + + // ------------------------------------------------------------------ + // Test 3c: RotateThreadReadyQueue(ownPriority) does NOT requeue+yield + // when the caller is alone at its priority (peerAtPriorityOrBetterRunnable==false). + // A lone fiber calls the syscall while a host worker is parked in + // async_guest_begin(). The short-circuit means the caller keeps the slot, + // logs, and finishes BEFORE the parked worker is granted the token. If the + // guard were removed the rotate would yield, the executor would hand the + // token to the worker (fairness), and the worker would log first -- the + // "always requeue+yield" regression this pins. + // ------------------------------------------------------------------ + tc.Run("RotateThreadReadyQueue(ownPriority) does not yield when caller is alone at its priority", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + static constexpr int32_t kHostWorkerMark = 0x00A5A5A5; + + runtime.registerFunction(0x00626000u, &stepSpinForHostWorkerThenRotate); + + rdramSeqReset(rdram); + std::memset(rdram.data() + kRunLog, 0, 32u); + gStartedFlag.store(0u, std::memory_order_release); + + // A (prio 10) is the only fiber. Start it and let it run up to the + // spin, at which point it owns the single executor slot. + const int32_t tidA = startSchedWorker(rdram.data(), &runtime, 0x00626000u, 10, 0x00526000u, 0x2000u); + t.IsTrue(tidA > 0, "T3c: A started"); + + const bool running = waitUntil([&]() + { + return gStartedFlag.load(std::memory_order_acquire) != 0u; + }, std::chrono::milliseconds(2000)); + t.IsTrue(running, "T3c: A is running (owns the executor slot)"); + + // Park a host worker in async_guest_begin(): it cannot get the token + // while A owns the slot, so it waits. Its presence is what A's rotate + // would wrongly yield to if the short-circuit were removed. When it + // finally gets the token it logs a sentinel and releases. + std::thread worker([&]() + { + ps2sched::async_guest_begin(); + const uint32_t seq = gSeq.load(std::memory_order_relaxed); + int32_t mark = kHostWorkerMark; + std::memcpy(rdram.data() + kRunLog + seq * 4, &mark, 4); + gSeq.fetch_add(1u, std::memory_order_release); + ps2sched::async_guest_end(); + }); + + const bool bothLogged = waitUntil([&]() + { + return rdramSeq(rdram) >= 2u; + }, std::chrono::milliseconds(2000)); + t.IsTrue(bothLogged, "T3c: caller and host worker both logged"); + + worker.join(); + + int32_t log[2] = {}; + std::memcpy(log, rdram.data() + kRunLog, 8); + t.Equals(log[0], tidA, "T3c: lone caller logs first (no requeue+yield)"); + t.Equals(log[1], kHostWorkerMark, "T3c: parked host worker logs only after the caller finishes"); + + drainedWithin(std::chrono::milliseconds(1000)); + }); + + // ------------------------------------------------------------------ + // Test 3d: RotateThreadReadyQueue(ownPriority) does NOT yield to a + // strictly-LOWER-priority ready fiber. The caller is alone at its own + // priority, but a worse-priority fiber is Ready in the queue. The + // peerAtPriorityOrBetterRunnable guard must still short-circuit: it is + // the "at this priority OR BETTER" clause -- not merely "the queue is + // non-empty" -- that this pins. Weakening the guard to a bare non-null + // g_run_queue check passes T3, T3b and T3c but fails here. Same + // host-worker lever as T3c: a wrongful yield hands the guest token to + // the parked worker (executor fairness), which then logs first. + // ------------------------------------------------------------------ + tc.Run("RotateThreadReadyQueue(ownPriority) does not yield to a strictly-lower-priority ready fiber", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + static constexpr int32_t kHostWorkerMark = 0x00A5A5A5; + + runtime.registerFunction(0x00628000u, &stepSpinForHostWorkerThenRotate); + runtime.registerFunction(0x00620000u, &stepLogAndExit); + + rdramSeqReset(rdram); + std::memset(rdram.data() + kRunLog, 0, 32u); + gStartedFlag.store(0u, std::memory_order_release); + + // A (prio 10) is the only fiber at its priority. Start it and let it + // run up to the spin, at which point it owns the single executor slot. + const int32_t tidA = startSchedWorker(rdram.data(), &runtime, 0x00628000u, 10, 0x00528000u, 0x2000u); + t.IsTrue(tidA > 0, "T3d: A started"); + + const bool running = waitUntil([&]() + { + return gStartedFlag.load(std::memory_order_acquire) != 0u; + }, std::chrono::milliseconds(2000)); + t.IsTrue(running, "T3d: A is running (owns the executor slot)"); + + // B (prio 20 -- strictly WORSE than A's 10) is Ready in the run queue + // but cannot run while A owns the slot. It is the strictly-lower- + // priority fiber A must NOT yield to. Started only after A is confirmed + // running, so the executor (parked inside A's spin) leaves B queued. + const int32_t tidB = startSchedWorker(rdram.data(), &runtime, 0x00620000u, 20, 0x00522000u, 0x2000u); + t.IsTrue(tidB > 0, "T3d: B started (lower priority, Ready)"); + + // Park a host worker in async_guest_begin(): its presence is what A's + // rotate would wrongly yield to if the guard decayed to bare non-null. + std::thread worker([&]() + { + ps2sched::async_guest_begin(); + const uint32_t seq = gSeq.load(std::memory_order_relaxed); + int32_t mark = kHostWorkerMark; + std::memcpy(rdram.data() + kRunLog + seq * 4, &mark, 4); + gSeq.fetch_add(1u, std::memory_order_release); + ps2sched::async_guest_end(); + }); + + const bool allLogged = waitUntil([&]() + { + return rdramSeq(rdram) >= 3u; + }, std::chrono::milliseconds(2000)); + t.IsTrue(allLogged, "T3d: caller, host worker and B all logged"); + + worker.join(); + + int32_t log[3] = {}; + std::memcpy(log, rdram.data() + kRunLog, 12); + t.Equals(log[0], tidA, "T3d: lone-at-priority caller logs first (no yield to a worse-priority fiber)"); + t.Equals(log[1], kHostWorkerMark, "T3d: parked host worker logs only after the caller finishes"); + t.Equals(log[2], tidB, "T3d: the worse-priority fiber runs last"); + + drainedWithin(std::chrono::milliseconds(1000)); + }); + + // ------------------------------------------------------------------ + // Test 3e: RotateThreadReadyQueue(foreignPriority) rotates the target + // group head-to-tail but does NOT yield the calling fiber. Pins the + // case-1 entry guard's priority-equality conjunct: only a caller AT ITS + // OWN priority takes the yield path. Dropping `self->priority == priority` + // from the guard (so any fiber caller yields) passes T3/T3b/T3c/T3d -- + // none has a foreign-priority fiber caller -- but fails here. Same + // host-worker lever as T3c/T3d: a wrongful yield hands the guest token to + // the parked worker, which then logs before the caller. + // ------------------------------------------------------------------ + tc.Run("RotateThreadReadyQueue(foreignPriority) rotates the target group without yielding the caller", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + static constexpr int32_t kHostWorkerMark = 0x00A5A5A5; + + runtime.registerFunction(0x0062A000u, &stepSpinForHostWorkerThenRotateForeign); + runtime.registerFunction(0x00620000u, &stepLogAndExit); + + rdramSeqReset(rdram); + std::memset(rdram.data() + kRunLog, 0, 32u); + gStartedFlag.store(0u, std::memory_order_release); + + // A (prio 10) is the only fiber at its priority. Run it up to the + // spin, at which point it owns the single executor slot. + const int32_t tidA = startSchedWorker(rdram.data(), &runtime, 0x0062A000u, 10, 0x0052A000u, 0x2000u); + t.IsTrue(tidA > 0, "T3e: A started"); + + const bool running = waitUntil([&]() + { + return gStartedFlag.load(std::memory_order_acquire) != 0u; + }, std::chrono::milliseconds(2000)); + t.IsTrue(running, "T3e: A is running (owns the executor slot)"); + + // C, D (prio 20) form the FOREIGN group A rotates. Started only after + // A is confirmed running so the executor (parked in A's spin) leaves + // them queued. Creation order C then D -> group [C, D]. + const int32_t tidC = startSchedWorker(rdram.data(), &runtime, 0x00620000u, 20, 0x0052C000u, 0x2000u); + const int32_t tidD = startSchedWorker(rdram.data(), &runtime, 0x00620000u, 20, 0x0052E000u, 0x2000u); + t.IsTrue(tidC > 0, "T3e: C started"); + t.IsTrue(tidD > 0, "T3e: D started"); + + // Park a host worker: its presence is what A's rotate would wrongly + // yield to if the entry guard dropped the priority-equality conjunct. + std::thread worker([&]() + { + ps2sched::async_guest_begin(); + const uint32_t seq = gSeq.load(std::memory_order_relaxed); + int32_t mark = kHostWorkerMark; + std::memcpy(rdram.data() + kRunLog + seq * 4, &mark, 4); + gSeq.fetch_add(1u, std::memory_order_release); + ps2sched::async_guest_end(); + }); + + const bool allLogged = waitUntil([&]() + { + return rdramSeq(rdram) >= 4u; + }, std::chrono::milliseconds(2000)); + t.IsTrue(allLogged, "T3e: caller, host worker and both group fibers logged"); + + worker.join(); + + int32_t log[4] = {}; + std::memcpy(log, rdram.data() + kRunLog, 16); + t.Equals(log[0], tidA, "T3e: foreign-priority caller logs first (no yield)"); + t.Equals(log[1], kHostWorkerMark, "T3e: parked host worker logs only after the caller finishes"); + t.Equals(log[2], tidD, "T3e: rotated group head-to-tail (D now runs first)"); + t.Equals(log[3], tidC, "T3e: original head C moved to tail runs last"); + + drainedWithin(std::chrono::milliseconds(1000)); + }); + + // ------------------------------------------------------------------ + // Test 3f: RotateThreadReadyQueue(ownPriority) from INSIDE an inline + // DMAC handler. The is_guest_thread() dispatch branch runs the handler + // on the calling fiber; case 1 yields mid-handler to an equal-priority + // peer. Pins that this EE-fidelity deviation is crash/deadlock-safe and + // deterministic: the peer runs first, then the handler resumes on its + // still-live scratch stack and returns. Reverting the caller-is-head + // rotation (old single-loop body) makes A log before B and fails this, + // exactly as it does T3b. + // ------------------------------------------------------------------ + tc.Run("RotateThreadReadyQueue(ownPriority) from an inline DMAC handler yields to an equal-priority peer and resumes safely", [](TestCase &t) + { + SchedFixture fx; + PS2Runtime &runtime = fx.runtime; + std::vector &rdram = fx.rdram; + + constexpr uint32_t kEntryA = 0x00624000u; // A: inline-dispatch entry + constexpr uint32_t kHandlerA = 0x00626000u; // A's DMAC handler body + + runtime.registerFunction(kEntryA, &stepDispatchInlineDmac); + runtime.registerFunction(kHandlerA, &stepHandlerRotateOwnPrioThenLog); + runtime.registerFunction(0x00620000u, &stepLogAndExit); // B: log + exit + + // Register the handler on the private cause; AddDmacHandler enables it. + int32_t hid = -1; + { R5900Context a{}; setRegU32(a, 4, kT3fDmacCause); setRegU32(a, 5, kHandlerA); + setRegU32(a, 6, 0u); setRegU32(a, 7, 0u); + ps2_syscalls::AddDmacHandler(rdram.data(), &a, &runtime); hid = getRegS32(a, 2); } + t.IsTrue(hid > 0, "T3f: inline DMAC handler registered"); + + rdramSeqReset(rdram); + std::memset(rdram.data() + kRunLog, 0, 32u); + + // Lock out the executor so both fibers are Ready before either runs, + // and so NO host worker lingers to contend for the token at the yield. + ps2sched::async_guest_begin(); + + // A (head, prio 10) dispatches inline then its handler rotates+yields. + // B (prio 10) just logs. A created first => A is the ready-queue head. + const int32_t tidA = startSchedWorker(rdram.data(), &runtime, kEntryA, 10, 0x00524000u, 0x2000u); + const int32_t tidB = startSchedWorker(rdram.data(), &runtime, 0x00620000u, 10, 0x00526000u, 0x2000u); + t.IsTrue(tidA > 0, "T3f: A started"); + t.IsTrue(tidB > 0, "T3f: B started"); + + ps2sched::async_guest_end(); // release: A runs, dispatches inline, handler yields to B + + const bool allDone = waitUntil([&](){ return rdramSeq(rdram) >= 2u; }, std::chrono::milliseconds(2000)); + t.IsTrue(allDone, "T3f: peer and caller both logged (no deadlock)"); + + int32_t log[2] = {}; + std::memcpy(log, rdram.data() + kRunLog, 8); + t.Equals(log[0], tidB, "T3f: equal-priority peer runs first (caller yielded mid-handler)"); + t.Equals(log[1], tidA, "T3f: caller resumes after the peer and returns from the handler"); + + const bool drained = drainedWithin(std::chrono::milliseconds(1000)); + t.IsTrue(drained, "T3f: both fibers completed and the executor drained (mid-handler switch is crash/deadlock-safe)"); + + // Cleanup the process-global DMAC handler so later suites are clean. + { R5900Context r{}; setRegU32(r, 4, kT3fDmacCause); setRegU32(r, 5, static_cast(hid)); + ps2_syscalls::RemoveDmacHandler(rdram.data(), &r, &runtime); } + }); + // ------------------------------------------------------------------ // Test 4: ChangeThreadPriority re-sorts Ready fiber // ------------------------------------------------------------------ From c7b2edc60ff32771ab382a3224dd28ef8e8e0eeb Mon Sep 17 00:00:00 2001 From: Shane Michael Mathews Date: Wed, 22 Jul 2026 16:26:35 -0400 Subject: [PATCH 15/15] fix(android): select the pthread fiber backend (bionic lacks ucontext) --- ps2xRuntime/CMakeLists.txt | 8 ++++++++ ps2xRuntime/src/lib/ps2_fiber.cpp | 5 +++-- 2 files changed, 11 insertions(+), 2 deletions(-) diff --git a/ps2xRuntime/CMakeLists.txt b/ps2xRuntime/CMakeLists.txt index b782a3cad..76fad0c53 100644 --- a/ps2xRuntime/CMakeLists.txt +++ b/ps2xRuntime/CMakeLists.txt @@ -546,6 +546,14 @@ if(PS2X_FIBER_PTHREAD) # Threads::Threads already linked above for all non-Vita builds. endif() +if(PS2X_IS_ANDROID) + # Bionic libc implements no getcontext/makecontext/swapcontext on any ABI, + # so the default POSIX ucontext fiber backend cannot link on Android; + # route it to the pthread backend above instead (Bionic-available + # primitives only, already fully wired into the scheduler). + target_compile_definitions(ps2_runtime PUBLIC PS2X_FIBER_PTHREAD) +endif() + target_link_libraries(ps2EntryRunner PRIVATE ps2_runtime diff --git a/ps2xRuntime/src/lib/ps2_fiber.cpp b/ps2xRuntime/src/lib/ps2_fiber.cpp index cbc38a463..affc3124d 100644 --- a/ps2xRuntime/src/lib/ps2_fiber.cpp +++ b/ps2xRuntime/src/lib/ps2_fiber.cpp @@ -1,5 +1,6 @@ -// Backend dispatch: PLATFORM_VITA -> SceFiber; PS2X_FIBER_PTHREAD -> pthread semaphore; -// _WIN32 -> Win32 Fibers; else -> POSIX ucontext +// Backend dispatch: PLATFORM_VITA -> SceFiber; PS2X_FIBER_PTHREAD (also selected for +// Android, whose Bionic libc lacks ucontext) -> pthread semaphore; _WIN32 -> Win32 +// Fibers; else -> POSIX ucontext #include "ps2_fiber.h" #include #include