Skip to content

feat: modes, environments, and a plugin host - #47

Merged
Drownek merged 15 commits into
Drownek:v3-devfrom
monikon22:pr/1-mode-architecture
Aug 23, 2026
Merged

feat: modes, environments, and a plugin host#47
Drownek merged 15 commits into
Drownek:v3-devfrom
monikon22:pr/1-mode-architecture

Conversation

@monikon22

Copy link
Copy Markdown
Contributor

Thanks for creating v3-dev — this is the first of the six, targeted there.

The biggest change in the series, and the one everything after it depends on. It replaces the assumption that a run means "one Paper server that plugwright downloaded and started" with a registry of environment modes, and gives the runner a real extension point.

Rebased onto v3-dev at 8a8b570 (so it sits directly on top of the merged #45), no conflicts, tsc --noEmit clean on the runner package. The two follow-up commits you took in #45 touch utils.ts and wrappers.ts, neither of which this PR modifies.

What changes

The gradle plugin splits into modules. plugwright-api holds the types a third-party mode compiles against, plugwright-core the shared machinery, plugwright-local the mode that downloads and runs Paper, plugwright-external a mode that connects to a server it does not own. plugwright-bundle is the module published under the plugin id; its jar embeds the others, which have no coordinates of their own.

A build declares environments instead of one server.

plugwright {
    primaryEnvironment.set("local")

    environments {
        create("local", LocalMode) {
            minecraftVersion.set("1.21.11")
            acceptEula.set(true)
        }
        create("stand", ExternalMode) { /* a server started by hand */ }
    }
}

PlugwrightMatrixTask runs the suite across them, writes JSON and JUnit reports per environment, and filters on requires and environments.

Configuration moves to a config.json transport. The gradle side writes what the runner needs; the runner stops reading gradle-shaped properties. Secrets travel as references and are resolved when they are used, not when the file is written.

The runner loses its module-level singletons. Session owns what was global, and Environment / ServerConsole abstract "the server" so a mode that owns no process can still op a player and read the console. ExternalMode does that through an admin bot and an account pool, with ping and cleanup entry points so an unowned server can be checked and put back.

Message buffers are the exception, and that is the one place where this series changed shape after your c4a89a4 — see below.

A plugin host. Hooks, fixtures, custom matchers, inherited tests and a cleanup journal, so a plugin can extend the test context rather than only observe it.

Meeting c4a89a4 and d0289e6

Both of those are addressed rather than merged around, and one of them changed my mind.

The message buffer is yours, not mine. An earlier draft of this PR moved the global messageBuffer onto Session. That was the wrong scope and your commit is what made it obvious: per-session is still one buffer shared by every bot, so expect(playerA).toHaveReceivedMessage(...) could be satisfied by something player B heard. Renaming the global is not fixing it.

So the buffer is per PlayerWrapper, as you had it, and Session keeps only what is genuinely one-per-session — the bot list, the disconnect lifecycle, consoleLog. Three things fell out:

  • AdminBotConsole was hand-rolling a private buffer and its own message listener specifically to stay out of the shared one, and PlayerWrapper.join() then registered a second listener on the same bot. It now reads player.messageBuffer and the duplicate is gone.
  • The AuthMe plugin's prompt-matching reads its own player's buffer, so two bots authenticating at once can no longer answer each other's prompt (that lands in PR 2).
  • disconnectAllBots delegates to disconnectBot instead of duplicating its promise logic, so your removeAllListeners cleanup happens on every path rather than one of them. PR 5 extends the same method with a keep list; it extends the delegating version.

There is one case your patch does not reach, because the feature that creates it is in PR 5: "clear on rejoin" never fires for a player checked out under stay, which never left and so never rejoins. That is fixed there, not here.

The Gradle refactor mostly agrees with this series already. Shared task setup, typed TaskProviders, the verification group and the banner all exist here through AbstractNodeTask and PlugwrightCompileTestsTask — arrived at independently, so no conflict worth reporting.

Two that did need a decision:

  • The IDEA sync trigger would have been dropped silently by the module split, since it hangs off a task that no longer exists under that name. It is carried over in its own commit, still behind plugins.withId("idea"), and now triggers the compile task — so a sync installs and compiles.
  • Your up-to-date checks on plugwrightNpmInstall are not carried over, and I want to flag that rather than have you find it. That task merged into PlugwrightCompileTestsTask, which declares outputs.upToDateWhen { false } on purpose: its output depends on node_modules and on the installed runner package, neither of which is a declared input, so claiming up-to-date risks serving stale compiled specs. Making it properly incremental means declaring those inputs, which is worth doing and is not something I wanted to rush inside a rebase.

Where to start reading

plugwright-api first — it is small and it is the contract. Then RunnerConfigWriter.kt and runner-package/lib/config.ts for the two ends of the transport, then session.ts.

What it does not do

No behaviour changes for an existing single-environment build. The pre-3.0 flat properties (minecraftVersion, jvmArgs, acceptEula, …) still work and still describe an implicit local environment; they are marked deprecated and read only when there is no environments { } block.

The mode API and the plugin host contract are both public surface, and I do not think either is finished — that is the point of building them on v3-dev rather than shipping them to 2.x users.

How to merge this one

Please use "Create a merge commit" or "Rebase and merge" — not "Squash and merge".

PRs 2 through 6 are stacked on this branch's exact commits. A squash would replace all 13 with one new commit, and then every later PR would carry the same changes under different SHAs: PR 2 would show ~85 files instead of its own 49, and I would have to rebase-and-resolve the whole remaining stack by hand after each merge. A merge commit (the way #45 went in) or a rebase-merge both keep the SHAs, and PR 2 then shows only its own 15 commits against v3-dev.

If you would rather have a tidier history on v3-dev, squashing is fine on the last one — by then there is nothing stacked behind it. Or squash the whole series into one commit when v3-dev eventually merges into master, which gets you a clean master without breaking the stack along the way.

Once this is in, I will retarget PR 2 to v3-dev and open it. Same for the rest, in order.

Part of #46.

monikon22 and others added 13 commits August 22, 2026 16:51
…transport

The build script and the runner talked through five flat env vars, which leaves
no room for a second environment. Phase 1 of the multi-mode work rearranges the
plumbing without changing behaviour:

- gradle-plugin becomes a multi-project build: plugwright-api holds the contract
  third-party modes compile against (PlugwrightMode, EnvironmentSpec, SecretRef,
  ConfigNode, RunnerPackageRef, TaskRegistrationContext), plugwright-core holds
  the plugin. api has no coordinates of its own yet, so its classes are merged
  into the core jar; the published artifactId changes to plugwright-core, the
  plugin id and its marker do not.
- plugwrightTest writes build/tmp/plugwright/local.json and passes it as
  --config. The runner resolves config in the order --config file,
  plugwright.config.json, then the old env vars, so an older plugin still drives
  a newer runner. Host, port, jvm args and the tests dir come from the file
  instead of being hardcoded in runner.ts.
- npm install and tsc move out of the test task into plugwrightCompileTests, so
  several environments can share one install.
- Process and Node.js plumbing moves to AbstractNodeTask, shared by the test,
  run-server and compile-tests tasks.

Secrets travel as references ({"from":"env","name":...}) and are read by the
runner, never resolved at configuration time.
…ment/ServerConsole

Phase 2 of the multi-mode runner redesign. local mode keeps its exact
behavior (spawn Paper, wait for "Done (", stdio console, process-tree
kill guards) but now lives behind the Environment/ServerConsole
contracts instead of being runner.ts's only code path.

- lib/session.ts: Session + MessageBuffer replace the module-level
  activeBots/messageBuffer/serverConsoleBuffer singletons that made it
  impossible to run two environments in one process.
- lib/environment.ts, lib/console.ts: Environment and ServerConsole
  interfaces.
- lib/environments/local.ts: LocalEnvironment + StdioConsole, carrying
  over spawn/waitForServerStart/killServerTree/teardown unchanged.
- PlayerWrapper and ServerWrapper now hold a session reference instead
  of importing module state; matchers.ts reads buffers off that
  reference instead of module imports.
- testRegistry/scopeStack stay module-level (documented why in
  session.ts) — still correct for one environment per process.

Public API (test/opTest/describe/expect/PlayerWrapper/ServerWrapper/
wrappers) is unchanged. Verified: tsc --noEmit clean, full build clean,
all 46 example_plugin e2e tests pass under local mode.
Phase 3 of the multi-mode architecture:

- plugwright-api: PlugwrightMode gains applyLegacyDefaults() for seeding
  an implicit environment from deprecated flat properties; TaskRegistrationContext
  gains environmentConfig() so a mode can hand over its runner-config node
  computed lazily at task execution time. New RunDirFile and
  LegacyEnvironmentProperties types.
- plugwright-core: PlugwrightExtension gains registerMode()/environments{}
  DSL and primaryEnvironment; new EnvironmentContainer (mode + spec registry),
  TaskRegistrationContextImpl and ValidationContextImpl. PlugwrightPlugin is
  renamed PlugwrightCorePlugin and made fully mode-agnostic: it creates the
  implicit 'local' environment when no environments{} block is present, then
  asks each environment's mode to validate and register its own tasks.
  PlugwrightTestTask no longer hardcodes local-server specifics — it just
  writes whatever ConfigNode its mode produced.
- plugwright-local (new module): LocalMode + LocalEnvironmentSpec, and the
  local-only tasks split out of the old monolithic task base
  (PaperProvisionTask, PlugwrightCleanTask, PlugwrightRunServerTask). Also
  hosts the published io.github.drownek.plugwright plugin id for now — a
  dedicated bundle module can take that over once a second built-in mode
  exists to combine with it.

Task names are now generated per environment (plugwrightTestLocal,
plugwrightCleanLocal, ...), with bare aliases (plugwrightTest, ...) pointing
at whatever environment is primaryEnvironment (defaults to 'local'). Builds
with no environments{} block behave exactly as before, verified by the full
example_plugin e2e suite (46/46 passing).
…environments filters

- plugwrightTest is now the matrix task: runs every environment with
  includeInMatrix=true through the same RunnerLauncher as
  plugwrightTest<Env>, aggregates a summary, fails the build on any
  non-allowFailure environment. -Pplugwright.env=a,b narrows it.
- matrix { parallel; maxParallel } runs environments concurrently
  (off by default), each with its own build/reports/plugwright/<env>.log.
- Extracted RunnerLauncher (config write + cli.js resolution) out of
  PlugwrightTestTask so both task types share it.
- Runner writes build/reports/plugwright/<env>.json and junit/<env>.xml
  when the config carries report paths.
- test()/opTest() accept an optional {requires, environments} filter;
  skips (plus the pre-existing tests.exclude and tests.names filters,
  the latter no longer silently continue) land in results/reports with
  a reason instead of vanishing.
…ed tests, and cleanup journal

PlugwrightPlugin contract (setup/onPlayerCreate/beforeEach/afterEach/extendContext/matchers/tests/cleanup/teardown)
loaded by PluginHost from config.json's new plugins[] list. Hook order matches
modes-and-plugins §6.6: plugin.beforeEach -> spec beforeEach -> body -> cleanup
finalizers -> spec afterEach -> plugin.afterEach.

- lib/plugin.ts: PlugwrightPlugin, definePlugin, SessionContext, CleanupContext, PluginTestRef
- lib/plugin-host.ts: loads/orders plugins, merges matchers into RunnerMatchers, runs hooks
- lib/account.ts: Account type + syntheticAccount() placeholder until AccountPool (phase 6)
- lib/journal.ts: CleanupJournal, typed-record crash journal for TestContext.cleanup
- lib/test-runner.ts: runTestCase() extracted from runner.ts, sequences all the above
- test-registry.ts: TestCase now exposes raw beforeHooks/afterHooks instead of a merged fn,
  so plugin hooks can be interleaved with spec hooks by the caller
- player.ts: join()/rejoin() fire session.onPlayerCreate on every connection
- runner.ts: wires PluginHost in, runs plugin preflight tests before user specs (abort on
  failure) and suite tests alongside them, tagged with plugin name in TestResult/reports
…g transport

Extends the mode contract so a mode can declare runner plugins to load
(TaskRegistrationContext.pluginConfigs) and reach the test project's
tests directory (TaskRegistrationContext.testsDir), and wires both -
plus a per-environment crash-recovery journal path - into the runner
config alongside the existing environment/tests/reports sections.

Also adds a small secret.env(...)/secret.file(...) DSL accessor on
Project, so secret references read naturally in a build script instead
of the fully-qualified Secrets.env(...).
… ping and cleanup tasks

New plugwright-external module, mirroring plugwright-local's shape for
a mode that attaches to an already-running server instead of spawning
one:

- ExternalEnvironmentSpec: host/port/minecraftVersion (mandatory),
  joinThrottleMs, plus nested console { rcon { ... }; adminBot(...) {
  ... } }, accounts { pool { ... }; autoRegister { ... }; microsoft
  { ... } } and plugins { npm(...); local(...) } blocks.
- ExternalMode: validates the spec, serializes it into the runner
  config (secrets stay references), and pulls in the RCON runner
  package only when a rcon console block is actually declared.
- PlugwrightPingTask (plugwrightPing<Env>) and PlugwrightCleanupTask
  (plugwrightClean<Env>) run the runner in a service mode instead of
  the normal test mode - reachability/auth check, and compensating
  cleanup + journal replay, respectively.
…ng local + external

Moves PlugwrightPlugin (the io.github.drownek.plugwright entry point)
out of plugwright-local into a new plugwright-bundle module that
applies the core engine and registers both built-in modes. Mirrors
plugwright-local's old jar-merging trick, now pulling in api, core,
local and external classes since none of them publish standalone
coordinates.

plugwright-local goes back to being just a mode module - no publish
plugin, no plugin-id registration - matching plugwright-external's
shape. Published plugin id and artifact coordinates are unchanged, so
existing consumer build scripts keep working.
…le, ping/cleanup entry points

- AccountPool merges pool/autoRegister/microsoft accounts, leased per
  test and released in the test's finally block. local's account
  generation is unchanged: it only degrades to a pool when
  Environment.accounts() is implemented, which local still doesn't do.
- externalEnvironment: attaches to a running server, probes declared
  console channels in order (rcon via a dynamic import of the optional
  @plugwright/console-rcon package, else admin-bot), and honours
  joinThrottleMs on every bot connect via a new Environment.beforeJoin
  hook.
- AdminBotConsole: a second mineflayer bot with staff rights, console
  commands sent through chat, responses read from its own buffer. Its
  connection goes through PlayerWrapper.join(), so it authenticates
  through the same onPlayerCreate hook a test bot does - runner.ts now
  wires that hook before env.setup() runs so this actually applies
  during environment setup, not just afterward.
- resolveEnvironment is now async and falls back to a dynamic
  import(runtime.package) for any mode besides the two built-ins.
- runPingSession/runCleanupSession + cli.ts --ping/--cleanup: connect
  and verify the console/auth without running tests, or replay the
  crash-recovery journal via each plugin's cleanup({ scope: 'manual' })
  handler.
A task a mode registers through TaskRegistrationContext never got nodeVersion,
downloadNode or nodeInstallDir, so any of them extending AbstractNodeTask failed
validation before running.
AccountPool resolved every password in its constructor, so a run that never
connects a bot — a cleanup pass, a console-only ping — died on an unset
variable it had no use for. The ping and cleanup entry points also relied on an
unref'd timer to set the exit code, which never fires when nothing else holds
the event loop open, so a failed check reported success.
…ket decode error

mineflayer's default logErrors:true does an unconditional console.log(err) on every
bot 'error' event. A backend sending a packet type outside the client's protocol
data (e.g. an unrecognised particle) can emit that error hundreds of times a second;
logging each one synchronously starves the event loop and the piped stdout, so
timers that would otherwise fail a test fast stop firing in any useful time.

Disable mineflayer's built-in logging and replace it with a throttled one (max once
per second) that still reports total error count, keeping the connection usable
against a server that outruns minecraft-data's coverage instead of hanging tests
until their own timeout.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Upstream runs `plugwrightNpmInstall` after an IDEA sync, so a fresh checkout has
its `node_modules` before anyone opens a spec file and finds every import
unresolved. Splitting the plugin across modules moved the code that did it and
merged that task into `plugwrightCompileTests`, which would have dropped the
feature without anyone deciding to.

It now hangs off `PlugwrightCorePlugin` and triggers the compile task, which
installs and compiles in one step — so a sync leaves the workspace in a better
state than it did before rather than the same one.

Still guarded by `plugins.withId("idea")`: `idea-ext` is what carries
`afterSync`, and applying it unconditionally would push a plugin onto builds that
never asked for one. The plugin marker it compiles against comes from the Gradle
Plugin Portal, which the root build now lists alongside Maven Central.
@Drownek

Drownek commented Aug 22, 2026

Copy link
Copy Markdown
Owner

Thanks for opening this, @monikon22, the architecture direction looks good.

One problem though: I can't get IntelliJ to sync the project. It fails with:

FAILURE: Build failed with an exception.
* What went wrong:
Failed to apply plugin class 'org.gradle.plugins.ide.idea.IdeaPlugin'.
Cannot run Project.afterEvaluate(Action) when the project is already evaluated.

Probably the IDEA sync trigger got carried over in the refactor and now it's firing after the project's already evaluated.

Separate thing: Gradle's warning that the Kotlin plugin is loaded multiple times with explicit versions across the new subprojects (plugwright-api, plugwright-bundle). Usually cleaner to declare the version once in the root with apply false and skip it in the subprojects.

If the IDEA sync trigger is more trouble than it's worth, just drop it for now, no big deal. Let me know when it's updated.

IntelliJ applies the `idea` plugin to an already-evaluated project during
sync, so the plugins.withId("idea") callback ran too late for
Project.afterEvaluate and the sync failed with "Failed to apply plugin
'org.gradle.idea': Cannot run Project.afterEvaluate(Action) when the
project is already evaluated".

Register the afterSync trigger straight away when the project is already
evaluated, and keep the deferred path for the normal case.
Applying `kotlin-dsl` from every subproject's plugins block loaded the
Kotlin plugin several times, which Gradle warns is unsupported:
"The Kotlin Gradle plugin was loaded multiple times in different
subprojects ... ':plugwright-api', ':plugwright-bundle'".

Declare it once in the root build with `apply false` and hand it to the
subprojects from the shared subprojects block.
@monikon22

Copy link
Copy Markdown
Contributor Author

Thanks for testing it, both are fixed.

IDEA sync. The trigger itself was fine, the afterEvaluate wrapped around it wasn't. IntelliJ applies the idea plugin to a project that has already been evaluated, so the plugins.withId("idea") callback fires after configuration is done and Project.afterEvaluate throws from inside IdeaPlugin.apply. That's why the message names the IDEA plugin instead of us. The frame under it:

at me.drownek.plugwright.PlugwrightCorePlugin$registerIdeaSyncTrigger$1.execute(PlugwrightCorePlugin.kt:85)

It now wires afterSync right away when the project is already evaluated, and defers only while configuration is still running:

project.plugins.withId("idea") {
    project.pluginManager.apply("org.jetbrains.gradle.plugin.idea-ext")
    if (project.state.executed) {
        wireIdeaSyncTrigger(project, plugwrightCompileTests)
    } else {
        project.afterEvaluate { wireIdeaSyncTrigger(project, plugwrightCompileTests) }
    }
}

One thing worth flagging: this isn't new in this PR. The same withId("idea") { ... afterEvaluate { ... } } shape sits on v3-dev today, in PlugwrightPlugin.kt lines 345-355. I reproduced the identical failure on a clean worktree at 8a8b570 using an init script that applies idea from gradle.projectsEvaluated, which is the moment the IDE applies it. The module split carried the bug into plugwright-core, it didn't create it. So I kept the trigger instead of dropping it, since it now survives both paths.

Kotlin plugin. Done the way you suggested. kotlin-dsl is declared once in the root build with apply false and handed to the modules from the shared subprojects { } block, and the five module scripts no longer declare it. The warning is gone from ./gradlew build.

Verification: the sync simulation passes now and fails without the fix; the trigger really does register in both cases, with taskTriggers.toMap() reporting afterSync: plugwrightCompileTests whether idea is applied before or after evaluation; the merged bundle jar is unchanged. Also confirmed in a real IntelliJ sync on the example project.

Two commits on the branch: 86fd6fb for the sync trigger, 5bdcc6d for the Kotlin plugin.

@Drownek

Drownek commented Aug 23, 2026

Copy link
Copy Markdown
Owner

Merging this into v3-dev as agreed
I'll bump the version to 3.0.0-dev.0 on the branch right now. Ready for second PR
whenever you are

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants