Skip to content

Port oracle/graal#14788: explicit engine preinitialization (GR-79669), with EE TruffleSupport compat - #172

Closed
Sam Gammon (sgammon) wants to merge 4 commits into
feat/optimized-value-proxiesfrom
feat/engine-preinit
Closed

Sam Gammon (sgammon) wants to merge 4 commits into
feat/optimized-value-proxiesfrom
feat/engine-preinit

Conversation

@sgammon

Copy link
Copy Markdown
Member

Ports oracle#14788 "[WIP] [GR-79669] Support configurable context preinitialization for explicit engines" (Christian Humer), stacked on #171.

Upstream head at port time: 941d912eae8 (commits fea15341c60 Support explicit engine preinitialization, 941d912eae8 Suppress intentional resource warnings in preinitialization tests), based on upstream 49add4294bb. The upstream PR is WIP and was already updated once during the port (fea15341c60 → 941d912eae8); compare against these SHAs.

What it adds

Engine.Builder.preinitialize(int contextCount, Consumer<Engine>) returns a one-shot Engine.Preinitialized (stored in a static field of a class initialized at image build time); Engine.newBuilder(Engine.Preinitialized) builds an explicit engine from it at run time and hands out its pool of pre-initialized contexts. Native Image static initializers can now create and run Truffle contexts. Also TruffleInstrument.onPreinitialize(Env), context-level engine.UsePreInitializedContext and the testing option engine.RequirePreinitializedContext (PolyglotContextOptions), and on the SVM side SubstrateTruffleSupport (the former TruffleSupport folded into the compilation support, created with the runtime) and hosted KnownMethods built in the runtime constructor.

Commits

  1. Support explicit engine preinitialization — fea15341c60 with -x. Conflicts, kept both sides:
  2. Suppress intentional resource warnings in preinitialization tests — 941d912eae8 with -x (javac try lint; needed for the gate's --warning-as-error --force-javac).
  3. Fork: Keep com.oracle.svm.truffle.TruffleSupport for Oracle's prebuilt EE Truffle jar. Oracle's ea.05 truffle-enterprise-svm.jar has EnterpriseSupport extends com.oracle.svm.truffle.TruffleSupport (overrides createTruffleCompiler with the enterprise configuration and enterprise compiler isolates, and createBackgroundCompileQueue), registered as the TruffleSupport image singleton in EnterpriseTruffleFeature.afterRegistration; upstream's replacement (com.oracle.svm.enterprise.truffle.EnterpriseTruffleSupport extends SubstrateTruffleSupport) is not in it. Without this commit LinkCheck fails (checked by removing the class from the jar set: 4 problems, EnterpriseSupport/EnterpriseTruffleFeature → TruffleSupport) and EE images would silently compile Truffle with the community configuration. TruffleSupport comes back as a subclass of SubstrateTruffleSupport with its old API, and SubstrateTruffleSupport delegates the compiler, the configuration name and the background compile queue to a registered TruffleSupport singleton. CE registers none, so CE runs upstream's code. Drop this commit (like the ObjectLayout hold) when the EE base contains GR-79669.
  4. Fork: Document explicit engine preinitialization as 25.5 API on the fork. Upstream targets 25.6; the fork stays on 25.5.5 for its EE base, so @since 25.6 → @since 25.5 (Engine, TruffleInstrument) and the new SDK and Truffle CHANGELOG entries move into the top "Version 25.5" sections, as Port oracle/graal#14787: @Value.Optimized allocation-free interface proxies (GR-79503) #171's @Value.Optimized entry.

Results (LabsJDK labsjdk-ce-latest-jvmci-25.5-b24, linux-amd64, on this branch)

  • mx build --warning-as-error --force-javac in substratevm and truffle: OK. mx checkstyle --primary in truffle, sdk, substratevm, compiler: clean. mx sigtest --check all in truffle and sdk: OK. mx eclipseformat on the two fork-touched SVM files: no changes.
  • mx -p truffle unittest ContextPreInitializationTest FileSystemsTest GuestFunctionToHostInterfaceProxyTest HostAccessFromModuleTest HostStackTraceTest: OK (1246 tests).
  • mx -p truffle unittest com.oracle.truffle.api.test.polyglot com.oracle.truffle.api.modularized.test: OK (2376 tests).
  • mx native-truffle-unittest com.oracle.truffle.api.test.polyglot.ContextPreInitializationNativeImageTest (own image, as the Truffle native gate runs it): 13/13 passed.
  • Ristretto gate set (ris-all-b24.sh): OK (50 tests) + OK (2 tests).
  • EE hybrid (build-ee.py on Oracle EA jdk-25i5-25.0.4.1.1-ea.05 with a CE home from this branch): 32 jars + 6 static libraries replaced, jdk.graal.compiler upgraded, LinkCheck clean (1424 enterprise classes, 15128 references), launcher rebuilt, smoke-ee.sh passed.
  • Explicit preinitialization in a native image: an SL program whose build-time-initialized holder runs Engine.newBuilder("sl").preinitialize(1, …) and whose main builds Engine.newBuilder(STATE) with -Dpolyglot.engine.RequirePreinitializedContext=true returns 42 on both the EE home (implementation "Oracle GraalVM", so the enterprise TruffleSupport is in use) and the CE home.

Known upstream issue (not fixed here)

In mx native-truffle-unittest runs of ContextPreInitializationTest and FileSystemsTest (the Truffle native unit-test gate; the Elide Gate does not run it), tests that pre-initialize contexts at image run time now fail: the new explicit-engine tests call Engine.Builder.preinitialize at run time, which upstream rejects by design (IllegalStateException), and older tests (testIsSameFile*, testSetCurrentWorkingDirectory, testSourcesOutsideInternalResources) and all of FileSystemsTest (its createConfigurations now calls FileSystems.prepareForNativeImage at run time) fail with "Pre-initialized path was not normalized during image generation". Same tests on #171 (before this port): 1152 run, 0 failed. The upstream test code has no AOT assumptions there, so this is in the WIP PR itself; build-time pre-initialization (the production path) works, see above. Left as upstream wrote it, to be picked up with the PR's next revision.

…ruffle jar

Upstream GR-79669 folds com.oracle.svm.truffle.TruffleSupport into the new
com.oracle.svm.truffle.api.SubstrateTruffleSupport, which looks up the
enterprise extension as com.oracle.svm.enterprise.truffle.EnterpriseTruffleSupport.
The fork's EE base, Oracle GraalVM EA jdk-25i5-25.0.4.1.1-ea.05, predates that:
its truffle-enterprise-svm.jar has EnterpriseSupport extends TruffleSupport
(overriding createTruffleCompiler with the enterprise compiler configuration and
enterprise compiler isolates, and createBackgroundCompileQueue), registered as
the TruffleSupport image singleton by EnterpriseTruffleFeature.afterRegistration.
Without TruffleSupport the EE hybrid fails LinkCheck, and EE images would lose
the enterprise Truffle compiler.

Bring TruffleSupport back as a subclass of SubstrateTruffleSupport with its old
API, and let SubstrateTruffleSupport delegate the compiler, the compiler
configuration name and the background compile queue to a registered
TruffleSupport singleton. Native Image CE registers none, so CE behaves as
upstream. Remove this when the EE base contains GR-79669.
Upstream GR-79669 targets 25.6 (`@since 25.6`, new "Version 25.6" CHANGELOG
sections). The fork stays on 25.5.5 to match its EE base, so the API ships in
25.5: `@since 25.5` in Engine and TruffleInstrument, and the SDK and Truffle
CHANGELOG entries move to the top "Version 25.5" sections, as with
`@Value.Optimized`.
@sgammon

Copy link
Copy Markdown
Member Author

Landed on elide-25.5.x by fast-forward to b2eb4ee (with #170), for 25.5.5-elide.21. The commits are unchanged; GitHub can't mark a stacked PR merged once its base already contains it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants