Repository navigation
Port oracle/graal#14788: explicit engine preinitialization (GR-79669), with EE TruffleSupport compat - #172
Closed
Sam Gammon (sgammon) wants to merge 4 commits into
Closed
Port oracle/graal#14788: explicit engine preinitialization (GR-79669), with EE TruffleSupport compat#172Sam Gammon (sgammon) wants to merge 4 commits into
Sam Gammon (sgammon) wants to merge 4 commits into
Conversation
(cherry picked from commit fea1534)
(cherry picked from commit 941d912)
…ruffle jar Upstream GR-79669 folds com.oracle.svm.truffle.TruffleSupport into the new com.oracle.svm.truffle.api.SubstrateTruffleSupport, which looks up the enterprise extension as com.oracle.svm.enterprise.truffle.EnterpriseTruffleSupport. The fork's EE base, Oracle GraalVM EA jdk-25i5-25.0.4.1.1-ea.05, predates that: its truffle-enterprise-svm.jar has EnterpriseSupport extends TruffleSupport (overriding createTruffleCompiler with the enterprise compiler configuration and enterprise compiler isolates, and createBackgroundCompileQueue), registered as the TruffleSupport image singleton by EnterpriseTruffleFeature.afterRegistration. Without TruffleSupport the EE hybrid fails LinkCheck, and EE images would lose the enterprise Truffle compiler. Bring TruffleSupport back as a subclass of SubstrateTruffleSupport with its old API, and let SubstrateTruffleSupport delegate the compiler, the compiler configuration name and the background compile queue to a registered TruffleSupport singleton. Native Image CE registers none, so CE behaves as upstream. Remove this when the EE base contains GR-79669.
Upstream GR-79669 targets 25.6 (`@since 25.6`, new "Version 25.6" CHANGELOG sections). The fork stays on 25.5.5 to match its EE base, so the API ships in 25.5: `@since 25.5` in Engine and TruffleInstrument, and the SDK and Truffle CHANGELOG entries move to the top "Version 25.5" sections, as with `@Value.Optimized`.
Member
Author
Open
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ports oracle#14788 "[WIP] [GR-79669] Support configurable context preinitialization for explicit engines" (Christian Humer), stacked on #171.
Upstream head at port time:
941d912eae8(commitsfea15341c60Support explicit engine preinitialization,941d912eae8Suppress intentional resource warnings in preinitialization tests), based on upstream49add4294bb. The upstream PR is WIP and was already updated once during the port (fea15341c60→941d912eae8); compare against these SHAs.What it adds
Engine.Builder.preinitialize(int contextCount, Consumer<Engine>)returns a one-shotEngine.Preinitialized(stored in a static field of a class initialized at image build time);Engine.newBuilder(Engine.Preinitialized)builds an explicit engine from it at run time and hands out its pool of pre-initialized contexts. Native Image static initializers can now create and run Truffle contexts. AlsoTruffleInstrument.onPreinitialize(Env), context-levelengine.UsePreInitializedContextand the testing optionengine.RequirePreinitializedContext(PolyglotContextOptions), and on the SVM sideSubstrateTruffleSupport(the formerTruffleSupportfolded into the compilation support, created with the runtime) and hostedKnownMethodsbuilt in the runtime constructor.Commits
Support explicit engine preinitialization—fea15341c60with-x. Conflicts, kept both sides:PolyglotImpl.buildLocalEngine: upstream's explicitpreinitializedEnginebranch, with the fork'stakePreInitializedEngine([truffle] parse guest sources during context pre-initialization #5,engine.ExplicitEngineUsesPreInitializedContext) and the check from 82e658a that an engine refusing the pre-initialized context leaves it for another engine, in theelsebranch. That check now readsPolyglotContextOptions.UsePreInitializedContext(upstream moved the option out ofPolyglotEngineOptions; engine option values include the context options).PolyglotEngineOptions: dropUsePreInitializedContext(moved), keep the fork'sDormantPreInitializedLanguages([truffle] Keep non-permitted pre-initialized languages dormant #23) andExplicitEngineUsesPreInitializedContext([truffle] parse guest sources during context pre-initialization #5).PolyglotSourceCache: strong caching whileinExplicitPreinitialization(upstream) orinEnginePreInitialization(fork, [truffle] parse guest sources during context pre-initialization #5).SubstrateTruffleSupport(renamed fromTruffleSupport): keeps the fork's separate Truffle suites under Ristretto ([svm] Make Truffle and Ristretto (GraalJITCompileAtRuntime) work in one image, with gate coverage #27).SubstrateTruffleRuntime: upstream'shostedKnownMethodsnext to Port oracle/graal#14787: @Value.Optimized allocation-free interface proxies (GR-79503) #171'sexactGeneratedProxyEntries; the fork's lazy compiler initialization unchanged.ContextPreInitializationTest: the shared test language keeps the fork's MIME types ([truffle] parse guest sources during context pre-initialization #5) and gets upstream'sinternalResources.Suppress intentional resource warnings in preinitialization tests—941d912eae8with-x(javactrylint; needed for the gate's--warning-as-error --force-javac).Keep com.oracle.svm.truffle.TruffleSupport for Oracle's prebuilt EE Truffle jar. Oracle's ea.05truffle-enterprise-svm.jarhasEnterpriseSupport extends com.oracle.svm.truffle.TruffleSupport(overridescreateTruffleCompilerwith theenterpriseconfiguration and enterprise compiler isolates, andcreateBackgroundCompileQueue), registered as theTruffleSupportimage singleton inEnterpriseTruffleFeature.afterRegistration; upstream's replacement (com.oracle.svm.enterprise.truffle.EnterpriseTruffleSupport extends SubstrateTruffleSupport) is not in it. Without this commit LinkCheck fails (checked by removing the class from the jar set: 4 problems,EnterpriseSupport/EnterpriseTruffleFeature→TruffleSupport) and EE images would silently compile Truffle with the community configuration.TruffleSupportcomes back as a subclass ofSubstrateTruffleSupportwith its old API, andSubstrateTruffleSupportdelegates the compiler, the configuration name and the background compile queue to a registeredTruffleSupportsingleton. CE registers none, so CE runs upstream's code. Drop this commit (like the ObjectLayout hold) when the EE base contains GR-79669.Document explicit engine preinitialization as 25.5 API on the fork. Upstream targets 25.6; the fork stays on 25.5.5 for its EE base, so@since 25.6→@since 25.5(Engine,TruffleInstrument) and the new SDK and Truffle CHANGELOG entries move into the top "Version 25.5" sections, as Port oracle/graal#14787: @Value.Optimized allocation-free interface proxies (GR-79503) #171's@Value.Optimizedentry.Results (LabsJDK
labsjdk-ce-latest-jvmci-25.5-b24, linux-amd64, on this branch)mx build --warning-as-error --force-javacin substratevm and truffle: OK.mx checkstyle --primaryin truffle, sdk, substratevm, compiler: clean.mx sigtest --check allin truffle and sdk: OK.mx eclipseformaton the two fork-touched SVM files: no changes.mx -p truffle unittest ContextPreInitializationTest FileSystemsTest GuestFunctionToHostInterfaceProxyTest HostAccessFromModuleTest HostStackTraceTest: OK (1246 tests).mx -p truffle unittest com.oracle.truffle.api.test.polyglot com.oracle.truffle.api.modularized.test: OK (2376 tests).mx native-truffle-unittest com.oracle.truffle.api.test.polyglot.ContextPreInitializationNativeImageTest(own image, as the Truffle native gate runs it): 13/13 passed.ris-all-b24.sh): OK (50 tests) + OK (2 tests).build-ee.pyon Oracle EAjdk-25i5-25.0.4.1.1-ea.05with a CE home from this branch): 32 jars + 6 static libraries replaced,jdk.graal.compilerupgraded, LinkCheck clean (1424 enterprise classes, 15128 references), launcher rebuilt,smoke-ee.shpassed.Engine.newBuilder("sl").preinitialize(1, …)and whosemainbuildsEngine.newBuilder(STATE)with-Dpolyglot.engine.RequirePreinitializedContext=truereturns 42 on both the EE home (implementation "Oracle GraalVM", so the enterpriseTruffleSupportis in use) and the CE home.Known upstream issue (not fixed here)
In
mx native-truffle-unittestruns ofContextPreInitializationTestandFileSystemsTest(the Truffle native unit-test gate; the Elide Gate does not run it), tests that pre-initialize contexts at image run time now fail: the new explicit-engine tests callEngine.Builder.preinitializeat run time, which upstream rejects by design (IllegalStateException), and older tests (testIsSameFile*,testSetCurrentWorkingDirectory,testSourcesOutsideInternalResources) and all ofFileSystemsTest(itscreateConfigurationsnow callsFileSystems.prepareForNativeImageat run time) fail with "Pre-initialized path was not normalized during image generation". Same tests on #171 (before this port): 1152 run, 0 failed. The upstream test code has no AOT assumptions there, so this is in the WIP PR itself; build-time pre-initialization (the production path) works, see above. Left as upstream wrote it, to be picked up with the PR's next revision.