Support async sources - #3248
Open
timcassell wants to merge 2 commits into
Open
Conversation
Refactored parameter discovery and codegen. Added/removed analyzer rules. Added tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adapts the branch to the toolchain/runtime refactor (#3232): - `InProcessEmitToolchain`'s constructor is now private behind the `From` factory; `AsyncEnumerableParamsSourceTests` updated accordingly. - The branch's `9.0` LangVersion floor is re-applied in master's renamed `CsProjNativeAotGenerator`; `MonoAOTLLVMCsProj.txt` is gone with its toolchain. - `BenchmarkCase` keeps both master's `GetToolchain()` and the branch's `DisposeAsync()`. - `MockInProcessDiagnoser` is a new netstandard2.0 satellite that bound the branch's internal `System.Linq.AsyncEnumerable` polyfill, which is compiled out of BenchmarkDotNet's .NET 10 asset. It now iterates locally, matching `EtwProfiler` and `FinalizerBlockerDiagnoser`. The analyzer resource files are restored to the repository's CRLF+BOM encoding. The branch had rewritten both to LF, which made every line conflict. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adding support for async sources revealed several issues with the system. Discovery and codegen were refactored here to be able to support it, and a few smaller issues were also fixed here. Other issues will be addressed separately.
Resolves #3120
Core changes:
[ParamsSource]/[ArgumentsSource]may returnIAsyncEnumerable<T>, which BenchmarkDotNet awaits when reading the values.[EnumeratorCancellation] CancellationToken.BenchmarkConverter.TypeToBenchmarksAsync/MethodsToBenchmarksAsync,TypeFilter.FilterAsync.outparameter to be assigned in the object initializer, sorequiredandinitmembers are respected andawaitworks.IParamreplaced by a closedParameterValuehierarchy. A toolchain readsParameterInstance.ParameterValueand emits whatever its language needs, instead of a param returning a C# string.ParameterDefinitionis metadata only.ParameterValuespairs a definition with the values it ranges over.Behavior changes:
[Params]/[ParamsSource]/[ParamsAllValues]member is now a parameter.FlattenHierarchyand discovery never asked, so such a member was silently no parameter at all and the benchmark ran once on the default value.[BenchmarkCancellation]member is now assigned. Same cause; cancellation silently did nothing.[ArgumentsSource]row share a single read in the child process (no change to in-process).ExecutionValidatorandReturnValueValidatorskip a benchmark whose by-ref-like shapes reflection cannot handle, instead of refusing it.IAsyncDisposableparameters are now disposed asynchronously.requiredmembers not annotated with[Params*]or[BenchmarkCancellation]now cause the benchmark type to be rejected[SetsRequiredMembers]to the runnable ctor. Since the codegen changed back to object initializer syntax, that hack was no longer necessary.Breaking changes:
IParam,ArrayParam<T>,EnumParam— replaced byParameterValue.ParameterInstance.ToSourceCode()— readParameterValueinstead. Theobject-taking constructor is nowinternal(for tests only).ParameterDefinition.Values, and its constructor'svaluesparameter with it — a definition is metadata, and the values live on the newParameterValues.ParameterDefinitionsParameterInstancesno longer implementsIEquatable<ParameterInstances>and no longer overridesEquals/GetHashCode. Value comparison lives inParameterEqualityComparer.ParameterExtractor.GetParameter<T>replaced byGetParameterAsync<T>(codegen-only,[EditorBrowsable(Never)]).CorrectionsSuggesteris nowinternal— used only by console-argument plumbing for--filtertypo suggestions.DisassemblerConstants— its single constant value was moved to the internalRunnableConstants.Other Changes:
[Params]member hiding an inherited one threwAmbiguousMatchExceptiononmasterunder both in-process toolchains; the lookup now matches the declared type and picks the most derived member. Relatedly, when both declarations carry the attribute the pair no longer becomes two parameters of one name - reflection reports a hidden field twice where it collapses a hidden property - which multiplied the cases against themselves and named the member twice in the runnable's object initializer.ExecutionValidatorandReturnValueValidatorreporting a spurious error for any benchmark taking[Arguments]—no property or field <name> foundonmaster, because an argument was looked up as though it were a member.DynamicAwaitHelpercrashed the process while awaiting a user's awaitable, both from asking it for an interface map it cannot supply. An awaiter implementing onlyINotifyCompletion- legal,ICriticalNotifyCompletionis optional in the pattern - threw an unhandledArgumentException: Interface not found; an awaiter whose declared type is an interface - the pattern binds on the declared type - threwArgumentException: 'this' type cannot be an interface itself. Both landed onAwaitUnsafeOnCompleted, where they were rethrown onto the thread pool with nothing to catch them.Run->__Run,FieldsContainer->__FieldsContainerSystem.Linq.AsyncEnumerabledependency, re-implemented operations inline to use theBenchmarkSynchronizationContextinstead of the ambientSynchronizationContext.init-only[Params]members are supported via assignment through an object initializer (in-process toolchains already supported them).CS8400).Note for anyone implementing
IValidator/IDiagnoseroutside this repo on a pre-net10 target:IAsyncEnumerable<T>itself flows from theMicrosoft.Bcl.AsyncInterfacesreference, so an async iterator needs nothing extra - only the LINQ helpers (AsyncEnumerable.Empty<T>(),ToAsyncEnumerable()) require adding theSystem.Linq.AsyncEnumerablepackage, which is in the framework from .NET 10.