Separate debuggee/tracee build from product build#5767
Conversation
- Install runtimes from dirs/test.proj and remove installation from the main build script. - Move debuggee and tracee projects out of the main build traversal into a new Debuggees.proj that is built using the test SDK in artifacts instead of the global.json one that's pulled by arcade.
d223099 to
99a4ba7
Compare
…into juhoyosa/separate-debuggee-build
…into juhoyosa/separate-debuggee-build
…/hoyosjs/diagnostics into juhoyosa/separate-debuggee-build
…#124851) dotnet/runtime#124851 ("Add ActivityTraceFlags.RandomTraceId flag", W3C Trace Context Level 2) changes how ActivityTraceFlags are emitted into EventPipe events: - Microsoft.Extensions.Logging.EventSource now emits the flags field as the raw integer ("0".."3") rather than just "0"/"1", and emits string.Empty (instead of null) for trace/span IDs when no current Activity exists. - ActivitySource-created activities now have RandomTraceId set automatically, so DiagnosticSourceEventSource Stop events carry flag values this consumer did not recognize. - net 8 servicing runtimes now honor ParentRatioSampler in the FilterAndPayloadSpec, where they previously dropped it. Make the consumer tolerant of these wire-format shifts: * LogRecordFactory: IsNullOrEmpty for trace/span IDs; integer parse with bit-mask for trace flags via new ParseActivityTraceFlags helper. * TraceEventExtensions: tolerant ActivityTraceFlags parse that masks unknown bits (handles either integer or comma-separated name form). * DistributedTracesPipelineUnitTests: drop the obsolete net 8 carveout in TestTracesPipelineWithSamplingRatio; all supported TFMs now produce a single un-recorded activity here. Also bump MicrosoftExtensionsLoggingEventSourceVersion 8.0.3 -> 10.0.5 (the 8.0.3 pin does not exist on the public feeds and was being substituted to 9.0.0 with NU1603 treated as error). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…into juhoyosa/separate-debuggee-build
.NET 11 is moving to cDAC as the supported diagnostic path; the legacy native DAC is being deprecated for that runtime line. CI on this PR showed 18 SOS test failures per Windows leg, all on the net 11.0 preview.5 config, with the pattern '!PrintException → Invalid exception object' against heap minidumps (live debugging worked; only minidump reading failed). Net 8 / 9 / 10 SOS tests passed. Auto-enable DOTNET_ENABLE_CDAC=1 + CDAC_NO_FALLBACK=1 in SOSRunner when the test config targets a runtime with RuntimeFrameworkVersionMajor >= 11. Existing -usecdac / -nofallback build flags continue to control behavior for older runtimes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Per @noahfalk in dotnet#5767 (review comment r3172316592): nothing downstream of TraceEventExtensions consumes anything beyond the Recorded bit, so detect that bit by substring match on the emitted name form and treat all other inputs as None. This avoids the version-coupling tension of parsing the full ActivityTraceFlags enum across a producer runtime that may carry flag bits this consumer doesn't yet know about (e.g. RandomTraceId from runtime#124851). DiagnosticSourceEventSource serializes the field via TypeAccessor.ToString, which emits the [Flags] name form ('Recorded', 'Recorded, RandomTraceId', etc.), so 'Contains("Recorded")' is sufficient. Note: the sibling LogRecordFactory.EmitLog path continues to use integer parsing because Microsoft.Extensions.Logging.EventSource emits the flags field as the numeric value (per runtime#124851's EventSourceLogger.cs change), not the name form. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…on and fix conditions
There was a problem hiding this comment.
Pull request overview
This PR restructures the build/test flow to decouple debuggee/tracee builds (and test runtime installation) from the main product build, while moving test infrastructure forward to include .NET 11 and updating tests/pipelines accordingly.
Changes:
- Move test runtime installation into
src/tests/dirs.proj(and remove the unit-test install hook fromsrc/Directory.Build.targets/ build scripts). - Build debuggee/tracee projects via a new
src/tests/Debuggees.projusing the installed test SDK (with cross-arch support via a host-arch SDK install). - Update tests/scripts and pipeline behavior for new runtime behavior (.NET 11 async/runtime-async, updated DiagnosticSource behavior, improved failure artifact collection).
Reviewed changes
Copilot reviewed 34 out of 34 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| start-vscode.sh | Updates VS Code target framework to net11.0. |
| src/tests/SOS.UnitTests/SOSRunner.cs | Simplifies runtime-version conditionals and define generation for supported runtimes. |
| src/tests/SOS.UnitTests/Scripts/WebApp.script | Adapts DumpAsync expectations for .NET 11+ warning output / behavior changes. |
| src/tests/SOS.UnitTests/Scripts/StackTests.script | Removes legacy runtime gating around EEVersion/DumpStack checks. |
| src/tests/SOS.UnitTests/Scripts/StackAndOtherTests.script | Adjusts stack tests; adds macOS-specific skips for known formatting differences. |
| src/tests/SOS.UnitTests/Scripts/Overflow.script | Removes pre-.NET 5 conditional expectations for exception output. |
| src/tests/SOS.UnitTests/Scripts/GCTests.script | Drops .NET 2-specific DumpObj expectations (aligns with supported TFMs). |
| src/tests/SOS.UnitTests/Scripts/DumpGen.script | Removes runtime gating around POH dumpgen command. |
| src/tests/SOS.UnitTests/Scripts/ConcurrentDictionaries.script | Skips a dumpobj check on Alpine due to a known crash. |
| src/tests/SOS.UnitTests/DumpGenerationFixture.cs | Broadens runtime directory enumeration for DAC registration (no longer “10*” only). |
| src/tests/Microsoft.Diagnostics.Monitoring.EventPipe/DistributedTracesPipelineUnitTests.cs | Updates sampling-ratio test expectations to reflect newer runtime behavior. |
| src/tests/dirs.proj | Removes debuggee traversal; adds runtime install reference and a post-build debuggee build using test SDK. |
| src/tests/Directory.Build.targets | Adds conditional ProjectReference to InstallRuntimes for projects that copy debuggee sources. |
| src/tests/Debuggees.proj | New traversal project for building debuggees/tracees with the test SDK. |
| src/tests/CommonTestRunner/Debuggees/EventPipeTracee/Program.cs | Adds logging/eventpipe workaround comments and changes pipe-connect ordering. |
| src/Microsoft.Diagnostics.TestHelpers/TestStep.cs | Exposes log file path to derived steps. |
| src/Microsoft.Diagnostics.TestHelpers/DotNetBuildDebuggeeTestStep.cs | Adds per-operation MSBuild binlog generation for restore/build. |
| src/Microsoft.Diagnostics.Monitoring.EventPipe/Logs/LogRecordFactory.cs | Makes ActivityTraceFlags parsing tolerant of integer-encoded flags. |
| src/Microsoft.Diagnostics.Monitoring.EventPipe/DistributedTraces/TraceEventExtensions.cs | Stops parsing full enum text; detects “Recorded” bit by name to tolerate newer flag bits. |
| src/Microsoft.Diagnostics.ExtensionCommands/DumpAsyncCommand.cs | Emits warnings for .NET 11+ runtime-async limitations. |
| src/Directory.Build.targets | Removes unit-test-time runtime installation target (now handled under test traversal). |
| NuGet.config | Adds dotnet11 feeds. |
| eng/Versions.props | Moves “Latest” runtime testing to net11; adds net10 servicing runtime version metadata. |
| eng/Version.Details.xml | Updates dotnet toolset dependencies to .NET 11 preview versions. |
| eng/Version.Details.props | Updates corresponding version properties for dotnet toolset dependencies. |
| eng/pipelines/build.yml | Removes CodeQL-specific parameterization; improves test result publishing and failure artifact handling. |
| eng/installruntimes.sh | Calls msbuild directly to run InstallTestRuntimes. |
| eng/InstallRuntimes.proj | Adds host-arch SDK install for cross-arch debuggee builds; hooks InstallTestRuntimes after Build. |
| eng/installruntimes.cmd | Calls msbuild to run InstallTestRuntimes on Windows. |
| eng/DisableSignatureCheck.ps1 | Stores/restores state as JSON; registers DAC/runtime paths for installed test runtimes. |
| eng/build.sh | Removes -installruntimes flag path; passes PrivateBuildTesting/LiveRuntimeDir; disables system core dumps during tests. |
| eng/build.ps1 | Removes -installruntimes switch path; routes PrivateBuildTesting/LiveRuntimeDir into managed build. |
| eng/AuxMsbuildFiles/SdkPackOverrides.targets | Adds net11 override support and makes override application conditional on populated RuntimeTestVersions. |
| Directory.Build.props | Adds net11.0 to supported subprocess TFMs. |
Comments suppressed due to low confidence (1)
eng/pipelines/build.yml:75
isCodeQLRunwas removed from this template, butdiagnostics-codeql.ymlstill passesisCodeQLRun: truewhen referencing/eng/pipelines/build.yml. This will break the CodeQL pipeline due to an unexpected parameter. Either reintroduce the parameter (even if it becomes a no-op) or update the callers in the same PR.
# Optional: container resource name
- name: container
type: string
default: ''
# Optional: build only job if true
- name: buildOnly
type: boolean
default: false
# Optional: test only job if true
- name: testOnly
type: boolean
default: false
# Optional: architecture cross build if true
- name: crossBuild
type: boolean
default: false
- name: dependsOn
type: string
default: ''
- name: disableComponentGovernance
type: boolean
default: false
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // defined in newer DiagnosticSource versions than this consumer | ||
| // was compiled against. | ||
| if (value is string traceFlagsValue | ||
| && traceFlagsValue.Contains("Recorded", StringComparison.OrdinalIgnoreCase)) |
Co-authored-by: Juan Hoyos <19413848+hoyosjs@users.noreply.github.com>
| /p:TargetArch=$architecture ` | ||
| /p:TestArchitectures=$architecture ` | ||
| /p:LiveRuntimeDir="$liveRuntimeDir" | ||
| Invoke-Expression "& `"$engroot\common\build.ps1`" -configuration $configuration -verbosity $verbosity $bl /p:TargetOS=$os /p:TargetArch=$architecture /p:TestArchitectures=$architecture /p:PrivateBuildTesting=$privatebuildtesting /p:LiveRuntimeDir=`"$liveRuntimeDir`" $remainingargs" |
There was a problem hiding this comment.
nit: I think the previous invocation format was easier to read and understand
|
This shows an E_NOIMPL from the cDAC in |
We can look into it in a follow up. This looks good to go! |
|
I am OK if @max-charlamb and @rcj1 are ok with. |
|
Good with me. Can investigate that cDAC failure after it's merged |
Uh oh!
There was an error while loading. Please reload this page.