Skip to content

[tests] Track evaluation time across performance runs - #12390

Merged
jonathanpeppers merged 1 commit into
mainfrom
jonathanpeppers-fix-performance-evaluation-tracking
Aug 14, 2026
Merged

[tests] Track evaluation time across performance runs#12390
jonathanpeppers merged 1 commit into
mainfrom
jonathanpeppers-fix-performance-evaluation-tracking

Conversation

@jonathanpeppers

Copy link
Copy Markdown
Member

Summary

  • track the maximum MSBuild project evaluation duration across every Profile iteration
  • apply the same aggregation to ProfileTask
  • use the aggregate for slow-machine classification instead of only the final binlog

Fixes the valid finding in #12389 (comment).

Testing

  • dotnet build tests\MSBuildDeviceIntegration\MSBuildDeviceIntegration.csproj --no-restore -v:minimal -p:JavaCPath='C:\Program Files\Microsoft\jdk-17.0.16.8-hotspot\bin\javac.exe' -p:JarPath='C:\Program Files\Microsoft\jdk-17.0.16.8-hotspot\bin\jar.exe' (succeeded; 16 existing warnings, 0 errors)

Aggregate the maximum MSBuild project evaluation duration from every profiling iteration so slow-machine detection does not depend on the final binlog.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings August 14, 2026 16:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the MSBuild device integration performance tests to track the maximum MSBuild project evaluation time across all iterations in a profiling run, and uses that aggregate when deciding whether to mark a failure as “slow CI machine”.

Changes:

  • Track and aggregate max ProjectEvaluation duration across Profile() iterations.
  • Apply the same max-evaluation aggregation to ProfileTask().
  • Use aggregated max evaluation time for slow-machine classification instead of relying on only the final iteration’s binlog.

Comment thread tests/MSBuildDeviceIntegration/Tests/PerformanceTest.cs
@jonathanpeppers jonathanpeppers added the ready-to-review This PR is ready to review/merge, I think any CI failures are just flaky (ignorable). label Aug 14, 2026
@jonathanpeppers

Copy link
Copy Markdown
Member Author

@dalexsoto review

@jonathanpeppers
jonathanpeppers enabled auto-merge (squash) August 14, 2026 18:59

@dalexsoto dalexsoto left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The maximum evaluation duration is now aggregated across all Profile and ProfileTask iterations, so slow-machine classification no longer depends on only the final binlog.

@jonathanpeppers
jonathanpeppers merged commit c5f002b into main Aug 14, 2026
45 checks passed
@jonathanpeppers
jonathanpeppers deleted the jonathanpeppers-fix-performance-evaluation-tracking branch August 14, 2026 19:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-to-review This PR is ready to review/merge, I think any CI failures are just flaky (ignorable).

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants