[release/10.0.1xx] Improve PerformanceTest reliability

[jonathanpeppers]

Backports #10756 and #11249 to release/10.0.1xx.

This PR contains only their two squash commits from main, cherry-picked in order.

[Copilot]

Pull request overview

This backport improves reliability of MSBuildDeviceIntegration performance tests on the release/10.0.1xx branch by removing known-flaky perf tests and by skipping failures on slow CI machines using MSBuild project evaluation time derived from the binlog.

Changes:

• Removed two flaky MSBuild performance tests (Build_From_Clean_DontIncludeRestore, Build_JLO_Change) and their reference timing entries.
• When perf results exceed expected thresholds, evaluates binlog ProjectEvaluation durations and marks the test inconclusive when evaluation indicates a slow machine.
• Refactored binlog reading so BinaryLog.ReadBuild() happens once per iteration and helper methods consume the parsed Build.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
tests/MSBuildDeviceIntegration/Tests/PerformanceTest.cs	Removes flaky tests; adds slow-machine detection via binlog evaluation time; refactors binlog reading to reduce repeated parsing.
tests/msbuild-times-reference/MSBuildDeviceIntegration.csv	Removes reference timing entries for the deleted flaky tests.

Suppressed comments (2)

tests/MSBuildDeviceIntegration/Tests/PerformanceTest.cs:90

• Same issue as Profile: ProfileTask averages across iterations but only checks evaluation time from the last binlog. This can still leave the test flaky on slow/variable CI machines.

Accumulate the max evaluation time across iterations and use that for the slow-machine check.

			double total = 0;
			StructuredBuild build = null;
			for (int i=0; i < iterations; i++) {
				action (builder);
				build = ReadBinLog (builder);
				var duration = GetTaskDuration (build, builder, task);

tests/MSBuildDeviceIntegration/Tests/PerformanceTest.cs:109

• AssertSlowMachine currently enumerates FindChildrenRecursive<ProjectEvaluation>() twice (Any() then Max()), and its signature forces callers to pass a Build even though the method only needs the evaluation-time value. Since this code runs on the failure path, simplify it to compute the max evaluation time once and accept that value as an argument (paired with tracking it across iterations).

		void AssertSlowMachine (StructuredBuild build, int expected, double actual)
		{
			var evaluations = build.FindChildrenRecursive<ProjectEvaluation> ();
			var maxEval = evaluations.Any () ? evaluations.Max (e => e.Duration.TotalMilliseconds) : 0;
			TestContext.Out.WriteLine ($"max evaluation time: {maxEval}ms (threshold: {MaxEvaluationTimeInMs}ms)");
			if (maxEval > MaxEvaluationTimeInMs) {
				Assert.Inconclusive ($"Exceeded expected time of {expected}ms, actual {actual}ms, but evaluation time was {maxEval:F0}ms (threshold: {MaxEvaluationTimeInMs}ms), indicating a slow CI machine.");
			}

---

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

[jonathanpeppers]

/azp run

[azure-pipelines]

Azure Pipelines:
1 pipeline(s) were filtered out due to trigger conditions.

[jonathanpeppers]

@dalexsoto review

[dalexsoto]

The two backported reliability changes match their stated scope. The final-binlog limitation is an accepted follow-up from main, and no independent blocker remains.