Stabilize SDCA logistic regression test - #7677
Open
matouskozak wants to merge 3 commits into
Open
Conversation
Make strict quality assertions deterministic and retain explicit coverage of multithreaded SDCA training. Fixes #7343. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: cf5862a0-24ff-48b6-bd23-a5fa8d0053f7
Contributor
There was a problem hiding this comment.
Pull request overview
This PR addresses flakiness in SdcaLogisticRegression test assertions by making the “quality gate” test deterministic while keeping a separate test that still exercises SDCA’s multithreaded (nondeterministic) training path.
Changes:
- Switches the strict
SdcaLogisticRegressionquality test to useNumberOfThreads = 1to stabilize metrics likeLogLoss. - Adds a new multithreaded smoke test (
NumberOfThreads = 4) that validates predictions are finite/calibrated and that the model learns the dataset’s class signal. - Updates the trainer construction to use the
SdcaLogisticRegressionBinaryTrainer.Optionsoverload (to setNumberOfThreads).
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #7677 +/- ##
=======================================
Coverage 69.88% 69.88%
=======================================
Files 1487 1487
Lines 276214 276240 +26
Branches 28287 28287
=======================================
+ Hits 193019 193056 +37
+ Misses 75708 75698 -10
+ Partials 7487 7486 -1
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
Require the four-thread trainer to preserve the original AUC quality and outperform a featureless prior model on LogLoss. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: cf5862a0-24ff-48b6-bd23-a5fa8d0053f7
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: cf5862a0-24ff-48b6-bd23-a5fa8d0053f7
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #7343.
The strict
SdcaLogisticRegressionquality assertions now use a single training thread, making their results deterministic. A separate four-thread test retains correctness coverage of SDCA's nondeterministic parallel path by preserving the original AUC requirement and verifying that its LogLoss outperforms a featureless Prior baseline.My theory is that the macOS arm64 runners have more threads than the Windows/Linux runners which leads to more non-determinism, triggering the flaky failure.