[SYSTEMDS-3948] Row-wise Sparsity Estimator

[ywcb00]

Hi,
this PR adds the row-wise sparsity estimator from:
Lin, Chunxu, Wensheng Luo, Yixiang Fang, Chenhao Ma, Xilin Liu and Yuchi Ma;
On Efficient Large Sparse Matrix Chain Multiplication;
Proceedings of the ACM on Management of Data 2 (2024): 1 - 27.

Note that the row sparsity propagation, as described in the publication, applies to MM chains only. Other operations use fallback methods for sparsity estimation w/ row sparsity vectors, which create a cut in the sparsity estimation DAG.

[Baunsgaard]

Some comments. hope it helps.

[Baunsgaard]

You can optionally have all of these classes extend another class where you add these tests. Instead of copying the code on all of them.

[ywcb00]

Thank you for the comment. I do not fully understand how you would consolidate them. Could you please clarify?

[Baunsgaard]

What I ment was something like :

OpElemWTest extends OpElemBaseTest {

}

and then you define a new OpElemBaseTest class that contains the variables all the different OpElem test cases share, and define in that base test the fundamental tests that all of the OpElem should run.

I hope this would work nicely, since it makes it easy to make the same test flavour run in all of the variations.

[ywcb00]

Thank you again for the explanation, I gave it another thought. While I also like such a hierarchical structure for tests in general, I cannot see the clear advantage here--given that I understood your suggestion correctly.
There are five common variables among the test cases m, n, sparsity, mult, plus, which would then be defined inside OPElemBaseTest class instead of OPElemWTest class. I would not consolidate tests for multiplication and addition (neither for different estimators) in one single function, as this would break the typical structure in which tests are defined (right?). Hence, I think such a base class would just add another class file, or have I misunderstood something?

[codecov]

Codecov Report

❌ Patch coverage is 76.28205% with 37 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.34%. Comparing base (c2d0e5a) to head (4855b08).
⚠️ Report is 6 commits behind head on main.

Files with missing lines	Patch %	Lines
.../org/apache/sysds/hops/estim/EstimatorRowWise.java	76.28%	24 Missing and 13 partials ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #2466      +/-   ##
============================================
+ Coverage     71.29%   71.34%   +0.04%     
- Complexity    48574    48686     +112     
============================================
  Files          1567     1569       +2     
  Lines        188401   188776     +375     
  Branches      36980    37040      +60     
============================================
+ Hits         134325   134673     +348     
+ Misses        43630    43628       -2     
- Partials      10446    10475      +29     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.