Tune Maven's own retry/concurrency to reduce Maven Central rate-limiting by ramakrishnap-nv · Pull Request #1835 · NVIDIA/cuopt

ramakrishnap-nv · 2026-09-01T13:29:11Z

Summary

java-static-test (from Explore self-contained Java classifier JARs with a statically linked libcuopt #1818) and, less often, java-build have hit 429 Too Many Requests from Maven Central while resolving plugins like maven-source-plugin on a cold repository. cuopt_mvn's retry loop (fixed in Fix cuopt_mvn retry loop being silently skipped under set -e #1823 to actually run under set -e) retries the whole mvn invocation with backoff, but that's compensating for Maven's own resolver never being tuned for CI -- 4 attempts don't reliably outlast a sustained rate-limit window, and by the time the outer wrapper retries, the burst of parallel requests that likely triggered the 429 in the first place repeats.
cuDF and cuVS already carry a fix for this exact problem in their own Java/Maven builds: a project-level .mvn/maven.config, auto-applied to every mvn invocation with no wrapper script needed, that caps concurrent downloads and adds a real backoff inside Maven's own transport-layer retry handler.

Fix

Add java/cuopt/.mvn/maven.config, matching cuDF's (java/.mvn/maven.config) and cuVS's (java/cuvs-java/.mvn/maven.config) content exactly:

-e
-B
-Daether.connector.basic.downstreamThreads=1
-Daether.transport.http.retryHandler.count=5
-Daether.transport.http.retryHandler.interval=10000
-Dmaven.wagon.http.retryHandler.count=5

aether.connector.basic.downstreamThreads=1 caps Maven's own concurrent download threads, reducing the burst of parallel requests against Maven Central that likely triggers the rate-limiting in the first place.
aether.transport.http.retryHandler.interval=10000 adds a real 10s backoff inside Maven's own resolver, at the transport layer.

cuopt_mvn's existing -D flags (java/cuopt/scripts/maven.sh) target the connector-layer retry handler (aether.connector.http.retryHandler.*), which recent Maven resolver versions may no longer consult now that retry logic lives at the transport layer -- this adds the layer that actually gets read. Verified via:

mvn org.apache.maven.plugins:maven-help-plugin:3.4.0:evaluate \
  -Dexpression=aether.transport.http.retryHandler.interval -q -DforceStdout
# -> 10000

cuopt_mvn's outer shell-level retry loop is left in place as a second layer -- it's still useful for failures Maven's own retry can't cover (network drops mid-request, etc).

Test plan

Confirmed the property resolves correctly via maven-help-plugin:evaluate (see above).
Ran the full packaged-jar-tests Maven Java suite locally with this config present; passes cleanly.
java-build and java-static-test CI run clean without hitting Maven Central 429s.

Split out of #1818, where this was found while investigating an unrelated flaky Surefire crash -- this fix is independently useful for the existing java-build job today, not specific to that PR's self-contained classifier JAR work.

java-static-test has repeatedly hit 429 Too Many Requests resolving plugins like maven-source-plugin from a cold repository -- the cuopt_mvn wrapper's retry loop (fixed in #1823) retries the whole mvn invocation with backoff, but that's compensating for Maven's own resolver never being tuned, and 4 attempts don't reliably outlast a sustained rate limit. cuDF and cuVS already carry this exact fix for their own Java/Maven Central builds: a project-level .mvn/maven.config (auto-applied to every mvn invocation, no wrapper needed) that caps concurrent downloads to reduce burst request rate and adds a real backoff inside Maven's own transport-layer retry handler, rather than only retrying around the outside of a failed process: -Daether.connector.basic.downstreamThreads=1 -Daether.transport.http.retryHandler.count=5 -Daether.transport.http.retryHandler.interval=10000 -Dmaven.wagon.http.retryHandler.count=5 cuopt_mvn's own -D flags target the connector-layer retry handler, which recent Maven resolver versions may no longer consult now that retry logic lives at the transport layer -- this adds the layer that actually gets read, verified via `mvn help:evaluate -Dexpression=aether.transport.http.retryHandler.interval` resolving to 10000. Verified the packaged-jar-tests suite still passes with this config present.

coderabbitai · 2026-09-01T13:35:50Z

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

@coderabbitai resume to resume automatic reviews.
@coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

▶️ Resume reviews
🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 5b2fc11b-ffd8-4f04-8456-e2ce1b3b6165

📥 Commits

Reviewing files that changed from the base of the PR and between 7f776e8 and 82e4e86.

📒 Files selected for processing (1)

java/cuopt/pom.xml

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.

📝 Walkthrough

Walkthrough

The Java build runs Maven from java/cuopt, applies connector-level HTTP retry settings, prefers a Google Cloud Maven Central mirror, and attaches source and Javadoc JARs only through the attach-source-javadoc profile.

Changes

Maven configuration

Layer / File(s)	Summary
Maven execution and retry settings `build.sh`, `java/cuopt/.mvn/maven.config`	The build enters `java/cuopt` before invoking `pom.xml`. Maven uses connector-level HTTP retries, including retries for HTTP 429 and 503 responses.
Repository and attachment profile `java/cuopt/pom.xml`	The POM prefers the Google Cloud Maven Central mirror and falls back to Maven Central with snapshots disabled. Source and Javadoc JAR attachment runs only when `attach-source-javadoc` is activated, with Javadoc doclint disabled.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to 82e4e

This PR adjusts Maven retry/concurrency settings and the Java build invocation; no actionable merge-blocking risk remains after normal checks and review.

Suggested reviewers: gforsyth, iroy30

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly identifies the main change: tuning Maven retry and concurrency settings to reduce Maven Central rate-limiting. It is concise and directly related to the changeset.
Description check	✅ Passed	The description accurately explains the Maven retry, concurrency, mirror, build configuration, and validation changes. It is directly related to the changeset.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 1…
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 1 files. (1 skipped: 1 unsupported.)

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch java-maven-retry-config

_{Comment @coderabbitai help to get the list of available commands.}

github-actions · 2026-09-01T14:03:58Z

CI Test Summary

✅ All 31 test job(s) passed.

java/cuopt/.mvn/maven.config is only auto-discovered when Maven's directory search (which walks up from the current working directory, not from -f's directory) reaches java/cuopt. build.sh invoked cuopt_mvn with -f but never changed into that directory, so the transport-layer retry/backoff settings added for the Maven Central 429 rate-limiting never took effect. test.sh already cd's into MODULE_DIR for the same reason; mirror that here, matching how cuDF cd's into java/ before its mvn calls for the same .mvn/maven.config. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…y mvn test/verify maven-source-plugin and maven-javadoc-plugin were declared unconditionally in <build><plugins>, so every `mvn test`/`verify` invocation -- including ci/test_java.sh's, which never packages anything for publishing -- still had to resolve maven-source-plugin:3.3.1 from Maven Central on every run. That's the same artifact that's repeatedly failed on 429 in the java-static-test investigation on #1818, and there's no reason java-build's plain test path should pay for it either. No script on this branch currently packages a JAR for real publishing (only java/cuopt/scripts/test.sh's `mvn verify`, for testing, and build.sh's `mvn clean package`, for local dev -- neither needs sources/javadoc jars), so nothing needs the new attach-source-javadoc profile activated; this is a pure reduction in unnecessary Maven Central resolution. Verified locally: `mvn compile` no longer resolves maven-source-plugin/maven-javadoc-plugin at all (checked via -X debug output).

ramakrishnap-nv · 2026-09-01T16:33:53Z

/ok to test af54fa7

…Handler.* The aether.transport.http.retryHandler.* properties (copied from cuDF/cuVS) are not read by Maven Resolver's 429/503 backoff feature; per the resolver's own ConfigurationProperties source (MRESOLVER-396, PREFIX_CONNECTOR), the correct namespace is aether.connector.http.retryHandler.*. Verified via mvn help:evaluate that these values actually resolve.

ramakrishnap-nv · 2026-09-01T21:13:14Z

/ok to test 7f776e8

NVIDIA-managed GitHub Actions runners egress through a small, shared NAT'd IP range, so every RAPIDS repo's Java CI shares the same rate-limit budget against Maven Central (rapidsai/build-infra#370): cuDF, cuVS, cuVS-lucene, and kvikio have all hit the same 429s we're seeing on exec-maven-plugin. kvikio#992 fixed it there by preferring the read-only GCS mirror of Central (the same one Apache ORC/Lucene/Spark use) with Central as fallback. Applying the same fix here. Verified locally: every plugin, including exec-maven-plugin, now resolves from the GCS mirror.

ramakrishnap-nv · 2026-09-02T00:52:15Z

/ok to test 82e4e86

ramakrishnap-nv requested a review from a team as a code owner September 1, 2026 13:29

ramakrishnap-nv requested a review from Iroy30 September 1, 2026 13:29

ramakrishnap-nv self-assigned this Sep 1, 2026

ramakrishnap-nv added non-breaking Introduces a non-breaking change improvement Improves an existing functionality labels Sep 1, 2026

ramakrishnap-nv requested a review from a team as a code owner September 1, 2026 15:10

ramakrishnap-nv requested a review from gforsyth September 1, 2026 15:10

ramakrishnap-nv and others added 3 commits September 1, 2026 14:16

Merge branch 'main' into java-maven-retry-config

c622743

Merge remote-tracking branch 'origin/main' into java-maven-retry-config

07b3afb

Iroy30 approved these changes Sep 1, 2026

View reviewed changes

Navigation Menu

Uh oh!

Tune Maven's own retry/concurrency to reduce Maven Central rate-limiting - #1835

Tune Maven's own retry/concurrency to reduce Maven Central rate-limiting#1835
ramakrishnap-nv wants to merge 7 commits into
mainfrom
java-maven-retry-config

ramakrishnap-nv commented Sep 1, 2026

Uh oh!

coderabbitai Bot commented Sep 1, 2026 •

edited

Loading

Reviews paused

Walkthrough

Changes

Uh oh!

github-actions Bot commented Sep 1, 2026 •

edited

Loading

Uh oh!

ramakrishnap-nv commented Sep 1, 2026

Uh oh!

ramakrishnap-nv commented Sep 1, 2026

Uh oh!

ramakrishnap-nv commented Sep 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

ramakrishnap-nv commented Sep 1, 2026

Summary

Fix

Test plan

Uh oh!

coderabbitai Bot commented Sep 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviews paused

Walkthrough

Changes

Uh oh!

github-actions Bot commented Sep 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CI Test Summary

Uh oh!

ramakrishnap-nv commented Sep 1, 2026

Uh oh!

ramakrishnap-nv commented Sep 1, 2026

Uh oh!

ramakrishnap-nv commented Sep 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

coderabbitai Bot commented Sep 1, 2026 •

edited

Loading

github-actions Bot commented Sep 1, 2026 •

edited

Loading