[flink] Respect low filesystem rate limits in remove_orphan_files

[platinumhamburg]

Search before asking

• I searched in the issues and found nothing similar.

Fluss version

0.9.0 (latest release)

Please describe the bug 🐞

remove_orphan_files assigns each scan subtask a minimum filesystem operation rate of 1/s. When the scan parallelism exceeds the configured total rate, this increases the aggregate rate above the requested limit.

For example, a total rate of 3 operations/s with parallelism 8 results in an aggregate target of 8 operations/s.

Solution

Allow fractional per-subtask rates by dividing the configured total rate by the scan parallelism, while retaining the existing best-effort rate-limiting semantics.

Are you willing to submit a PR?

• I'm willing to submit a PR!