Skip to content

fix(core): validate CPU/IO env vars with a diagnosable error#7856

Open
LuciferYang wants to merge 2 commits into
lance-format:mainfrom
LuciferYang:fix/validate-cpu-threads-env-vars
Open

fix(core): validate CPU/IO env vars with a diagnosable error#7856
LuciferYang wants to merge 2 commits into
lance-format:mainfrom
LuciferYang:fix/validate-cpu-threads-env-vars

Conversation

@LuciferYang

Copy link
Copy Markdown
Contributor

Summary

LANCE_CPU_THREADS and LANCE_IO_CORE_RESERVATION were parsed with .parse().unwrap(). Two problems followed from a misconfigured value:

  1. A non-integer value panicked inside the LazyLock initializer with a bare ParseIntError that named neither the variable nor the cause. A stray quote or trailing space (common in .env files, Docker, and k8s manifests) was enough to trigger it, and the process stayed panicked because the poisoned LazyLock re-panics on every subsequent access.
  2. LANCE_CPU_THREADS=0 parsed fine, then flowed into create_runtime's max_blocking_threads(0), which asserts inside tokio (Max blocking threads cannot be set to 0) on the first compute task. The panic surfaced deep in tokio, far from the actual misconfiguration, with no hint that the CPU-threads variable was at fault.

Both variables are documented as user-tunable in the performance guide, so a hand-set bad value is a realistic, reachable failure — not a corrupt-input edge case.

Change

Route both through a small pure helper, parse_env_usize(name, raw, min), which trims surrounding whitespace, names the offending variable in the error, and rejects values below a per-variable minimum. LANCE_CPU_THREADS must be at least 1 (the floor tokio requires); LANCE_IO_CORE_RESERVATION keeps allowing 0, meaning reserve no cores for IO. Unset still defaults to 2. Bad values still fail fast — the contract is unchanged — but now with an actionable message instead of a bare ParseIntError or an unrelated tokio assertion.

The only inputs newly rejected are ones that never worked: LANCE_CPU_THREADS=0 (previously a guaranteed tokio assertion) and values that already panicked as unparseable. Whitespace-padded values that used to panic now parse, so the change only widens the set of accepted configurations aside from those two.

Test plan

The variables feed process-global LazyLocks that read once and are read in parallel by other tests, so the environment cannot be mutated reliably from a test. The pure parser is unit-tested directly instead:

  • parses_valid_value_and_trims_surrounding_whitespace
  • rejects_non_integer_naming_the_variable
  • rejects_value_below_minimum (the LANCE_CPU_THREADS=0 case)
  • allows_zero_when_minimum_is_zero (the LANCE_IO_CORE_RESERVATION=0 case)

cargo fmt --all and cargo clippy -p lance-core --tests -- -D warnings are clean.

`LANCE_CPU_THREADS` and `LANCE_IO_CORE_RESERVATION` were parsed with
`.parse().unwrap()`. A non-integer value panicked inside the `LazyLock`
init with a bare `ParseIntError` naming neither the variable nor the
cause. Worse, `LANCE_CPU_THREADS=0` parsed fine, then flowed into
`max_blocking_threads(0)`, which asserts inside tokio on the first
compute task — a confusing panic far from the misconfiguration.

Route both through `parse_env_usize`, which trims surrounding
whitespace, names the offending variable in the error, and rejects
values below a per-variable minimum: `LANCE_CPU_THREADS` must be at
least 1, while `LANCE_IO_CORE_RESERVATION` still allows 0 (reserve no
cores for IO). Bad values still fail fast, now with an actionable
message.

The variables feed process-global `LazyLock`s that read once and are
read in parallel by other tests, so the pure parser is unit-tested
directly rather than by mutating the environment.
@github-actions github-actions Bot added the bug Something isn't working label Jul 20, 2026
@coderabbitai

coderabbitai Bot commented Jul 20, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: QUIET

Plan: Pro Plus

Run ID: 50ac3f05-9b16-4c60-a679-59f6e89f5aff

📥 Commits

Reviewing files that changed from the base of the PR and between 40f21b9 and c1460f2.

📒 Files selected for processing (1)
  • rust/lance-core/src/utils/tokio.rs

📝 Walkthrough

Walkthrough

CPU and I/O environment-variable parsing now uses a shared validator that trims input, enforces minimum values, reports named errors, and is covered by unit tests.

Changes

Environment variable parsing

Layer / File(s) Summary
Shared parser and CPU/I/O integration
rust/lance-core/src/utils/tokio.rs
parse_env_usize validates trimmed integer values and minimums; CPU thread parsing and I/O reservation initialization use it with explicit defaults and errors. Unit tests cover whitespace, invalid values, minimum bounds, and zero.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: wjones127

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: validating CPU/IO environment variables with better error messages.
Description check ✅ Passed The description is directly related to the changes and explains the validation and parsing updates in detail.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Note

Quiet mode is enabled, so only the most important comments were posted inline. Other review comments are grouped below.

🟡 Other comments (1)
rust/lance-core/src/utils/tokio.rs-24-25 (1)

24-25: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Reject non-Unicode environment values
std::env::var treats VarError::NotUnicode as an error, but both LANCE_CPU_THREADS and LANCE_IO_CORE_RESERVATION currently handle it as if the variable were unset. Match only NotPresent as absence and return a descriptive error for NotUnicode instead of silently falling back to the defaults.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@rust/lance-core/src/utils/tokio.rs` around lines 24 - 25, Update the
environment-variable handling in the CPU-thread and IO-core reservation logic
around LANCE_CPU_THREADS and LANCE_IO_CORE_RESERVATION to distinguish
VarError::NotPresent from VarError::NotUnicode. Continue using defaults only
when variables are absent, and return or panic with a descriptive error
containing the affected variable name when its value is not valid Unicode.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@rust/lance-core/src/utils/tokio.rs`:
- Around line 63-69: Add a Rust doc comment to the public static
IO_CORE_RESERVATION documenting the LANCE_IO_CORE_RESERVATION environment
variable, default value of 2, accepted nonnegative values including 0, and panic
behavior for invalid input. Explain its relationship to
get_num_compute_intensive_cpus, include a compiling example, and link to the
related API symbols and environment-variable documentation.
- Around line 24-25: Update the runtime initialization flow around the
LANCE_CPU_THREADS and corresponding second parse_env_usize call sites to return
a typed error instead of invoking panic!. Propagate the Result through the
initialization boundary, preserving the environment variable name and raw value
in the contextual error.

---

Other comments:
In `@rust/lance-core/src/utils/tokio.rs`:
- Around line 24-25: Update the environment-variable handling in the CPU-thread
and IO-core reservation logic around LANCE_CPU_THREADS and
LANCE_IO_CORE_RESERVATION to distinguish VarError::NotPresent from
VarError::NotUnicode. Continue using defaults only when variables are absent,
and return or panic with a descriptive error containing the affected variable
name when its value is not valid Unicode.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: QUIET

Plan: Pro Plus

Run ID: d34fff57-b221-44a6-a052-55fd8f450fde

📥 Commits

Reviewing files that changed from the base of the PR and between 654f0d4 and 40f21b9.

📒 Files selected for processing (1)
  • rust/lance-core/src/utils/tokio.rs

Comment thread rust/lance-core/src/utils/tokio.rs
Comment thread rust/lance-core/src/utils/tokio.rs
Document the `LANCE_IO_CORE_RESERVATION` variable, the default of 2, that
0 is allowed, the invalid-input behavior, and the relationship to
get_num_compute_intensive_cpus.
@codecov

codecov Bot commented Jul 20, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 88.23529% with 4 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
rust/lance-core/src/utils/tokio.rs 88.23% 3 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant