Skip to content

perf(db): bound multi-worker connection pools - #789

Open
dr-hoseyn wants to merge 1 commit into
PasarGuard:devfrom
dr-hoseyn:codex/perf-db-pool-budget
Open

perf(db): bound multi-worker connection pools#789
dr-hoseyn wants to merge 1 commit into
PasarGuard:devfrom
dr-hoseyn:codex/perf-db-pool-budget

Conversation

@dr-hoseyn

@dr-hoseyn dr-hoseyn commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Summary

  • reduce the default non-SQLite pool ceiling from 85 to 10 connections per process so documented multi-worker deployments do not multiply unsafe defaults
  • preserve explicit pool overrides, validate unsafe values, and expose SQLALCHEMY_POOL_TIMEOUT
  • log the credential-free per-process and configured service connection ceilings at startup
  • document the deployment-wide formula and cover PostgreSQL, MySQL, SQLite, override, validation, and summary behavior

Closes #788

Type of change

  • Bug fix
  • New feature
  • Breaking change
  • Refactor / cleanup
  • Documentation
  • Tests / CI

Checklist

  • I tested the change locally or explained why it cannot be tested.
  • I added or updated tests for behavior changes.
  • I updated documentation, translations, or examples if needed.
  • I checked database migrations when models or schema changed. (No schema change.)
  • I did not include secrets, tokens, private keys, or unrelated changes.

Testing

  • .venv/Scripts/python.exe -m ruff check . — passed
  • focused pool/app-factory tests — 17 passed
  • full SQLite suite on a clean database — 535 passed, 2 skipped

Screenshots

Not applicable (backend/configuration-only change).

Notes for reviewers

  • SQLALCHEMY_POOL_SIZE and SQLALCHEMY_MAX_OVERFLOW remain per-process settings and explicit environment values retain their existing meaning.
  • Defaults change from 25 + 60 to 5 + 5, limiting four Uvicorn workers to 40 potential connections instead of 340. Operators with a deliberately sized external pool can keep larger values explicitly.
  • SQLite continues to omit QueuePool options.
  • Startup summaries never include the database URL, host, username, or password.
  • No migration or API contract change.

Summary by CodeRabbit

  • Improvements

    • Database connection pooling now uses safer, bounded defaults to improve resource management.
    • Added configurable connection timeouts and validation for pool settings.
    • Startup logs now report effective database connection limits without exposing connection details.
    • SQLite connections avoid unsupported pooling options, while MySQL retains appropriate connection settings.
  • Bug Fixes

    • Invalid pool configurations and process counts are now rejected early with clear validation errors.

@dr-hoseyn

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 14, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai

coderabbitai Bot commented Aug 14, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 03038ba2-f02d-44ac-b60a-2aef230804b8

📥 Commits

Reviewing files that changed from the base of the PR and between 3331421 and 6ab1359.

📒 Files selected for processing (5)
  • .env.example
  • app/app_factory.py
  • app/db/base.py
  • config.py
  • tests/test_database_pool_config.py

Walkthrough

Database pool defaults, validation, engine configuration, connection ceiling calculation, startup reporting, documentation, and regression tests were updated for per-process budgeting.

Changes

Database pool budget

Layer / File(s) Summary
Pool settings and connection ceiling contract
config.py, .env.example
DatabaseSettings now validates pool values, uses bounded defaults, adds pool_timeout, and calculates per-process connection ceilings. The example configuration documents per-process limits.
Dialect-specific engine configuration
app/db/base.py
Engine options now use configured pooling for non-SQLite databases and retain backend-specific connection arguments. Pool summaries report sanitized per-process and service-wide ceilings.
Startup reporting and regression coverage
app/app_factory.py, tests/test_database_pool_config.py
Application startup logs the database pool budget. Tests cover defaults, overrides, SQLite and MySQL behavior, validation, ceilings, and URL redaction.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: ⚪ Minimal · up to 6ab13

The PR bounds default database connection pools while preserving explicit overrides and adds validation, logging, and documentation; no actionable merge-blocking risk remains after normal checks and review.

Suggested reviewers: m03ed, x0sina, immohammad20000

Poem

A rabbit tuned the pools with care,
Five connections here, five overflow there.
At startup, ceilings shine,
Safe summaries trace the line.
“No secrets spilled!” the rabbit sings.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The changes satisfy the linked issue objectives for bounded defaults, overrides, validation, startup budgeting, documentation, and regression tests [#788].
Out of Scope Changes check ✅ Passed All reported changes support the linked issue objectives and contain no unrelated code or configuration changes.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: limiting database connection pools for multi-worker deployments.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant