Skip to content

fix(oracle): load large schemas on demand - #771

Open
2160039878-cyber wants to merge 2 commits into
libredb:mainfrom
2160039878-cyber:fix/oracle-lazy-schema-765
Open

fix(oracle): load large schemas on demand#771
2160039878-cyber wants to merge 2 commits into
libredb:mainfrom
2160039878-cyber:fix/oracle-lazy-schema-765

Conversation

@2160039878-cyber

@2160039878-cyber 2160039878-cyber commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Description

Oracle connections with tens of thousands of tables currently read every column, constraint and index on connect and after DDL. This change loads the owner-scoped table inventory first and reads one table's details when requested. The explorer and command palette also bound their rendered table rows, so the inventory itself does not mount tens of thousands of controls.

Closes #765.

Type of Change

  • Bug fix
  • Performance improvement
  • Documentation update
  • Test addition or update

Changes Made

  • getSchemaList() reads only ALL_TABLES names and estimated row counts. Inventory entries explicitly carry detailsLoaded: false; empty columns are not treated as a completed read.
  • getTableSchema(name) applies owner and table binds to each of the existing five catalog queries. The existing full getSchema() remains available. Authenticated POST /api/db/schema?table=... returns a one-element array, with 400 for an empty name and 404 for an absent/invisible table; providers without a single-table reader retain a full-reader fallback.
  • Both shells load details for expansion and table tools, coalesce duplicate requests, and discard responses invalidated by connection changes or schema refreshes. Query generation waits for the actual columns. Embedded hosts can opt into onSchemaListFetch and onTableSchemaFetch; the required onSchemaFetch still supplies a complete schema.
  • The explorer renders 100 tables per page and searches all names plus already-loaded columns. The command palette shows up to 100 matching tables. ERD, documentation and schema comparison require an explicit Load full schema action when details are pending, including after refresh.
  • Keep the Oracle provider implementation, provider documentation and integration tests together; update API and embedding contracts.

Testing

  • TDD: the new 43,500-table inventory, table-bound query and bounded explorer regressions failed against the original implementation.
  • Targeted project runners: 154 Oracle integration tests, 41 schema API tests, 5 API documentation type tests, 112 connection/detail/tab hook tests, and 335 relevant component tests passed. Additional lazy-loading component regressions were rerun successfully after the wiring changes.
  • Actual production components in headless Microsoft Edge with a 43,500-table synthetic catalog: first/next pages contain 100 rows; the last table is searchable; expanding it performs one table read and shows its column; typing in the editor works; refresh resets details without another bulk read; the command palette finds the last table; full metadata is fetched only after clicking its explicit load button. Switching connections resets loaded details and search state.
  • Passed local gates: format, lint, typecheck, knip, readme:check, chart:check, channels:showcase:check, security:check, build, build:lib, attw. Builds used clean commit c44c5f07b416ae637f0e5b30366b6fc078c67391.
  • Full local suite/coverage/app E2E were not run on this Windows host (Helm/chart dependencies and working Docker are unavailable; existing SQLite cleanup encounters Windows file locks). The required CI jobs remain the full-suite and 100% line-coverage gate.
  • Remote CI on final commit 30eaa32277f79a0b0e4024a6bed435e63052dfa7: 20 checks succeeded and 2 conditional checks skipped. The full suite, 100% line-coverage gate, browser E2E, PostgreSQL functional smoke, Node 24/26 engine smoke and channel E2E passed.

Environment: Windows, Node.js 24.18.1, Bun 1.4.2, Microsoft Edge. Oracle catalog-query tests use the project's driver mock; no live Oracle 19c/PeopleSoft/NNE environment was available. The browser fixture validates the real UI and hook code with synthetic metadata, not a real Oracle server.

Checklist

  • Claimed the issue before implementation and reviewed the final diff.
  • Followed the existing provider/API/UI conventions without adding a dependency.
  • Added regressions, updated documentation and kept the provider triad together.
  • Relevant local tests and required local build/static gates pass.
  • Required CI jobs pass, including the 100% line-coverage gate (verified on head 30eaa32277f79a0b0e4024a6bed435e63052dfa7: 20 SUCCESS, 2 SKIPPED).

Additional Notes

AI-assisted implementation and validation using Codex. Whole-schema operations and AI workflows that explicitly request the complete schema can still be expensive on a large owner; this change removes that cost from automatic explorer refreshes. Column search/autocomplete grows as tables are loaded. Existing owner-only visibility and row-count estimates are preserved.

@cevheri cevheri added the loop:needs-moderator-action Flagged by the maintainer loop: suspicious content or a decision only a human can make label Sep 10, 2026
@codecov

codecov Bot commented Sep 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

loop:needs-moderator-action Flagged by the maintainer loop: suspicious content or a decision only a human can make

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Oracle: eager full-owner getSchema() on connect freezes the UI on large schemas (no lazy/paged introspection)

2 participants