diff --git a/docs/API_DOCS.md b/docs/API_DOCS.md index aa2becb1e..d82d81401 100644 --- a/docs/API_DOCS.md +++ b/docs/API_DOCS.md @@ -699,7 +699,9 @@ Redis is a key-value store, so the `sql` field carries a Redis command instead o #### POST /api/db/schema -Get database schema including tables, columns, indexes, and foreign keys. +Get database schema including tables, columns, indexes, and foreign keys. Add the optional +`?table=` query parameter to fetch one table's full detail; without it, the complete +schema is returned. **Authentication:** Required @@ -769,6 +771,10 @@ Get database schema including tables, columns, indexes, and foreign keys. ] ``` +With `?table=`, the `200 OK` response uses the same full `TableSchema` shape in a +single-element array. An empty `table` value returns `400 Bad Request`; a table that no longer +exists or is not visible returns `404` with `{ "error": "Table no longer exists or is not visible" }`. + **Response (503 Service Unavailable):** ```json { @@ -1316,6 +1322,7 @@ precedence; the connectivity check still uses its own 10000 ms timeout. ```typescript interface TableSchema { name: string; // Table name + detailsLoaded?: boolean; // False for inventory entries until table detail is loaded columns: ColumnSchema[]; // Column definitions indexes: IndexSchema[]; // Index definitions foreignKeys?: ForeignKeySchema[]; @@ -1605,6 +1612,10 @@ curl -X POST http://localhost:3000/api/db/schema \ }' ``` +Append `?table=` to the same request to load one table's full detail. The response is a +single-element array; an empty parameter returns `400`, and a missing or invisible table returns +`404`. + #### AI Explanation of a Plan ```bash curl -X POST http://localhost:3000/api/ai/explain \ diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 97bfa34ca..fae41605b 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -189,7 +189,7 @@ Multi-statement queries execute sequentially via `POST /api/db/multi-query`. Studio ships both as a standalone app and as the `@libredb/studio` npm package consumed by libredb-platform (built with `tsup` via `build:lib`). -- **`src/workspace/`** — `StudioWorkspace.tsx` is the embeddable shell. Its adapter hooks (`hooks/use-connection-adapter`, `hooks/use-query-adapter`) let the host (standalone or platform) supply connections and query execution, so the same UI runs in both contexts. +- **`src/workspace/`** — `StudioWorkspace.tsx` is the embeddable shell. Its adapter hooks (`hooks/use-connection-adapter`, `hooks/use-query-adapter`) let the host (standalone or platform) supply connections, query execution, and schema readers. `onSchemaFetch` is the required complete-schema reader; optional `onSchemaListFetch` supplies the fast inventory, and optional `onTableSchemaFetch` supplies one-table detail and falls back to the complete reader when omitted. - **`src/exports/`** — barrel modules (`components.ts`, `providers.ts`, `workspace.ts`, `types.ts`) that define the package's public surface; `package.json` `exports`/`main`/`module` point at the tsup `dist/` output. - **`src/styles/theme.css`** — the semantic colour tokens every exported component resolves through, shipped as `dist/styles.css` (`exports["./styles.css"]`) because `globals.css` is not packaged. A host imports it once: `import "@libredb/studio/styles.css"`. `build:lib` is `tsup && node scripts/copy-theme.mjs` in that order — tsup cleans `dist/`, so the copy has to follow it. See [`docs/ui/theming.md`](ui/theming.md). - Platform integration rules (Tailwind tokens, Lucide stroke widths, chunk scanning) live in `CLAUDE.md`. diff --git a/docs/providers/oracle.md b/docs/providers/oracle.md index 2f57e482b..4c20da294 100644 --- a/docs/providers/oracle.md +++ b/docs/providers/oracle.md @@ -154,15 +154,20 @@ deliberately stricter than node-oracledb's tokenizer, which opens a q-string at `q`/`Q` whatever comes before it; the strict side is the one whose mistake costs a bound — and, since #297, a confirmation prompt on that statement — rather than a misplaced clause. -### 3.3 Owner-scoped, five-query schema introspection +### 3.3 Owner-scoped, two-phase schema introspection -`getSchema()` ([`oracle.ts`](../../src/lib/db/providers/sql/oracle.ts)) runs **five bulk queries** -over the `ALL_*` data-dictionary views — tables, columns, primary keys, foreign keys, indexes — -all filtered by `OWNER = :1` (the connecting user, upper-cased) and then **grouped in memory** by -table. This is neither the Postgres single-CTE approach nor MySQL's per-table N+1: it is a fixed -5 round-trips regardless of table count. There is no `getSchemaList()`/`getSchemaRelations()` -(no two-phase split), and the returned `TableSchema` has **no `size` field** (only `rowCount` from -`NUM_ROWS`, an optimizer estimate that can be stale/`NULL`). +Automatic connection and DDL refresh use `getSchemaList()` ([`oracle.ts`](../../src/lib/db/providers/sql/oracle.ts)). +It reads only `ALL_TABLES`, filtered by `OWNER = :1` (the connecting user, upper-cased), and returns +table names plus the estimated `NUM_ROWS`. Each entry has `detailsLoaded: false` and empty +`columns`, `indexes`, and `foreignKeys` arrays. Expanding a table or using a table-level tool calls +`getTableSchema(tableName)`, which runs the five detail queries with both owner and table name bound +on every query; it returns `null` when no matching visible table exists. + +`getSchema()` remains the full-schema path and runs the five bulk queries over the `ALL_*` +data-dictionary views — tables, columns, primary keys, foreign keys, and indexes — all filtered by +`OWNER = :1` and grouped in memory. This is a fixed five round-trips regardless of table count. +There is still no `getSchemaRelations()`, and the returned `TableSchema` has **no `size` field** +(only `rowCount` from `NUM_ROWS`, an optimizer estimate that can be stale/`NULL`). ### 3.4 No transaction auto-rollback timeout @@ -832,8 +837,8 @@ Surfaced via `POST /api/db/transaction`. ## 7. Schema introspection -`getSchema()` returns one `TableSchema` per table owned by the connecting user. Five `ALL_*` queries -(`OWNER = :user`), grouped client-side: +`getSchema()` remains the complete schema read: one `TableSchema` per table owned by the connecting +user, populated by five `ALL_*` queries (`OWNER = :user`) grouped client-side: | Data | Source view(s) | |------|----------------| @@ -843,7 +848,15 @@ Surfaced via `POST /api/db/transaction`. | Foreign keys | `ALL_CONSTRAINTS` (type `'R'`) joined to the referenced constraint's columns | | Indexes | `ALL_INDEXES` + `ALL_IND_COLUMNS` (`unique` = `UNIQUENESS = 'UNIQUE'`) | -No `getSchemaList()`/`getSchemaRelations()`; no `size` on the returned tables (see [§3.3](#33-owner-scoped-five-query-schema-introspection)). +`getSchemaList()` is the fast inventory used by automatic connection and DDL refresh. It reads only +`ALL_TABLES` and returns names plus estimated `NUM_ROWS`, with `detailsLoaded: false` and empty +`columns`, `indexes`, and `foreignKeys`. Table expansion and table-level tools call +`getTableSchema(tableName)`, which repeats the five queries with `OWNER` and `TABLE_NAME` bound on +each query and returns `null` when the table is not visible. There is no `getSchemaRelations()`; +tables have no `size` field (see [§3.3](#33-owner-scoped-two-phase-schema-introspection)). + +Both shells show 100 tables per page and search all table names plus loaded columns. ERD, Docs, and +SchemaDiff require the user to click **Load full schema**; the full read may still be expensive. --- @@ -1235,12 +1248,15 @@ const provider = await createDatabaseProvider({ await provider.connect(); const res = await provider.query('SELECT id, email FROM users WHERE active = :1', [1]); -const schema = await provider.getSchema(); // 5 ALL_* queries, grouped in memory +const tables = await provider.getSchemaList(); // 1 ALL_TABLES query; detailsLoaded: false +const users = await provider.getTableSchema('USERS'); // 5 owner/table-bound queries, or null +const schema = await provider.getSchema(); // full 5 ALL_* queries, grouped in memory await provider.disconnect(); ``` Over the API: `POST /api/db/query`, `POST /api/db/transaction`, `POST /api/db/cancel`, -`POST /api/db/maintenance` (admin), `POST /api/db/schema/list` (falls back to `getSchema()`). +`POST /api/db/maintenance` (admin), `POST /api/db/schema/list` (fast inventory), and +`POST /api/db/schema?table=...` (one-element detail response or 404). --- @@ -1324,7 +1340,10 @@ Over the API: `POST /api/db/query`, `POST /api/db/transaction`, `POST /api/db/ca ([§7.2](#72-when-the-connection-count-is-not-measurable)). `getPerformanceMetrics()` reports only the cache-hit ratio (no QPS, deadlocks, or buffer-pool usage), and **omits even that** when `V$SYSSTAT` is unreadable rather than substituting a figure — [§7.1](#71-when-the-cache-hit-ratio-is-not-measurable). -- **No two-phase schema loading** — `/api/db/schema/list` falls back to the full `getSchema()`. +- **Schema loading is two-phase.** Automatic connection and DDL refresh use the cheap + `getSchemaList()` inventory; table expansion and table-level tools load one table on demand. + Both shells use the same 100-row paging and search table names plus loaded columns. ERD, Docs, and + SchemaDiff require **Load full schema**, and that complete read may still be expensive. --- diff --git a/src/app/api/db/schema/list/route.ts b/src/app/api/db/schema/list/route.ts index 7df14eacf..895a530cf 100644 --- a/src/app/api/db/schema/list/route.ts +++ b/src/app/api/db/schema/list/route.ts @@ -4,13 +4,9 @@ import { handleSchemaRequest } from "@/lib/api/schema-route"; export const dynamic = "force-dynamic"; /** - * Fast structural schema (tables + columns + PKs), excluding the expensive - * foreign-key/index introspection. Used by the schema explorer to render the - * table tree immediately; relationships/indexes are fetched separately via - * /api/db/schema/relations and merged in asynchronously. - * - * Falls back to the full getSchema() for providers that don't implement the - * fast path, so non-postgres databases keep working unchanged. + * Fast schema list. Structural lists are enriched with /schema/relations; + * name-only inventories (detailsLoaded:false) use /schema?table=... on demand. + * Providers without a fast list fall back to the full getSchema(). */ export async function POST(req: NextRequest) { return handleSchemaRequest(req, "api/db/schema/list", (provider) => diff --git a/src/app/api/db/schema/route.ts b/src/app/api/db/schema/route.ts index 576dffcae..fb190a0fe 100644 --- a/src/app/api/db/schema/route.ts +++ b/src/app/api/db/schema/route.ts @@ -35,6 +35,16 @@ export async function POST(req: NextRequest) { } const provider = await getOrCreateProvider(connection); + const tableName = new URL(req.url).searchParams.get("table"); + if (tableName !== null) { + if (!tableName) return NextResponse.json({ error: "Table name is required" }, { status: 400 }); + const table = provider.getTableSchema + ? await provider.getTableSchema(tableName) + : (await provider.getSchema()).find((entry) => entry.name === tableName); + return table + ? NextResponse.json([table]) + : NextResponse.json({ error: "Table no longer exists or is not visible" }, { status: 404 }); + } const schema = await provider.getSchema(); return NextResponse.json(schema); diff --git a/src/components/CommandPalette.tsx b/src/components/CommandPalette.tsx index 4f80534c8..c651e92af 100644 --- a/src/components/CommandPalette.tsx +++ b/src/components/CommandPalette.tsx @@ -74,6 +74,11 @@ export function CommandPalette({ onLogout, }: CommandPaletteProps) { const [open, setOpen] = useState(false); + const [search, setSearch] = useState(""); + const visibleTables = useMemo(() => { + if (schema.length <= 100) return schema; + return schema.filter((table) => table.name.toLowerCase().includes(search.toLowerCase())).slice(0, 100); + }, [schema, search]); // Register Cmd+K / Ctrl+K keyboard shortcut useEffect(() => { @@ -81,6 +86,7 @@ export function CommandPalette({ if ((e.metaKey || e.ctrlKey) && e.key === "k") { e.preventDefault(); setOpen((prev) => !prev); + setSearch(""); } }; document.addEventListener("keydown", handleKeyDown); @@ -106,7 +112,12 @@ export function CommandPalette({ className="sm:max-w-[560px] bg-surface border-hairline-strong" showCloseButton={false} > - + No results found. @@ -180,13 +191,13 @@ export function CommandPalette({ {/* Tables */} {schema.length > 0 && ( - - {schema.map((table) => ( + 100 ? "Tables (up to 100 matches; type to narrow)" : "Tables"}> + {visibleTables.map((table) => ( runAction(() => onTableClick(table.name))}> {table.name} - {table.columns.length} cols + {table.detailsLoaded === false ? "Details on demand" : `${table.columns.length} cols`} {table.rowCount !== undefined && ` / ${table.rowCount} rows`} diff --git a/src/components/Studio.tsx b/src/components/Studio.tsx index e7b7db90a..bf36d433b 100644 --- a/src/components/Studio.tsx +++ b/src/components/Studio.tsx @@ -1,5 +1,6 @@ "use client"; +import { SchemaLoadGate } from "@/components/schema-explorer/SchemaLoadGate"; import type { CsvDelimiter } from "@/lib/export/csv"; import { appFetch } from "@/lib/config/base-path"; @@ -109,6 +110,7 @@ export default function Studio() { activeConnection: conn.activeConnection, metadata, schema: conn.schema, + ensureSchema: conn.ensureSchema, }); // 4. Transaction Control @@ -412,6 +414,16 @@ export default function Studio() { downloadText(file.content, file.mimeType, resultExportFileName(file.extension, hydrated?.runId)); }; + const openTableTool = (name: string, open: (name: string) => void) => { + if (conn.schema.find((table) => table.name === name)?.detailsLoaded === false) { + void conn.ensureSchema(name).then((details) => { + if (details) open(name); + }); + } else { + open(name); + } + }; + const onTableClick = (tableName: string) => { tabMgr.handleTableClick(tableName, queryExec.executeQuery); }; @@ -500,6 +512,7 @@ export default function Studio() { <> setProfilerTable(name)} - onGenerateCode={(name) => setCodeGenTable(name)} - onGenerateTestData={(name) => setTestDataTable(name)} + onProfileTable={(name) => openTableTool(name, setProfilerTable)} + onGenerateCode={(name) => openTableTool(name, setCodeGenTable)} + onGenerateTestData={(name) => openTableTool(name, setTestDataTable)} /> @@ -598,7 +611,15 @@ export default function Studio() { } > - setShowDiagram(false)} /> + setShowDiagram(false)} + className="absolute inset-0 z-20" + > + setShowDiagram(false)} /> + )} @@ -637,6 +658,8 @@ export default function Studio() {
{conn.activeConnection ? ( setProfilerTable(name)} - onGenerateCode={(name) => setCodeGenTable(name)} - onGenerateTestData={(name) => setTestDataTable(name)} + onProfileTable={(name) => openTableTool(name, setProfilerTable)} + onGenerateCode={(name) => openTableTool(name, setCodeGenTable)} + onGenerateTestData={(name) => openTableTool(name, setTestDataTable)} /> ) : (
@@ -708,6 +731,7 @@ export default function Studio() { Promise; onTableClick?: (tableName: string) => void; onGenerateSelect?: (tableName: string) => void; onCreateTableClick?: () => void; @@ -34,6 +35,7 @@ export function SchemaExplorer({ schema, isLoadingSchema, schemaError = null, + onLoadTable, onTableClick, onGenerateSelect, onCreateTableClick, @@ -46,6 +48,7 @@ export function SchemaExplorer({ }: SchemaExplorerProps) { const labels = metadata?.labels; const capabilities = metadata?.capabilities; + const [page, setPage] = useState(0); const [searchQuery, setSearchQuery] = useState(""); const [expandedTables, setExpandedTables] = useState>(new Set()); @@ -72,6 +75,12 @@ export function SchemaExplorer({ }); }, [schema, searchQuery]); + const pageSize = 100; + const lastPage = Math.max(0, Math.ceil(filteredSchema.length / pageSize) - 1); + const currentPage = Math.min(page, lastPage); + const visibleSchema = filteredSchema.slice(currentPage * pageSize, (currentPage + 1) * pageSize); + const hasUnloadedTables = schema.some((table) => table.detailsLoaded === false); + if (isLoadingSchema) { return (
@@ -166,25 +175,61 @@ export function SchemaExplorer({ className="absolute left-2.5 top-1/2 -translate-y-1/2 w-3.5 h-3.5 text-muted-foreground group-focus-within:text-brand transition-colors" /> setSearchQuery(e.target.value)} + onChange={(e) => { + setSearchQuery(e.target.value); + setPage(0); + }} className="h-8 pl-8 pr-8 text-xs bg-muted/50 border-border focus-visible:ring-1 focus-visible:ring-brand-tint/50 placeholder:text-muted-foreground/50" /> {searchQuery && ( )}
+ {lastPage > 0 && ( + + )}
- {filteredSchema.map((table) => ( + {visibleSchema.map((table) => ( Promise; + onClose?: () => void; + className?: string; + children: ReactNode; +}) { + const [loading, setLoading] = useState(false); + if (!schema.some((table) => table.detailsLoaded === false)) return children; + return ( +
+

This view needs details for all {schema.length.toLocaleString()} tables.

+

Loading the full schema can take time and memory for large databases.

+ + {onClose && ( + + )} +
+ ); +} diff --git a/src/components/schema-explorer/TableItem.tsx b/src/components/schema-explorer/TableItem.tsx index 73a4da6b2..0dbd8a3b8 100644 --- a/src/components/schema-explorer/TableItem.tsx +++ b/src/components/schema-explorer/TableItem.tsx @@ -40,6 +40,7 @@ interface TableItemProps { table: TableSchema; isExpanded: boolean; onToggle: () => void; + onLoadTable?: (tableName: string) => Promise; // `labels` is itself optional on ProviderMetadata, so the indexed access already // carries `undefined`; NonNullable keeps the `?` from restating it (#427). labels?: NonNullable; @@ -193,6 +194,7 @@ export const TableItem = React.memo(function TableItem({ table, isExpanded, onToggle, + onLoadTable, labels, capabilities, isAdmin, @@ -203,6 +205,15 @@ export const TableItem = React.memo(function TableItem({ onGenerateTestData, onOpenMaintenance, }: TableItemProps) { + const [loadingDetails, setLoadingDetails] = React.useState(false); + const loadDetails = async () => { + setLoadingDetails(true); + try { + await onLoadTable?.(table.name); + } finally { + setLoadingDetails(false); + } + }; const copyToClipboard = (text: string, label: string) => { // The toast waits for the write to report an outcome (B43). It used to fire in the // same statement that started it, which announced a copy that never happened over @@ -237,7 +248,10 @@ export const TableItem = React.memo(function TableItem({ type="button" aria-expanded={isExpanded} className="flex items-center gap-1.5 flex-1 min-w-0 py-1.5 cursor-pointer text-left" - onClick={onToggle} + onClick={() => { + onToggle(); + if (!isExpanded && table.detailsLoaded === false && !loadingDetails) void loadDetails(); + }} > @@ -317,7 +331,19 @@ export const TableItem = React.memo(function TableItem({ transition={{ duration: 0.2 }} className="overflow-hidden" > - + {table.detailsLoaded === false ? ( +
+ {loadingDetails ? ( + Loading table details... + ) : ( + + )} +
+ ) : ( + + )} )} diff --git a/src/components/sidebar/Sidebar.tsx b/src/components/sidebar/Sidebar.tsx index 1e7c9304e..b8995a25e 100644 --- a/src/components/sidebar/Sidebar.tsx +++ b/src/components/sidebar/Sidebar.tsx @@ -23,6 +23,7 @@ interface SidebarProps { onEditConnection?: (conn: DatabaseConnection) => void; onDuplicateConnection?: (conn: DatabaseConnection) => void; onAddConnection: () => void; + onLoadTable?: (tableName: string) => Promise; onTableClick?: (tableName: string) => void; onGenerateSelect?: (tableName: string) => void; onCreateTableClick?: () => void; @@ -47,6 +48,7 @@ export function Sidebar({ onEditConnection, onDuplicateConnection, onAddConnection, + onLoadTable, onTableClick, onGenerateSelect, onCreateTableClick, @@ -105,6 +107,8 @@ export function Sidebar({ {activeConnection && ( void; currentTab: QueryTab; schema: TableSchema[]; + onLoadSchema?: () => Promise; schemaContext: string; activeConnection: DatabaseConnection | null; metadata: ProviderMetadata | null; @@ -187,6 +189,7 @@ export function BottomPanel({ onSetMode, currentTab, schema, + onLoadSchema, schemaContext, activeConnection, metadata, @@ -476,7 +479,9 @@ export function BottomPanel({ databaseType={activeConnection?.type} /> ) : mode === "docs" ? ( - + + + ) : mode === "history" ? ( ) : mode === "schemadiff" ? ( - + + + ) : mode === "dashboard" ? ( ) : mode === "explain" ? ( diff --git a/src/hooks/use-connection-manager.ts b/src/hooks/use-connection-manager.ts index 006d3c5c0..d4b72037f 100644 --- a/src/hooks/use-connection-manager.ts +++ b/src/hooks/use-connection-manager.ts @@ -3,6 +3,7 @@ import { appFetch } from "@/lib/config/base-path"; import { useState, useEffect, useCallback, useMemo } from "react"; import type { DatabaseConnection, TableSchema, TableRelations } from "@/lib/types"; +import { useSchemaDetails } from "@/hooks/use-schema-details"; import { useToast } from "@/hooks/use-toast"; import { storage } from "@/lib/storage"; import { logger } from "@/lib/logger"; @@ -47,12 +48,38 @@ export function useConnectionManager(storageReady = false) { const { toast } = useToast(); - // Fetch schema for a connection — two phases so a slow/failing stats query - // never blocks the table list: - // 1. /api/db/schema/list → tables + columns + PKs (fast) → render tree - // 2. /api/db/schema/relations → foreign keys + indexes (heavy) → async merge + const loadDetails = useCallback( + async (tableName?: string): Promise => { + const conn = activeConnection!; + const payload = conn.managed && conn.seedId ? { connectionId: `seed:${conn.seedId}` } : conn; + const response = await appFetch( + `/api/db/schema${tableName === undefined ? "" : `?table=${encodeURIComponent(tableName)}`}`, + { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify(payload), + }, + ); + if (!response.ok) { + const error = await response.json().catch(() => ({})); + throw new Error(error.error || "Failed to fetch table details"); + } + return response.json(); + }, + [activeConnection], + ); + const { ensureSchema, startSchemaLoad, isLoadingFullSchema } = useSchemaDetails( + activeConnection?.id, + schema, + setSchema, + loadDetails, + ); + + // Structural lists can load relationships in the background. A name-only + // inventory skips that bulk work and loads details only for requested tables. const fetchSchema = useCallback( async (conn: DatabaseConnection) => { + const isCurrent = startSchemaLoad(); setIsLoadingSchema(true); const payload = conn.managed && conn.seedId ? { connectionId: `seed:${conn.seedId}` } : conn; // bare conn for backward compat with schema route @@ -69,9 +96,13 @@ export function useConnectionManager(storageReady = false) { throw new Error(errorData.error || "Failed to fetch schema"); } const list: TableSchema[] = await response.json(); + if (!isCurrent()) return; setSchema(list); setSchemaError(null); + // Inventories defer all column/key/index reads until a table is requested. + if (list.some((table) => table.detailsLoaded === false)) return; } catch (error) { + if (!isCurrent()) return; const errorMessage = error instanceof Error ? error.message : "Unknown error"; // Nothing read for THIS connection, so nothing may stay on screen as its // tables — the previous connection's list is not evidence about this one. @@ -80,7 +111,7 @@ export function useConnectionManager(storageReady = false) { toast({ title: "Schema Error", description: errorMessage, variant: "destructive" }); return; // finally still clears the loading flag; skip relations } finally { - setIsLoadingSchema(false); + if (isCurrent()) setIsLoadingSchema(false); } // Phase 2 — relationships + indexes (best-effort; never breaks the list) @@ -91,6 +122,7 @@ export function useConnectionManager(storageReady = false) { throw new Error(errorData.error || "Failed to fetch schema relations"); } const relations: TableRelations[] = await relRes.json(); + if (!isCurrent()) return; const byName = new Map(relations.map((r) => [r.name, r])); setSchema((prev) => prev.map((t) => { @@ -105,7 +137,7 @@ export function useConnectionManager(storageReady = false) { }); } }, - [toast], + [toast, startSchemaLoad], ); // Memoized derived values @@ -322,7 +354,8 @@ export function useConnectionManager(storageReady = false) { schema, setSchema, schemaError, - isLoadingSchema, + isLoadingSchema: isLoadingSchema || isLoadingFullSchema, + ensureSchema, // Derived rather than reset in the pulse effect: with no active connection // there is nothing to report on, and the render already knows that. connectionPulse: activeConnection === null ? null : pulseState, diff --git a/src/hooks/use-schema-details.ts b/src/hooks/use-schema-details.ts new file mode 100644 index 000000000..666a911dc --- /dev/null +++ b/src/hooks/use-schema-details.ts @@ -0,0 +1,82 @@ +"use client"; + +import { useCallback, useEffect, useRef, useState, type Dispatch, type SetStateAction } from "react"; +import type { TableSchema } from "@/lib/types"; +import { useToast } from "@/hooks/use-toast"; + +/** Shared by the standalone and host-managed shells. A refresh invalidates pending details. */ +export function useSchemaDetails( + connectionId: string | undefined, + schema: TableSchema[], + setSchema: Dispatch>, + load: (tableName?: string) => Promise, +) { + const generation = useRef(0); + const pending = useRef(new Map }>()); + const [isLoadingFullSchema, setIsLoadingFullSchema] = useState(false); + const { toast } = useToast(); + + useEffect(() => { + if (!connectionId) return; + const requests = pending.current; + return () => { + generation.current += 1; + requests.clear(); + setIsLoadingFullSchema(false); + }; + }, [connectionId]); + + const startSchemaLoad = useCallback(() => { + const version = ++generation.current; + return () => version === generation.current; + }, []); + + const ensureSchema = useCallback( + async (tableName?: string): Promise => { + if (!connectionId) return null; + const selected = tableName === undefined ? schema : schema.filter((table) => table.name === tableName); + if (tableName !== undefined && selected.length === 0) return null; + if (selected.every((table) => table.detailsLoaded !== false)) return selected; + + const existing = pending.current.get(tableName); + if (existing?.version === generation.current) return existing.request; + const version = generation.current; + if (tableName === undefined) setIsLoadingFullSchema(true); + const request: Promise = Promise.resolve().then(async () => { + try { + const details = await load(tableName); + if (version !== generation.current) return null; + if ( + details.some((table) => table.detailsLoaded === false) || + (tableName !== undefined && (details.length !== 1 || details[0].name !== tableName)) + ) { + throw new Error("Table details could not be read. Refresh the schema and try again."); + } + setSchema((current) => + tableName === undefined ? details : current.map((table) => (table.name === tableName ? details[0] : table)), + ); + return details; + } catch (error) { + if (version === generation.current) { + toast({ + title: "Schema Error", + description: error instanceof Error ? error.message : "Failed to fetch table details", + variant: "destructive", + }); + } + return null; + } finally { + if (pending.current.get(tableName)?.request === request) { + pending.current.delete(tableName); + if (tableName === undefined) setIsLoadingFullSchema(false); + } + } + }); + pending.current.set(tableName, { version, request }); + return request; + }, + [connectionId, schema, setSchema, load, toast], + ); + + return { ensureSchema, startSchemaLoad, isLoadingFullSchema }; +} diff --git a/src/hooks/use-tab-manager.ts b/src/hooks/use-tab-manager.ts index 440fc16e6..9e5da861e 100644 --- a/src/hooks/use-tab-manager.ts +++ b/src/hooks/use-tab-manager.ts @@ -36,9 +36,16 @@ interface UseTabManagerParams { metadata: ProviderMetadata | null; schema: TableSchema[]; persistWorkspace?: boolean; + ensureSchema?: (tableName: string) => Promise; } -export function useTabManager({ activeConnection, metadata, schema, persistWorkspace }: UseTabManagerParams) { +export function useTabManager({ + activeConnection, + metadata, + schema, + persistWorkspace, + ensureSchema, +}: UseTabManagerParams) { const [tabs, setTabs] = useState([DEFAULT_TAB]); const [activeTabId, setActiveTabId] = useState("default"); const [editingTabId, setEditingTabId] = useState(null); @@ -183,12 +190,17 @@ export function useTabManager({ activeConnection, metadata, schema, persistWorks // handleTableClick takes executeQuery as callback param to avoid circular dependency const handleTableClick = useCallback( - (tableName: string, executeQueryFn: (query: string, tabId: string) => void) => { + async (tableName: string, executeQueryFn: (query: string, tabId: string) => void) => { const capabilities = metadata?.capabilities; // Look the table up exactly as handleGenerateSelect does: the Redis // generator is type-aware, and the sampled key type lives on the schema // node's `type` column (#427). - const table = schema.find((t) => t.name === tableName); + let table = schema.find((t) => t.name === tableName); + if (table?.detailsLoaded === false) { + const details = await ensureSchema?.(tableName); + if (!details) return; + table = details[0]; + } const columns = table?.columns || []; const newQuery = capabilities ? generateTableQuery(tableName, capabilities, columns) @@ -207,13 +219,18 @@ export function useTabManager({ activeConnection, metadata, schema, persistWorks setActiveTabId(newId); setTimeout(() => executeQueryFn(newQuery, newId), 100); }, - [metadata, schema], + [metadata, schema, ensureSchema], ); const handleGenerateSelect = useCallback( - (tableName: string) => { + async (tableName: string) => { const capabilities = metadata?.capabilities; - const table = schema.find((t) => t.name === tableName); + let table = schema.find((t) => t.name === tableName); + if (table?.detailsLoaded === false) { + const details = await ensureSchema?.(tableName); + if (!details) return; + table = details[0]; + } const columns = table?.columns || []; const newQuery = capabilities @@ -236,7 +253,7 @@ export function useTabManager({ activeConnection, metadata, schema, persistWorks ]); setActiveTabId(newId); }, - [metadata, schema], + [metadata, schema, ensureSchema], ); return { diff --git a/src/lib/db/providers/sql/oracle.ts b/src/lib/db/providers/sql/oracle.ts index e81c2e110..c33203980 100644 --- a/src/lib/db/providers/sql/oracle.ts +++ b/src/lib/db/providers/sql/oracle.ts @@ -765,28 +765,62 @@ export class OracleProvider extends SQLBaseProvider { // Schema Operations // ============================================================================ - public async getSchema(): Promise { + public async getSchemaList(): Promise { + this.ensureConnected(); + const conn = await this.pool!.getConnection(); + try { + const result = await conn.execute( + "SELECT TABLE_NAME, NUM_ROWS FROM ALL_TABLES WHERE OWNER = :1 ORDER BY TABLE_NAME", + [this.config.user?.toUpperCase() || ""], + { outFormat: oracledb.OUT_FORMAT_OBJECT }, + ); + return ((result.rows || []) as Record[]).map((row) => ({ + name: String(row.TABLE_NAME || ""), + rowCount: Number(row.NUM_ROWS || 0), + columns: [], + indexes: [], + foreignKeys: [], + detailsLoaded: false, + })); + } finally { + await conn.close(); + } + } + + public getSchema(): Promise { + return this.loadSchema(); + } + + public async getTableSchema(tableName: string): Promise { + return (await this.loadSchema(tableName))[0] ?? null; + } + + private async loadSchema(tableName?: string): Promise { this.ensureConnected(); let conn: oracledb.Connection | undefined; try { conn = await this.pool!.getConnection(); const owner = this.config.user?.toUpperCase() || ""; + const params = tableName === undefined ? [owner] : [owner, tableName]; + // Each catalog starts with the owner bind. Keep that bind first and constrain + // every detail query on the server, before materializing any catalog rows. + const read = (sql: string, tableColumn = "TABLE_NAME") => + conn!.execute(tableName === undefined ? sql : sql.replace(":1", `:1 AND ${tableColumn} = :2`), params, { + outFormat: oracledb.OUT_FORMAT_OBJECT, + }); // Get tables - const tablesRes = await conn.execute( - `SELECT TABLE_NAME, NUM_ROWS FROM ALL_TABLES WHERE OWNER = :1 ORDER BY TABLE_NAME`, - [owner], - { outFormat: oracledb.OUT_FORMAT_OBJECT }, - ); + const tablesRes = await read(`SELECT TABLE_NAME, NUM_ROWS FROM ALL_TABLES WHERE OWNER = :1 ORDER BY TABLE_NAME`); const tables = (tablesRes.rows || []) as Record[]; + if (tableName !== undefined && tables.length === 0) return []; // Get columns - const colsRes = await conn.execute(SCHEMA_COLUMNS_SQL, [owner], { outFormat: oracledb.OUT_FORMAT_OBJECT }); + const colsRes = await read(SCHEMA_COLUMNS_SQL); const allCols = (colsRes.rows || []) as Record[]; // Get primary keys - const pkRes = await conn.execute(SCHEMA_PRIMARY_KEYS_SQL, [owner], { outFormat: oracledb.OUT_FORMAT_OBJECT }); + const pkRes = await read(SCHEMA_PRIMARY_KEYS_SQL, "ac.TABLE_NAME"); const pkRows = (pkRes.rows || []) as Record[]; const pkMap = new Map>(); for (const row of pkRows) { @@ -797,11 +831,11 @@ export class OracleProvider extends SQLBaseProvider { } // Get foreign keys - const fkRes = await conn.execute(SCHEMA_FOREIGN_KEYS_SQL, [owner], { outFormat: oracledb.OUT_FORMAT_OBJECT }); + const fkRes = await read(SCHEMA_FOREIGN_KEYS_SQL, "ac.TABLE_NAME"); const fkRows = (fkRes.rows || []) as Record[]; // Get indexes - const idxRes = await conn.execute(SCHEMA_INDEXES_SQL, [owner], { outFormat: oracledb.OUT_FORMAT_OBJECT }); + const idxRes = await read(SCHEMA_INDEXES_SQL, "ai.TABLE_NAME"); const idxRows = (idxRes.rows || []) as Record[]; // Group columns, indexes, foreign keys by table diff --git a/src/lib/db/types.ts b/src/lib/db/types.ts index ad4c031ea..7195258c3 100644 --- a/src/lib/db/types.ts +++ b/src/lib/db/types.ts @@ -582,12 +582,15 @@ export interface DatabaseProvider { getSchema(): Promise; /** - * Fast structural schema (tables + columns + PKs), excluding the expensive - * foreign-key/index introspection. Optional: providers that don't implement - * it fall back to getSchema(). Pairs with getSchemaRelations(). + * Fast schema list. May include columns/PKs (paired with getSchemaRelations), + * or name-only entries marked detailsLoaded:false (paired with getTableSchema). + * Providers without this method fall back to getSchema(). */ getSchemaList?(): Promise; + /** Load one inventory entry without scanning the other tables. Null means it no longer exists. */ + getTableSchema?(tableName: string): Promise; + /** * Heavy relationship/index data (foreign keys + indexes) keyed by table * display name, for async merge into getSchemaList() results. Optional. diff --git a/src/lib/types.ts b/src/lib/types.ts index a29835471..67367ba78 100644 --- a/src/lib/types.ts +++ b/src/lib/types.ts @@ -194,6 +194,8 @@ export interface DatabaseConnection { export interface TableSchema { name: string; + /** False for a table-name inventory entry whose columns/keys/indexes have not been loaded. */ + detailsLoaded?: boolean; columns: ColumnSchema[]; indexes: IndexSchema[]; foreignKeys?: ForeignKeySchema[]; diff --git a/src/workspace/StudioWorkspace.tsx b/src/workspace/StudioWorkspace.tsx index 4d308a278..8caa6230f 100644 --- a/src/workspace/StudioWorkspace.tsx +++ b/src/workspace/StudioWorkspace.tsx @@ -1,5 +1,6 @@ "use client"; +import { SchemaLoadGate } from "@/components/schema-explorer/SchemaLoadGate"; import type { CsvDelimiter } from "@/lib/export/csv"; import React, { useState, useEffect, useRef, useMemo, useCallback } from "react"; @@ -167,6 +168,8 @@ export function StudioWorkspace({ currentUser, onQueryExecute, onSchemaFetch, + onSchemaListFetch, + onTableSchemaFetch, onSaveQuery: onSaveQueryProp, // onLoadSavedQueries — reserved for future saved-queries panel integration features: featuresProp, @@ -182,6 +185,8 @@ export function StudioWorkspace({ const conn = useConnectionAdapter({ connections: externalConnections, onSchemaFetch, + onSchemaListFetch, + onTableSchemaFetch, }); // 2. Tab Manager (pure UI state, reused as-is) @@ -189,6 +194,7 @@ export function StudioWorkspace({ activeConnection: conn.activeConnection, metadata: conn.metadata, schema: conn.schema, + ensureSchema: conn.ensureSchema, }); // 3. Query Adapter (platform-delegated execution) @@ -276,6 +282,16 @@ export function StudioWorkspace({ [tabMgr.currentTab, conn.activeConnection?.type], ); + const openTableTool = (name: string, open: (name: string) => void) => { + if (conn.schema.find((table) => table.name === name)?.detailsLoaded === false) { + void conn.ensureSchema(name).then((details) => { + if (details) open(name); + }); + } else { + open(name); + } + }; + // === Table click handler === const onTableClick = useCallback( (tableName: string) => { @@ -313,6 +329,7 @@ export function StudioWorkspace({ <> setProfilerTable(name) : undefined} - onGenerateCode={features.codeGenerator ? (name: string) => setCodeGenTable(name) : undefined} - onGenerateTestData={features.testDataGenerator ? (name: string) => setTestDataTable(name) : undefined} + onProfileTable={ + features.codeGenerator ? (name: string) => openTableTool(name, setProfilerTable) : undefined + } + onGenerateCode={ + features.codeGenerator ? (name: string) => openTableTool(name, setCodeGenTable) : undefined + } + onGenerateTestData={ + features.testDataGenerator ? (name: string) => openTableTool(name, setTestDataTable) : undefined + } /> @@ -367,7 +390,15 @@ export function StudioWorkspace({ } > - setShowDiagram(false)} /> + setShowDiagram(false)} + className="absolute inset-0 z-20" + > + setShowDiagram(false)} /> + )} @@ -426,6 +457,7 @@ export function StudioWorkspace({ Promise; + onSchemaListFetch?: (connectionId: string) => Promise; + onTableSchemaFetch?: (connectionId: string, tableName: string) => Promise; } -export function useConnectionAdapter({ connections: externalConnections, onSchemaFetch }: UseConnectionAdapterParams) { +export function useConnectionAdapter({ + connections: externalConnections, + onSchemaFetch, + onSchemaListFetch, + onTableSchemaFetch, +}: UseConnectionAdapterParams) { const connections: DatabaseConnection[] = useMemo( () => externalConnections.map((c) => ({ @@ -58,19 +66,39 @@ export function useConnectionAdapter({ connections: externalConnections, onSchem setActiveConnectionId(conn?.id ?? null); }, []); + const loadDetails = useCallback( + async (tableName?: string): Promise => { + const id = activeConnection!.id; + if (tableName !== undefined && onTableSchemaFetch) { + const table = await onTableSchemaFetch(id, tableName); + return table ? [table] : []; + } + const full = await onSchemaFetch(id); + return tableName === undefined ? full : full.filter((table) => table.name === tableName); + }, + [activeConnection, onSchemaFetch, onTableSchemaFetch], + ); + const { ensureSchema, startSchemaLoad, isLoadingFullSchema } = useSchemaDetails( + activeConnection?.id, + schema, + setSchema, + loadDetails, + ); + const fetchSchema = useCallback( async (conn: DatabaseConnection) => { + const isCurrent = startSchemaLoad(); setIsLoadingSchema(true); try { - const result = await onSchemaFetch(conn.id); - setSchema(result); + const result = await (onSchemaListFetch ?? onSchemaFetch)(conn.id); + if (isCurrent()) setSchema(result); } catch { - setSchema([]); + if (isCurrent()) setSchema([]); } finally { - setIsLoadingSchema(false); + if (isCurrent()) setIsLoadingSchema(false); } }, - [onSchemaFetch], + [onSchemaFetch, onSchemaListFetch, startSchemaLoad], ); const schemaContext = useMemo(() => JSON.stringify(schema), [schema]); @@ -99,7 +127,8 @@ export function useConnectionAdapter({ connections: externalConnections, onSchem setActiveConnection, schema, setSchema, - isLoadingSchema, + isLoadingSchema: isLoadingSchema || isLoadingFullSchema, + ensureSchema, connectionPulse: null as "healthy" | "degraded" | "error" | null, fetchSchema, schemaContext, diff --git a/src/workspace/types.ts b/src/workspace/types.ts index 2798d28ac..de38c8297 100644 --- a/src/workspace/types.ts +++ b/src/workspace/types.ts @@ -169,7 +169,12 @@ export interface StudioWorkspaceProps { unlimited?: boolean; }, ) => Promise; + /** Complete schema, also used for explicit ERD, documentation and schema-diff requests. */ onSchemaFetch: (connectionId: string) => Promise; + /** Optional fast inventory. Entries with detailsLoaded: false are expanded on demand. */ + onSchemaListFetch?: (connectionId: string) => Promise; + /** Optional single-table reader; falls back to onSchemaFetch when omitted. */ + onTableSchemaFetch?: (connectionId: string, tableName: string) => Promise; onTestConnection?: (config: { type: DatabaseType; diff --git a/tests/api/db/schema.test.ts b/tests/api/db/schema.test.ts index ba1d60506..d5af809f2 100644 --- a/tests/api/db/schema.test.ts +++ b/tests/api/db/schema.test.ts @@ -98,6 +98,7 @@ const validConnection = { // ─── Tests ────────────────────────────────────────────────────────────────── describe("POST /api/db/schema", () => { beforeEach(() => { + delete mockProvider.getTableSchema; clearRateLimitState(); mockGetOrCreateProvider.mockClear(); (mockProvider.getSchema as ReturnType).mockClear(); @@ -107,6 +108,57 @@ describe("POST /api/db/schema", () => { ); }); + test("lazy schema returns only the decoded table using the provider fast path", async () => { + const name = 'Odd & quoted" table'; + const table = { ...mockSchema[0], name }; + mockProvider.getTableSchema = mock(async () => table); + const res = await POST( + createMockRequest(`/api/db/schema?table=${encodeURIComponent(name)}`, { + method: "POST", + body: validConnection, + }) as never, + ); + expect(res.status).toBe(200); + expect(await res.json()).toEqual([table]); + expect(mockProvider.getTableSchema).toHaveBeenCalledWith(name); + expect(mockProvider.getSchema).not.toHaveBeenCalled(); + }); + + test("lazy schema falls back to full schema for providers without a table reader", async () => { + const res = await POST( + createMockRequest("/api/db/schema?table=users", { method: "POST", body: validConnection }) as never, + ); + expect(res.status).toBe(200); + expect(await res.json()).toEqual([mockSchema[0]]); + expect(mockProvider.getSchema).toHaveBeenCalledTimes(1); + }); + + test.each([true, false])("lazy schema returns 404 for a missing table (fast path: %s)", async (fast) => { + if (fast) mockProvider.getTableSchema = mock(async () => null); + const res = await POST( + createMockRequest("/api/db/schema?table=gone", { method: "POST", body: validConnection }) as never, + ); + expect(res.status).toBe(404); + }); + + test("lazy schema rejects an empty table name", async () => { + const res = await POST( + createMockRequest("/api/db/schema?table=", { method: "POST", body: validConnection }) as never, + ); + expect(res.status).toBe(400); + expect(mockProvider.getSchema).not.toHaveBeenCalled(); + }); + + test("lazy schema requires authentication before reading table details", async () => { + mockGetSession.mockResolvedValueOnce(null); + mockProvider.getTableSchema = mock(async () => mockSchema[0]); + const res = await POST( + createMockRequest("/api/db/schema?table=users", { method: "POST", body: validConnection }) as never, + ); + expect(res.status).toBe(401); + expect(mockGetOrCreateProvider).not.toHaveBeenCalled(); + }); + test("returns 401 when no session exists", async () => { mockGetSession.mockResolvedValueOnce(null); diff --git a/tests/components/CommandPalette.test.tsx b/tests/components/CommandPalette.test.tsx index df185d07e..aa2676ef3 100644 --- a/tests/components/CommandPalette.test.tsx +++ b/tests/components/CommandPalette.test.tsx @@ -13,8 +13,15 @@ mock.module("cmdk", () => { ); Command.displayName = "Command"; - const CommandInput = React.forwardRef((props: Record, ref: React.Ref) => - React.createElement("input", { ...props, ref, "data-testid": "command-input" }), + const CommandInput = React.forwardRef( + ({ onValueChange, ...props }: Record, ref: React.Ref) => + React.createElement("input", { + ...props, + ref, + onChange: (e: React.ChangeEvent) => + (onValueChange as ((value: string) => void) | undefined)?.(e.target.value), + "data-testid": "command-input", + }), ); CommandInput.displayName = "CommandInput"; Command.Input = CommandInput; @@ -120,6 +127,21 @@ function createDefaultProps(overrides: Partial } describe("CommandPalette", () => { + test("large catalogs keep table commands bounded and search all names", () => { + const schema = Array.from({ length: 43500 }, (_, i) => ({ + ...mockSchema[0], + name: `PS_${String(i).padStart(5, "0")}`, + detailsLoaded: false, + columns: [], + })); + const view = render(); + fireEvent.keyDown(document, { key: "k", ctrlKey: true }); + expect(view.getAllByText("Details on demand", { exact: false })).toHaveLength(100); + fireEvent.change(view.getByTestId("command-input"), { target: { value: "PS_43499" } }); + expect(view.getByText("PS_43499")).toBeDefined(); + expect(view.queryByText("PS_00000")).toBeNull(); + }); + afterEach(() => { cleanup(); }); diff --git a/tests/components/Studio.test.tsx b/tests/components/Studio.test.tsx index 581c6f4be..7265a3b46 100644 --- a/tests/components/Studio.test.tsx +++ b/tests/components/Studio.test.tsx @@ -1429,6 +1429,19 @@ describe("Studio", () => { expect(mockSetSchema).toHaveBeenCalledWith([]); }); + test.each([true, false])("lazy standalone tools wait for details and stop on failure: %s", async (success) => { + const ensureSchema = mock(async () => (success ? [{ name: "users", columns: [], indexes: [] }] : null)); + connMgrOverride = { schema: [{ name: "users", columns: [], indexes: [], detailsLoaded: false }], ensureSchema }; + const view = render(); + await act(async () => { + (capturedSidebarProps.onGenerateCode as (name: string) => void)("users"); + }); + expect(ensureSchema).toHaveBeenCalledWith("users"); + expect(view.queryByTestId("codegenerator") !== null).toBe(success); + expect(capturedSidebarProps.onLoadTable).toBe(ensureSchema); + expect(capturedBottomPanelProps.onLoadSchema).toBe(ensureSchema); + }); + // --- Sidebar profiler/codegen/testdata callbacks --- test("Sidebar onProfileTable opens profiler", () => { const { queryByTestId } = render(); diff --git a/tests/components/StudioWorkspace.test.tsx b/tests/components/StudioWorkspace.test.tsx index f97f61c69..cfea670be 100644 --- a/tests/components/StudioWorkspace.test.tsx +++ b/tests/components/StudioWorkspace.test.tsx @@ -414,6 +414,19 @@ describe("StudioWorkspace", () => { // Rendering // ========================================================================= + test.each([true, false])("lazy embedded tools wait for details and stop on failure: %s", async (success) => { + const ensureSchema = mock(async () => (success ? [usersTable] : null)); + connAdapterOverride = { schema: [{ ...usersTable, detailsLoaded: false }], ensureSchema }; + renderWorkspace(); + await act(async () => { + (capturedSidebarProps.onGenerateCode as (name: string) => void)("users"); + }); + expect(ensureSchema).toHaveBeenCalledWith("users"); + expect(capturedCodeGeneratorProps.isOpen).toBe(success); + expect(capturedSidebarProps.onLoadTable).toBe(ensureSchema); + expect(capturedBottomPanelProps.onLoadSchema).toBe(ensureSchema); + }); + test("renders shell with sidebar, tab bar, toolbar, editor and bottom panel", () => { const { getByTestId, queryByTestId } = renderWorkspace(); expect(getByTestId("sidebar").textContent).toBe("Sidebar"); diff --git a/tests/components/schema-explorer/SchemaExplorer.test.tsx b/tests/components/schema-explorer/SchemaExplorer.test.tsx index ad0ef33e6..9e37dbf70 100644 --- a/tests/components/schema-explorer/SchemaExplorer.test.tsx +++ b/tests/components/schema-explorer/SchemaExplorer.test.tsx @@ -30,10 +30,11 @@ mock.module("@/components/schema-explorer/TableItem", () => ({ })); import { describe, test, expect, beforeEach, afterEach } from "bun:test"; -import { render, within, cleanup } from "@testing-library/react"; +import { render, within, cleanup, fireEvent, waitFor } from "@testing-library/react"; import userEvent from "@testing-library/user-event"; import React from "react"; +import { SchemaLoadGate } from "@/components/schema-explorer/SchemaLoadGate"; import { SchemaExplorer } from "@/components/schema-explorer/SchemaExplorer"; import type { ProviderMetadata } from "@/hooks/use-provider-metadata"; import type { ProviderLabels } from "@/lib/db/types"; @@ -94,6 +95,73 @@ function createDefaultProps(overrides: Partial } describe("SchemaExplorer", () => { + test("lazy full-schema tools wait for explicit loading and can retry", async () => { + const pending = mockSchema.map((table) => ({ ...table, detailsLoaded: false })); + let resolve!: (value: typeof mockSchema | null) => void; + const load = mock( + () => + new Promise((done) => { + resolve = done; + }), + ); + const close = mock(() => {}); + const view = render( + +
Complete diagram
+ , + ); + expect(view.queryByText("Complete diagram")).toBeNull(); + expect(load).not.toHaveBeenCalled(); + fireEvent.click(view.getByRole("button", { name: "Load full schema" })); + expect((view.getByRole("button", { name: "Loading full schema..." }) as HTMLButtonElement).disabled).toBe(true); + resolve(null); + await waitFor(() => expect(view.queryByRole("button", { name: "Load full schema" })).not.toBeNull()); + fireEvent.click(view.getByRole("button", { name: "Close" })); + expect(close).toHaveBeenCalled(); + view.rerender( + +
Complete diagram
+ , + ); + expect(view.getByText("Complete diagram")).toBeDefined(); + }); + + test("large schema pagination clamps after a refresh and resets when a search is cleared", async () => { + const user = userEvent.setup(); + const schema = Array.from({ length: 201 }, (_, i) => ({ ...mockSchema[0], name: `table_${i}` })); + const view = render(); + await user.click(view.getByRole("button", { name: "Next tables" })); + await user.click(view.getByRole("button", { name: "Previous tables" })); + expect(view.getByTestId("table-table_0")).toBeDefined(); + await user.click(view.getByRole("button", { name: "Next tables" })); + view.rerender(); + expect(view.getAllByTestId(/^table-/)).toHaveLength(1); + view.rerender(); + expect(view.getByTestId("table-table_0")).toBeDefined(); + await user.type(view.getByRole("textbox"), "table_0"); + await user.clear(view.getByRole("textbox")); + expect(view.getByTestId("table-table_0")).toBeDefined(); + }); + + test("large schemas render a bounded page and search all table names", async () => { + const user = userEvent.setup(); + const schema = Array.from({ length: 43500 }, (_, i) => ({ + name: `PS_${String(i).padStart(5, "0")}`, + columns: [], + indexes: [], + detailsLoaded: false, + })); + const { container } = render(); + const view = within(container); + expect(view.queryAllByTestId(/^table-/)).toHaveLength(100); + await user.click(view.getByRole("button", { name: "Next tables" })); + expect(view.queryByTestId("table-PS_00100")).not.toBeNull(); + expect(view.queryByTestId("table-PS_00000")).toBeNull(); + await user.type(view.getByRole("textbox"), "PS_43499"); + expect(view.queryAllByTestId(/^table-/)).toHaveLength(1); + expect(view.queryByTestId("table-PS_43499")).not.toBeNull(); + }); + afterEach(() => { cleanup(); }); diff --git a/tests/components/schema-explorer/TableItem.test.tsx b/tests/components/schema-explorer/TableItem.test.tsx index 7b372e4c5..6a2267abf 100644 --- a/tests/components/schema-explorer/TableItem.test.tsx +++ b/tests/components/schema-explorer/TableItem.test.tsx @@ -184,6 +184,46 @@ describe("TableItem", () => { else Object.defineProperty(globalThis.document, "execCommand", originalExecCommand); }); + test("lazy table expansion loads once and displays details after the parent merges them", async () => { + const table = { ...largeTable, columns: [], indexes: [], detailsLoaded: false }; + let resolve!: (value: TableSchema[] | null) => void; + const onLoadTable = mock( + () => + new Promise((done) => { + resolve = done; + }), + ); + const props = { table, isExpanded: false, onToggle: mock(() => {}), onLoadTable, isAdmin: false }; + const view = render(); + fireEvent.click(view.getByRole("button", { name: "users" })); + view.rerender(); + expect(view.getByRole("status").textContent).toContain("Loading table details"); + expect(view.queryByTestId("column-list")).toBeNull(); + expect(onLoadTable).toHaveBeenCalledWith("users"); + resolve([largeTable]); + await waitFor(() => expect(view.queryByRole("status")).toBeNull()); + view.rerender(); + expect(view.getByTestId("column-list").textContent).toContain("2 cols"); + }); + + test("lazy table failures remain retryable and never display an empty column list", async () => { + const onLoadTable = mock(async () => null); + const view = render( + {}} + onLoadTable={onLoadTable} + isAdmin={false} + />, + ); + fireEvent.click(view.getByRole("button", { name: "Load table details" })); + await waitFor(() => expect(view.queryByRole("status")).toBeNull()); + expect(view.queryByTestId("column-list")).toBeNull(); + fireEvent.click(view.getByRole("button", { name: "Load table details" })); + await waitFor(() => expect(onLoadTable).toHaveBeenCalledTimes(2)); + }); + // ── Rendering ───────────────────────────────────────────────────────────── test("renders table name", () => { diff --git a/tests/hooks/use-connection-adapter.test.ts b/tests/hooks/use-connection-adapter.test.ts index 2192d1966..2aeebbaf6 100644 --- a/tests/hooks/use-connection-adapter.test.ts +++ b/tests/hooks/use-connection-adapter.test.ts @@ -41,6 +41,69 @@ const makeSchema = (): TableSchema[] => [ // useConnectionAdapter Tests // ============================================================================= describe("useConnectionAdapter", () => { + test("lazy embedded inventory reads one table and full schema only when requested", async () => { + const full = makeSchema(); + const list = full.map((table) => ({ ...table, columns: [], indexes: [], detailsLoaded: false })); + const onSchemaFetch = mock(async () => full); + const onSchemaListFetch = mock(async () => list); + const onTableSchemaFetch = mock( + async (_id: string, name: string) => full.find((table) => table.name === name) ?? null, + ); + const { result } = renderHook(() => + useConnectionAdapter({ + connections: [makeWorkspaceConnection()], + onSchemaFetch, + onSchemaListFetch, + onTableSchemaFetch, + }), + ); + await act(async () => { + await result.current.fetchSchema(result.current.activeConnection!); + }); + expect(result.current.schema).toEqual(list); + expect(onSchemaFetch).not.toHaveBeenCalled(); + await act(async () => { + await result.current.ensureSchema("users"); + }); + expect(onTableSchemaFetch).toHaveBeenCalledWith("ws-conn-1", "users"); + expect(result.current.schema).toEqual([full[0], list[1]]); + await act(async () => { + await result.current.ensureSchema(); + }); + expect(result.current.schema).toEqual(full); + expect(onSchemaFetch).toHaveBeenCalledTimes(1); + }); + + test("lazy embedded table loading falls back to the required full reader", async () => { + const full = makeSchema(); + const onSchemaFetch = mock(async () => full); + const { result } = renderHook(() => + useConnectionAdapter({ connections: [makeWorkspaceConnection()], onSchemaFetch }), + ); + act(() => result.current.setSchema(full.map((table) => ({ ...table, detailsLoaded: false })))); + await act(async () => { + await result.current.ensureSchema("users"); + }); + expect(result.current.schema[0]).toEqual(full[0]); + expect(result.current.schema[1].detailsLoaded).toBe(false); + }); + + test("lazy embedded missing tables leave their inventory entry retryable", async () => { + const full = makeSchema(); + const { result } = renderHook(() => + useConnectionAdapter({ + connections: [makeWorkspaceConnection()], + onSchemaFetch: async () => full, + onTableSchemaFetch: async () => null, + }), + ); + act(() => result.current.setSchema(full.map((table) => ({ ...table, detailsLoaded: false })))); + await act(async () => { + expect(await result.current.ensureSchema("users")).toBeNull(); + }); + expect(result.current.schema[0].detailsLoaded).toBe(false); + }); + // ── Initializes with first connection as active ───────────────────────── test("initializes with first connection as active", () => { diff --git a/tests/hooks/use-connection-manager.test.ts b/tests/hooks/use-connection-manager.test.ts index dad73dcb5..c873ec3b0 100644 --- a/tests/hooks/use-connection-manager.test.ts +++ b/tests/hooks/use-connection-manager.test.ts @@ -62,6 +62,69 @@ describe("useConnectionManager", () => { restoreGlobalFetch(); }); + test("lazy standalone inventory skips bulk relations and binds an encoded table request", async () => { + const name = 'Weird & "table'; + const full = { ...makeSchema()[0], name }; + const list = [ + { ...full, columns: [], indexes: [], detailsLoaded: false }, + { ...makeSchema()[1], detailsLoaded: false }, + ]; + const fetchMock = mockGlobalFetch({ + "/api/db/schema/list": { json: list }, + "/api/db/schema": async (req) => { + expect(new URL(req.url).searchParams.get("table")).toBe(name); + expect(await req.json()).toEqual({ connectionId: "seed:oracle" }); + return { json: [full] }; + }, + }); + const { result } = renderHook(() => useConnectionManager()); + const conn = makeConnection({ managed: true, seedId: "oracle" }); + act(() => result.current.setActiveConnection(conn)); + await act(async () => { + await result.current.fetchSchema(conn); + }); + expect(fetchMock.mock.calls.filter(([url]) => String(url).includes("schema/relations"))).toHaveLength(0); + await act(async () => { + await result.current.ensureSchema(name); + }); + expect(result.current.schema).toEqual([full, list[1]]); + }); + + test.each(["catalog denied", null])("lazy standalone failed details preserve the inventory: %s", async (message) => { + const list = makeSchema().map((table) => ({ ...table, detailsLoaded: false })); + mockGlobalFetch({ + "/api/db/schema": message ? { status: 403, json: { error: message } } : { status: 500, text: "invalid JSON" }, + }); + const { result } = renderHook(() => useConnectionManager()); + act(() => { + result.current.setActiveConnection(makeConnection()); + result.current.setSchema(list); + }); + await act(async () => { + expect(await result.current.ensureSchema("users")).toBeNull(); + }); + expect(result.current.schema).toEqual(list); + }); + + test("lazy standalone explicit full request has no table filter", async () => { + const full = makeSchema(); + mockGlobalFetch({ + "/api/db/schema": (req) => { + expect(new URL(req.url).search).toBe(""); + return { json: full }; + }, + }); + const { result } = renderHook(() => useConnectionManager()); + act(() => { + result.current.setActiveConnection(makeConnection()); + result.current.setSchema(full.map((table) => ({ ...table, detailsLoaded: false }))); + }); + await act(async () => { + await result.current.ensureSchema(); + }); + expect(result.current.schema).toEqual(full); + }); + // ── Initial State ───────────────────────────────────────────────────────── test("starts with empty connections and null activeConnection", () => { diff --git a/tests/hooks/use-schema-details.test.ts b/tests/hooks/use-schema-details.test.ts new file mode 100644 index 000000000..fff854149 --- /dev/null +++ b/tests/hooks/use-schema-details.test.ts @@ -0,0 +1,182 @@ +import "../setup-dom"; +import { mockToastError } from "../helpers/mock-sonner"; +import { describe, test, expect, mock, beforeEach } from "bun:test"; +import { useState } from "react"; +import { renderHook, act } from "@testing-library/react"; +import { useSchemaDetails } from "@/hooks/use-schema-details"; +import type { TableSchema } from "@/lib/types"; + +const inventory: TableSchema[] = ["users", "orders"].map((name) => ({ + name, + columns: [], + indexes: [], + detailsLoaded: false, +})); +const complete: TableSchema[] = inventory.map(({ detailsLoaded: _, ...table }) => ({ + ...table, + columns: [{ name: "id", type: "INTEGER", nullable: false, isPrimary: true }], +})); +function setup(load: (name?: string) => Promise, initial = inventory, id: string | undefined = "one") { + return renderHook( + ({ connectionId }: { connectionId: string | undefined }) => { + const [schema, setSchema] = useState(initial); + return { schema, ...useSchemaDetails(connectionId, schema, setSchema, load) }; + }, + { initialProps: { connectionId: id } as { connectionId: string | undefined } }, + ); +} +function deferred() { + let resolve!: (value: T) => void; + const promise = new Promise((done) => { + resolve = done; + }); + return { promise, resolve }; +} + +describe("lazy schema details", () => { + beforeEach(() => mockToastError.mockClear()); + + test("loads one table, preserves pending peers, then uses cached details", async () => { + const load = mock(async () => [complete[0]]); + const { result } = setup(load); + await act(async () => { + expect(await result.current.ensureSchema("users")).toEqual([complete[0]]); + }); + expect(result.current.schema).toEqual([complete[0], inventory[1]]); + await act(async () => { + expect(await result.current.ensureSchema("users")).toEqual([complete[0]]); + }); + expect(load).toHaveBeenCalledTimes(1); + expect(load).toHaveBeenCalledWith("users"); + }); + + test("coalesces concurrent table requests", async () => { + const response = deferred(); + const load = mock(() => response.promise); + const { result } = setup(load); + await act(async () => { + const first = result.current.ensureSchema("users"); + const second = result.current.ensureSchema("users"); + response.resolve([complete[0]]); + expect(await first).toEqual(await second); + }); + expect(load).toHaveBeenCalledTimes(1); + }); + + test("full schema is explicit and replaces the inventory", async () => { + const response = deferred(); + const load = mock(() => response.promise); + const { result } = setup(load); + let request!: Promise; + act(() => { + request = result.current.ensureSchema(); + }); + expect(result.current.isLoadingFullSchema).toBe(true); + await act(async () => { + response.resolve(complete); + await request; + }); + expect(result.current.schema).toEqual(complete); + expect(result.current.isLoadingFullSchema).toBe(false); + expect(await result.current.ensureSchema()).toEqual(complete); + expect(load).toHaveBeenCalledTimes(1); + }); + + test("no connection and unknown tables never initiate reads", async () => { + const load = mock(async () => complete); + const { result, rerender } = setup(load); + expect(await result.current.ensureSchema("missing")).toBeNull(); + rerender({ connectionId: undefined }); + expect(await result.current.ensureSchema()).toBeNull(); + expect(load).not.toHaveBeenCalled(); + }); + + test.each([new Error("timed out"), "unavailable"])( + "failed reads preserve the inventory and can be retried: %s", + async (error) => { + const load = mock(async (): Promise => { + throw error; + }); + const { result } = setup(load); + await act(async () => { + expect(await result.current.ensureSchema("users")).toBeNull(); + }); + expect(result.current.schema).toEqual(inventory); + expect(mockToastError).toHaveBeenCalled(); + load.mockResolvedValue([complete[0]]); + await act(async () => { + expect(await result.current.ensureSchema("users")).toEqual([complete[0]]); + }); + }, + ); + + test.each([{ response: [] }, { response: [complete[1]] }, { response: inventory }])( + "rejects missing, wrong or still-pending table responses: %j", + async ({ response }) => { + const { result } = setup(async () => [...response]); + await act(async () => { + expect(await result.current.ensureSchema("users")).toBeNull(); + }); + expect(result.current.schema).toEqual(inventory); + }, + ); + + test("connection changes discard old full reads and clear the loading indicator", async () => { + const response = deferred(); + const { result, rerender } = setup(() => response.promise); + let request!: Promise; + act(() => { + request = result.current.ensureSchema(); + }); + rerender({ connectionId: "two" }); + expect(result.current.isLoadingFullSchema).toBe(false); + await act(async () => { + response.resolve(complete); + expect(await request).toBeNull(); + }); + expect(result.current.schema).toEqual(inventory); + }); + + test("refresh supersedes an in-flight table read even when names match", async () => { + const old = deferred(); + const fresh = deferred(); + const load = mock(() => old.promise) + .mockImplementationOnce(() => old.promise) + .mockImplementationOnce(() => fresh.promise); + const { result } = setup(load); + let first!: Promise; + await act(async () => { + first = result.current.ensureSchema("users"); + }); + result.current.startSchemaLoad(); + let second!: Promise; + await act(async () => { + second = result.current.ensureSchema("users"); + }); + await act(async () => { + old.resolve([complete[0]]); + expect(await first).toBeNull(); + }); + const renamedColumn = { ...complete[0], columns: [{ ...complete[0].columns[0], name: "new_id" }] }; + await act(async () => { + fresh.resolve([renamedColumn]); + expect(await second).toEqual([renamedColumn]); + }); + expect(result.current.schema[0]).toEqual(renamedColumn); + }); + + test("errors from a previous connection do not notify the current one", async () => { + const response = deferred(); + const { result, rerender } = setup(async () => { + await response.promise; + throw new Error("old connection"); + }); + const request = result.current.ensureSchema("users"); + rerender({ connectionId: "two" }); + await act(async () => { + response.resolve([]); + expect(await request).toBeNull(); + }); + expect(mockToastError).not.toHaveBeenCalled(); + }); +}); diff --git a/tests/hooks/use-tab-manager.test.ts b/tests/hooks/use-tab-manager.test.ts index e88e3162b..45e3cccbd 100644 --- a/tests/hooks/use-tab-manager.test.ts +++ b/tests/hooks/use-tab-manager.test.ts @@ -75,6 +75,45 @@ describe("useTabManager", () => { localStorage.clear(); }); + test.each([true, false])( + "lazy table actions wait for real columns before creating a query (execute: %s)", + async (execute) => { + const ensureSchema = mock(async () => testSchema); + const executeQuery = mock(() => {}); + const { result } = renderHook(() => + useTabManager({ + activeConnection: makeConnection(), + metadata: defaultMetadata, + schema: [{ ...testSchema[0], columns: [], detailsLoaded: false }], + ensureSchema, + }), + ); + await act(async () => { + if (execute) await result.current.handleTableClick("users", executeQuery); + else await result.current.handleGenerateSelect("users"); + }); + expect(ensureSchema).toHaveBeenCalledWith("users"); + expect(result.current.tabs).toHaveLength(2); + if (!execute) expect(result.current.currentTab.query).toContain("id"); + }, + ); + + test.each([true, false])("lazy table actions stop when details could not be read (execute: %s)", async (execute) => { + const { result } = renderHook(() => + useTabManager({ + activeConnection: makeConnection(), + metadata: defaultMetadata, + schema: [{ ...testSchema[0], detailsLoaded: false }], + ensureSchema: async () => null, + }), + ); + await act(async () => { + if (execute) await result.current.handleTableClick("users", () => {}); + else await result.current.handleGenerateSelect("users"); + }); + expect(result.current.tabs).toHaveLength(1); + }); + test("starts with one default tab", () => { const { result } = renderHook(() => useTabManager({ diff --git a/tests/integration/db/oracle-provider.test.ts b/tests/integration/db/oracle-provider.test.ts index 5c4ead663..bdf6c731f 100644 --- a/tests/integration/db/oracle-provider.test.ts +++ b/tests/integration/db/oracle-provider.test.ts @@ -1208,6 +1208,85 @@ describe("OracleProvider", () => { // ========================================================================= describe("getSchema()", () => { + test("lazy Oracle inventory reads only table names for a large owner", async () => { + await provider.connect(); + const execute = mock(async (_sql: string, _params?: unknown[]) => ({ + rows: Array.from({ length: 43500 }, (_, i) => ({ TABLE_NAME: `PS_${i}`, NUM_ROWS: 10 })), + })); + mockExecuteFn = execute; + const schema = await provider.getSchemaList(); + expect(execute).toHaveBeenCalledTimes(1); + expect(execute.mock.calls[0][0]).not.toMatch(/ALL_TAB_COLUMNS|ALL_CONSTRAINTS|ALL_INDEXES/); + expect(schema).toHaveLength(43500); + expect(schema[0]).toMatchObject({ name: "PS_0", columns: [], indexes: [], detailsLoaded: false }); + }); + + test("lazy Oracle details bind one table in every catalog query", async () => { + await provider.connect(); + const execute = mock(async (sql: string, params?: unknown[]) => { + expect(params).toEqual(["TEST_USER", "USERS"]); + expect(sql).toMatch(/TABLE_NAME\s*=\s*:2/); + const result = await defaultExecute(sql); + return { + ...result, + rows: (result.rows as Record[] | undefined)?.filter((row) => row.TABLE_NAME === "USERS"), + }; + }); + mockExecuteFn = execute; + const table = await provider.getTableSchema("USERS"); + expect(execute).toHaveBeenCalledTimes(5); + expect(table?.name).toBe("USERS"); + expect(table?.columns.find((c) => c.name === "ID")?.isPrimary).toBe(true); + }); + + test("lazy Oracle missing table stops before loading other catalog rows", async () => { + await provider.connect(); + const execute = mock(async () => ({ rows: [] })); + mockExecuteFn = execute; + expect(await provider.getTableSchema("DROPPED")).toBeNull(); + expect(execute).toHaveBeenCalledTimes(1); + }); + + test("lazy Oracle table names remain bind values even when they contain SQL syntax", async () => { + await provider.connect(); + const name = "x' OR 1=1 --"; + const execute = mock(async (sql: string, params?: unknown[]) => { + expect(sql).not.toContain(name); + expect(params).toEqual(["TEST_USER", name]); + return { rows: [{ TABLE_NAME: name }] }; + }); + mockExecuteFn = execute; + expect((await provider.getTableSchema(name))?.name).toBe(name); + expect(execute).toHaveBeenCalledTimes(5); + }); + + test("lazy Oracle inventory handles absent statistics and empty driver rows", async () => { + await provider.connect(); + mockExecuteFn = async () => ({ rows: [{}] }); + expect(await provider.getSchemaList()).toEqual([ + { name: "", rowCount: 0, columns: [], indexes: [], foreignKeys: [], detailsLoaded: false }, + ]); + mockExecuteFn = async () => ({}); + expect(await provider.getSchemaList()).toEqual([]); + }); + + test.each(["list", "table"])("lazy Oracle %s releases the connection after a catalog error", async (kind) => { + await provider.connect(); + const close = mock(async () => {}); + mockConnCloseFn = close; + mockExecuteFn = async () => { + throw new Error("catalog unavailable"); + }; + await expect(kind === "list" ? provider.getSchemaList() : provider.getTableSchema("USERS")).rejects.toThrow( + "catalog unavailable", + ); + expect(close).toHaveBeenCalledTimes(1); + }); + + test("lazy Oracle inventory requires a live connection", async () => { + await expect(provider.getSchemaList()).rejects.toThrow(); + }); + test("returns tables with columns, indexes, PKs, and FKs", async () => { await provider.connect(); const schema = await provider.getSchema();