Skip to content

Commit c6ef5f3

Browse files
ericallamericallam
authored
perf(webapp): paginate the environment variables settings page (#4597)
## What The environment variables settings page loaded **every** variable in the project in one shot, with a nested `values` read plus a `valueReference` (SecretReference) sub-load that was selected but never read. For a project with many variables this pulled `variables × environments` value rows (~18k for large projects) on every page load, plus a matching ~18k-row `SecretReference IN` query. This paginates the presenter by variable key and removes the dead include. - Remove the never-read `valueReference: { select: { key } }` include → the `SecretReference` query is gone entirely. - Paginate the parent variable query: `count` + `orderBy key` + `skip/take`, page size 50 → the value read is bounded to `pageSize × environments` per page. - Scope the count and the page to variables that have a value in a displayed environment (`values: { some: { environmentId: { in } } }`), so `totalCount`/`totalPages` and the `skip/take` window match what actually renders (no phantom empty pages from variables that live only in archived branches or another member's dev env). - Display order comes from the DB `orderBy: { key: "asc" }` — the presenter no longer re-sorts each page with `localeCompare`, which under pagination could disagree with the DB collation at page boundaries. - The secret-value lookup (`SecretStore` keys) and the updater lookup (`user` by id) are now scoped to the current page instead of the whole project. - Search moves server-side (variable key, case-insensitive) and drives both the count and the page; the UI gains standard pagination controls. ## Why The two correlated ~18k-row control-plane queries flagged in the ticket come from this settings-page presenter, not from any hot path. Both are index-covered (`rows_read == rows_returned`); the issue is the sheer volume fetched in one burst. Bounding it per page removes the burst. ## Evidence Measured on an isolated stack with a seeded project of 1000 variables × 3 environments (3000 value rows), using Prisma's emitted-SQL log: | | SecretReference query | value rows fetched | | --- | --- | --- | | before | 1 | 3000 | | after | **0** | **150** (page 1) + one `count` | `EXPLAIN` on Prisma's verbatim statements (index confirmed via `enable_seqscan=off`; the local table is too small for the planner to choose them by default): - `count` (`WHERE projectId AND EXISTS(values in displayed envs)`) → Hash Join: Index Scan on `EnvironmentVariable_pkey` + Bitmap Index Scan on `EnvironmentVariableValue_environmentId_idx` - paginated parent (`WHERE projectId AND EXISTS(...) ORDER BY key LIMIT/OFFSET`) → Nested Loop Semi Join: Index Scan on `EnvironmentVariable_projectId_key_key` (**no Sort node**) driving an Index-Only Scan on `EnvironmentVariableValue_variableId_environmentId_key` - nested values (`variableId = ANY … AND environmentId = ANY …`) → index scan on `EnvironmentVariableValue_environmentId_idx` - `SecretStore` keys (`key = ANY …`) → index scan on `SecretStore_key_idx` No new index required. Verified in the browser on the seeded project: 20 pages, page navigation, server-side search (matches across all pages), last page renders, no app console errors. `typecheck`, `oxlint`, `oxfmt` all clean. ## Behavior change The previous client-side search matched variable **name and value** (and environment type / branch name). Values are encrypted at rest and resolved separately, so they cannot be searched server-side under pagination. Search is now **variable-name only**, server-side, case-insensitive. Projects with fewer than one page of variables see no pagination bar and no visible change. ## Rollout / rollback Pure read-path change on a dashboard loader, no schema or data migration. Rollback is a straight revert. ## Screenshots <img width="2400" height="1794" alt="01-page1" src="https://github.com/user-attachments/assets/d4a7effd-d167-4dd6-92f4-6e9174818acd" /> <img width="2400" height="1794" alt="02-search-single" src="https://github.com/user-attachments/assets/cd113ca9-ff87-431f-b2f6-7f7d36f2b32a" />
1 parent 8d0f693 commit c6ef5f3

3 files changed

Lines changed: 119 additions & 74 deletions

File tree

‎.server-changes/paginate-environment-variables.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
area: webapp
3+
type: improvement
4+
---
5+
6+
The environment variables page now loads a page at a time, keeping it fast for projects with a large number of variables. Search matches variable names across every page.

‎apps/webapp/app/presenters/v3/EnvironmentVariablesPresenter.server.ts‎

Lines changed: 86 additions & 59 deletions
Original file line numberDiff line numberDiff line change
@@ -8,10 +8,12 @@ import type { SyncEnvVarsMapping, EnvSlug } from "~/v3/vercel/vercelProjectInteg
88
import { VercelIntegrationService } from "~/services/vercelIntegration.server";
99
import { loadEnvironmentVariablesEnvironments } from "./environmentVariablesEnvironments.server";
1010

11-
import { boundedIn } from "@trigger.dev/database";
11+
import { boundedIn, type Prisma } from "@trigger.dev/database";
1212
type Result = Awaited<ReturnType<EnvironmentVariablesPresenter["call"]>>;
1313
export type EnvironmentVariableWithSetValues = Result["environmentVariables"][number];
1414

15+
export const DEFAULT_ENV_VARS_PAGE_SIZE = 50;
16+
1517
export class EnvironmentVariablesPresenter {
1618
#prismaClient: PrismaClient;
1719
#replicaClient: PrismaReplicaClient;
@@ -21,7 +23,19 @@ export class EnvironmentVariablesPresenter {
2123
this.#replicaClient = replicaClient;
2224
}
2325

24-
public async call({ userId, projectSlug }: { userId: User["id"]; projectSlug: Project["slug"] }) {
26+
public async call({
27+
userId,
28+
projectSlug,
29+
page = 1,
30+
pageSize = DEFAULT_ENV_VARS_PAGE_SIZE,
31+
search,
32+
}: {
33+
userId: User["id"];
34+
projectSlug: Project["slug"];
35+
page?: number;
36+
pageSize?: number;
37+
search?: string;
38+
}) {
2539
const project = await this.#replicaClient.project.findFirst({
2640
select: {
2741
id: true,
@@ -53,6 +67,18 @@ export class EnvironmentVariablesPresenter {
5367
// values in archived branch environments, which would otherwise all be loaded here.
5468
const environmentIds = sortedEnvironments.map((env) => env.id);
5569

70+
const variableWhere: Prisma.EnvironmentVariableWhereInput = {
71+
projectId: project.id,
72+
values: { some: { environmentId: { in: boundedIn(environmentIds) } } },
73+
...(search ? { key: { contains: search, mode: "insensitive" } } : {}),
74+
};
75+
76+
const totalCount = await this.#replicaClient.environmentVariable.count({
77+
where: variableWhere,
78+
});
79+
const totalPages = Math.max(1, Math.ceil(totalCount / pageSize));
80+
const currentPage = Math.min(Math.max(1, page), totalPages);
81+
5682
const environmentVariables = await this.#replicaClient.environmentVariable.findMany({
5783
select: {
5884
id: true,
@@ -64,11 +90,6 @@ export class EnvironmentVariablesPresenter {
6490
version: true,
6591
lastUpdatedBy: true,
6692
updatedAt: true,
67-
valueReference: {
68-
select: {
69-
key: true,
70-
},
71-
},
7293
isSecret: true,
7394
},
7495
where: {
@@ -78,9 +99,12 @@ export class EnvironmentVariablesPresenter {
7899
},
79100
},
80101
},
81-
where: {
82-
projectId: project.id,
102+
where: variableWhere,
103+
orderBy: {
104+
key: "asc",
83105
},
106+
skip: (currentPage - 1) * pageSize,
107+
take: pageSize,
84108
});
85109

86110
const userIds = new Set(
@@ -152,58 +176,61 @@ export class EnvironmentVariablesPresenter {
152176
}
153177

154178
return {
155-
environmentVariables: environmentVariables
156-
.flatMap((environmentVariable) => {
157-
return sortedEnvironments.flatMap((env) => {
158-
const valueRecord = environmentVariable.values.find((v) => v.environmentId === env.id);
159-
const isSecret = valueRecord?.isSecret ?? false;
160-
161-
if (!valueRecord) {
162-
return [];
163-
}
164-
165-
const val = isSecret
166-
? undefined
167-
: variableValuesByEnvAndKey.get(`${env.id}:${environmentVariable.key}`);
168-
169-
if (!isSecret && val === undefined) {
170-
return [];
171-
}
172-
173-
const lastUpdatedBy = valueRecord.lastUpdatedBy as EnvironmentVariableUpdater | null;
174-
175-
const updatedByUser =
176-
lastUpdatedBy?.type === "user"
177-
? (() => {
178-
const user = usersRecord[lastUpdatedBy.userId];
179-
return user
180-
? {
181-
id: user.id,
182-
name: user.displayName || user.name || "Unknown",
183-
avatarUrl: user.avatarUrl,
184-
}
185-
: null;
186-
})()
187-
: null;
188-
189-
return [
190-
{
191-
id: environmentVariable.id,
192-
key: environmentVariable.key,
193-
environment: { type: env.type, id: env.id, branchName: env.branchName },
194-
value: isSecret ? "" : val!,
195-
isSecret,
196-
version: valueRecord.version,
197-
lastUpdatedBy,
198-
updatedByUser,
199-
updatedAt: valueRecord.updatedAt,
200-
},
201-
];
202-
});
203-
})
204-
.sort((a, b) => a.key.localeCompare(b.key)),
179+
environmentVariables: environmentVariables.flatMap((environmentVariable) => {
180+
return sortedEnvironments.flatMap((env) => {
181+
const valueRecord = environmentVariable.values.find((v) => v.environmentId === env.id);
182+
const isSecret = valueRecord?.isSecret ?? false;
183+
184+
if (!valueRecord) {
185+
return [];
186+
}
187+
188+
const val = isSecret
189+
? undefined
190+
: variableValuesByEnvAndKey.get(`${env.id}:${environmentVariable.key}`);
191+
192+
if (!isSecret && val === undefined) {
193+
return [];
194+
}
195+
196+
const lastUpdatedBy = valueRecord.lastUpdatedBy as EnvironmentVariableUpdater | null;
197+
198+
const updatedByUser =
199+
lastUpdatedBy?.type === "user"
200+
? (() => {
201+
const user = usersRecord[lastUpdatedBy.userId];
202+
return user
203+
? {
204+
id: user.id,
205+
name: user.displayName || user.name || "Unknown",
206+
avatarUrl: user.avatarUrl,
207+
}
208+
: null;
209+
})()
210+
: null;
211+
212+
return [
213+
{
214+
id: environmentVariable.id,
215+
key: environmentVariable.key,
216+
environment: { type: env.type, id: env.id, branchName: env.branchName },
217+
value: isSecret ? "" : val!,
218+
isSecret,
219+
version: valueRecord.version,
220+
lastUpdatedBy,
221+
updatedByUser,
222+
updatedAt: valueRecord.updatedAt,
223+
},
224+
];
225+
});
226+
}),
205227
environments: sortedEnvironments,
206228
hasStaging,
229+
pagination: {
230+
currentPage,
231+
totalPages,
232+
totalCount,
233+
},
207234
// Vercel integration data
208235
vercelIntegration: vercelIntegration
209236
? {

‎apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.environment-variables/route.tsx‎

Lines changed: 27 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,7 @@ import { Input } from "~/components/primitives/Input";
4040
import { InputGroup } from "~/components/primitives/InputGroup";
4141
import { Label } from "~/components/primitives/Label";
4242
import { NavBar, PageAccessories, PageTitle } from "~/components/primitives/PageHeader";
43+
import { PaginationControls } from "~/components/primitives/Pagination";
4344
import { Paragraph } from "~/components/primitives/Paragraph";
4445
import { SearchInput } from "~/components/primitives/SearchInput";
4546
import { Switch } from "~/components/primitives/Switch";
@@ -55,10 +56,8 @@ import {
5556
import { SimpleTooltip } from "~/components/primitives/Tooltip";
5657
import { prisma } from "~/db.server";
5758
import { useEnvironment } from "~/hooks/useEnvironment";
58-
import { useFuzzyFilter } from "~/hooks/useFuzzyFilter";
5959
import { useOrganization } from "~/hooks/useOrganizations";
6060
import { useProject } from "~/hooks/useProject";
61-
import { useSearchParams } from "~/hooks/useSearchParam";
6261
import { redirectWithSuccessMessage } from "~/models/message.server";
6362
import { resolveOrgIdFromSlug } from "~/models/organization.server";
6463
import {
@@ -117,6 +116,8 @@ export type EnvironmentVariablesPageLoaderData = {
117116
accessibleEnvironmentIds: string[];
118117
// Environment ids whose env vars the current role can write (create/edit/delete).
119118
writableEnvironmentIds: string[];
119+
pagination: { currentPage: number; totalPages: number; totalCount: number };
120+
search?: string;
120121
};
121122

122123
export const environmentVariablesRouteId =
@@ -125,22 +126,28 @@ export const environmentVariablesRouteId =
125126
export const loader = dashboardLoader(
126127
{
127128
params: EnvironmentParamSchema,
129+
searchParams: z.object({
130+
page: z.coerce.number().int().min(1).catch(1),
131+
search: z.string().trim().min(1).optional().catch(undefined),
132+
}),
128133
context: async (params) => {
129134
const organizationId = await resolveOrgIdFromSlug(params.organizationSlug);
130135
return organizationId ? { organizationId } : {};
131136
},
132137
// No hard authorization: the page lists every environment. Values in
133138
// environments the role can't read are masked per-tier below.
134139
},
135-
async ({ params, user, ability }) => {
140+
async ({ params, searchParams, user, ability }) => {
136141
const { projectParam } = params;
137142

138143
try {
139144
const presenter = new EnvironmentVariablesPresenter();
140-
const { environmentVariables, environments, hasStaging, vercelIntegration } =
145+
const { environmentVariables, environments, hasStaging, vercelIntegration, pagination } =
141146
await presenter.call({
142147
userId: user.id,
143148
projectSlug: projectParam,
149+
page: searchParams.page,
150+
search: searchParams.search,
144151
});
145152

146153
const accessibleEnvironmentIds = environments
@@ -176,6 +183,8 @@ export const loader = dashboardLoader(
176183
vercelIntegration,
177184
accessibleEnvironmentIds,
178185
writableEnvironmentIds,
186+
pagination,
187+
search: searchParams.search,
179188
});
180189
} catch (error) {
181190
console.error(error);
@@ -392,17 +401,12 @@ function EnvironmentVariablesListPage({
392401
loaderData: EnvironmentVariablesPageLoaderData;
393402
}) {
394403
const [revealAll, setRevealAll] = useState(false);
395-
const { environmentVariables, vercelIntegration } = loaderData;
404+
const { environmentVariables, vercelIntegration, pagination, search } = loaderData;
405+
const hasSearch = Boolean(search);
396406
const organization = useOrganization();
397407
const project = useProject();
398408
const environment = useEnvironment();
399-
const { value } = useSearchParams();
400-
const urlSearch = value("search") ?? "";
401-
const { filteredItems } = useFuzzyFilter<EnvironmentVariableWithSetValues>({
402-
items: environmentVariables,
403-
keys: ["key", "value", "environment.type", "environment.branchName"],
404-
filterText: urlSearch,
405-
});
409+
const filteredItems = environmentVariables;
406410

407411
const tableScrollRef = useRef<HTMLDivElement>(null);
408412

@@ -477,9 +481,9 @@ function EnvironmentVariablesListPage({
477481
</NavBar>
478482
<PageBody scrollable={false}>
479483
<div className={cn("flex h-full min-h-0 flex-col")}>
480-
{environmentVariables.length > 0 && (
484+
{(environmentVariables.length > 0 || hasSearch) && (
481485
<div className="flex items-center justify-between gap-2 px-2 py-2">
482-
<SearchInput placeholder="Search variables…" autoFocus />
486+
<SearchInput placeholder="Search variables…" resetParams={["page"]} autoFocus />
483487
<div className="flex items-center justify-end gap-1.5">
484488
<Switch
485489
variant="secondary/small"
@@ -574,7 +578,7 @@ function EnvironmentVariablesListPage({
574578
<TableBody>
575579
<TableRow>
576580
<TableCell colSpan={vercelColumnCount}>
577-
{environmentVariables.length === 0 ? (
581+
{!hasSearch ? (
578582
<div className="flex flex-col items-center justify-center gap-y-4 py-8">
579583
<Header2>You haven't set any environment variables yet.</Header2>
580584
<LinkButton
@@ -597,6 +601,14 @@ function EnvironmentVariablesListPage({
597601
)}
598602
</Table>
599603
</div>
604+
{pagination.totalPages > 1 && (
605+
<div className="flex items-center justify-end border-t border-grid-dimmed px-2 py-2">
606+
<PaginationControls
607+
currentPage={pagination.currentPage}
608+
totalPages={pagination.totalPages}
609+
/>
610+
</div>
611+
)}
600612
</div>
601613
</PageBody>
602614
<Outlet />

0 commit comments

Comments
 (0)