feat: schedule ingestion engine#31
Open
chiptus wants to merge 23 commits into
Open
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Implements the dry-run diff engine for the new schedule ingestion system. Compares a CSV payload against current DB state for a festival edition and returns clean operations + conflicts requiring user resolution (orphaned sets, stage name mismatches). Core business logic extracted into diff.ts with 22 unit tests covering slugging, timezone conversion, B2B matching, and midnight crossing. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds the atomic write path for the schedule ingestion system: - Migration: adds UNIQUE constraints on artists.slug and stages(festival_edition_id, name), creates the commit_schedule PL/pgSQL RPC that wraps all writes (artist upserts, stage upserts, set inserts/updates, set_artists sync, orphan archiving) in a single transaction with full rollback on failure. - Edge Function: thin admin-gated HTTP handler that calls the RPC via service role key. - Integration tests for the RPC covering create, update, archive, and time storage. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR replaces the legacy client-side CSV schedule import with a server-driven ingestion workflow backed by Supabase Edge Functions and a transactional Postgres RPC, plus a new admin import wizard UI (upload → diff/conflicts → commit).
Changes:
- Added
diff-scheduleandcommit-scheduleEdge Functions, plus acommit_schedulePostgres RPC to perform atomic schedule writes. - Implemented a new admin schedule import wizard UI with stage-mismatch and orphan-set resolution flows.
- Added unit tests for diffing logic and integration tests for the commit RPC.
Reviewed changes
Copilot reviewed 25 out of 26 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| vite.config.ts | Adds test runner config exclusions. |
| supabase/migrations/20260509142022_commit_schedule_rpc.sql | Adds constraints and the transactional commit_schedule RPC used by ingestion. |
| supabase/functions/_shared/auth.ts | Shared admin auth + CORS helpers for Edge Functions. |
| supabase/functions/diff-schedule/index.ts | Edge Function endpoint to compute a diff from CSV rows vs DB. |
| supabase/functions/diff-schedule/diff.ts | Core diff/matching logic (artists, stages, sets, orphan detection). |
| supabase/functions/diff-schedule/diff.test.ts | Unit tests covering slugging, time conversion, matching rules, and conflicts. |
| supabase/functions/commit-schedule/index.ts | Edge Function endpoint that calls the commit_schedule RPC. |
| supabase/functions/commit-schedule/commit-schedule.test.ts | Integration tests targeting the RPC behavior against local Supabase. |
| src/services/scheduleImportService.ts | Frontend service layer for parsing CSV + invoking diff/commit + building commit payloads. |
| src/pages/admin/FestivalScheduleImport.tsx | New admin page wrapper for the import wizard route. |
| src/pages/admin/FestivalEdition.tsx | Adds an “Import” tab and routing to the new import page. |
| src/components/router/GlobalRoutes.tsx | Wires the /import sub-route under festival edition admin routes. |
| src/components/Admin/ScheduleImport/ScheduleImportWizard.tsx | Wizard state machine: upload → review → commit result, plus cache invalidation. |
| src/components/Admin/ScheduleImport/CsvUploadStep.tsx | CSV upload + timezone selection + invokes diff. |
| src/components/Admin/ScheduleImport/DiffReviewStep.tsx | Review UI container including conflicts and commit action. |
| src/components/Admin/ScheduleImport/DiffSummaryBanner.tsx | Summary banner for diff results. |
| src/components/Admin/ScheduleImport/StageMismatchResolver.tsx | UI to map mismatched stage names or create new stages. |
| src/components/Admin/ScheduleImport/OrphanedSetsPanel.tsx | UI to archive/keep orphaned sets not present in CSV. |
| src/components/Admin/ScheduleImport/CommitResultCard.tsx | Success UI and “import another file” reset action. |
Adds the full schedule ingestion frontend: - ScheduleImportWizard: 3-step flow (upload → review → result) - CsvUploadStep: file drop zone, timezone picker, CSV parse + diff call - DiffSummaryBanner: counts for new artists/stages/sets/conflicts - StageMismatchResolver: map-to-existing (dropdown) or create-new per mismatch - OrphanedSetsPanel: per-set archive/keep toggle, bulk action, default keep - scheduleImportService: CSV parser, Edge Function callers, commit payload builder - Import tab added to FestivalEdition page, route wired in GlobalRoutes Refactors diff.ts SetPayload to use stageName instead of stage_id so the payload aligns directly with what the commit RPC expects. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Extracts DiffReviewStep and CommitResultCard from ScheduleImportWizard to keep all components under 150 lines per codebase conventions. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Prevents Vitest from picking up supabase/functions Deno tests and tests/e2e Playwright specs, which caused import errors. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Deletes CSVImportDialog and csvImportService (326 lines of client-side import logic). Import CSV buttons removed from StageManagement and SetManagement — replaced by the dedicated Import tab on the edition page. parseCSV inlined into scheduleImportService to remove the dependency. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
chiptus
force-pushed
the
feat/schedule-ingestion
branch
from
4f1b288 to
c8110dd
Compare
|
✅ DB Migrate succeeded for |
chiptus
commented
May 9, 2026
Previously v_sets_updated incremented unconditionally regardless of whether the UPDATE matched a row, and the subsequent DELETE FROM set_artists ran without any edition filter. Combined with the service-role context, a forged or wrong-edition set id could wipe artist links across editions while still being reported as a successful update. Capture ROW_COUNT after the UPDATE, raise if no row matched (so the transaction rolls back), and join the DELETE through sets with the edition filter as a defense-in-depth guard.
Duplicate slugs in the CSV (or duplicate rows resolving to the same artist) would error against the UNIQUE(set_id, artist_id) constraint and roll back the whole import. Match the update path's behaviour and skip duplicates.
Hand-rolled keys like ["sets", editionId] don't match the actual factories
(setsKeys.byEdition is ["sets","edition",editionId], stagesKeys.byEdition is
["stages", { editionId }]), so the schedule view silently failed to refresh
after a successful import. Use the factories so the invalidation actually
hits the live caches.
The earlier removal commit (c8110dd) deleted CSVImportDialog and csvImportService but left their dependents — CSVImportPage, the old $festivalId/$editionId/import route, the entire src/services/csv tree, useMatchingSetsQuery, and a stale Upload-CSV button in StageManagement — all of which still pointed at the deleted route. Also imports CsvRow into CsvUploadStep where it was used as a state generic but never declared (PR review comment) and drops the now-unused CsvRow import in ScheduleImportWizard. Regenerates routeTree.gen.ts so the new edition-scoped /import route is registered. These are bundled because the project-wide typecheck pre-commit hook won't pass with either fix in isolation.
Raw mismatch.csvValue went straight into id/htmlFor, so stage names with spaces or special characters produced invalid HTML, and duplicate names across rows would collide and break label/radio binding. Extracting a MismatchRow child component lets us use useId() per row so each option gets a unique, valid id without sanitisation gymnastics. Addresses both the DOM-id PR comment and the request to break the loop body into its own component.
Two leak fixes for the integration tests: - The first test only deleted the artist, leaving the created set in the edition forever. Use a unique set name and delete by name+edition. - The midnight-crossing test selected sets without id, then tried to delete by sets[0].id, which silently no-op'd. Include id in the select.
Replace the manual loading/error/result state in CsvUploadStep (file read + analyse) and ScheduleImportWizard (commit) with useMutation. Extract the FileReader-based handler into a plain async readFile(file) so the mutation can simply await it instead of wrapping the callback API. Mutation status drives the UI state directly — no more setLoading/setError/setCommitting bookkeeping.
…Zone CsvUploadStep was carrying a Select, a drop zone, hidden input, helper text, and the action button all in one component. Pull the timezone Select into TimezonePicker and the drop zone (icon, file name, row count, hidden input, format helper) into CsvDropZone. The parent now reads as the actual flow: pick a timezone, drop a file, hit analyse.
The per-row layout with switch and labels was inline inside the panel mapper. Pull it into OrphanedItem so the panel reads as 'header + list' and the row owns its own switch wiring.
Inline the page component into the import route. The wrapper was a 3-line file that only forwarded params, so keeping it as a separate module didn't add anything.
The component-level useQuery duplicated work the parent route already does via beforeLoad ensureQueryData. Hoist the call into a route loader so the component receives a resolved FestivalEdition through useLoaderData and the loading/not-found branches go away — the router blocks rendering until the data is ready.
The original 100+ line loop body did artist resolution, stage resolution with a four-branch tree, time computation, set matching, and dispatch all inline. Pull each phase into a named helper: - buildIndexes for the lookup maps - resolveArtists for slug derivation + new-artist accumulation - resolveStage returning a tagged kind so the caller maps it onto the right accumulator - computeTimes for the date/time conversion (incl. midnight crossing) - findMatchingSet for the candidate-narrowing logic The orchestrator now reads as the actual pipeline: resolve → match → dispatch. Behaviour is unchanged.
Replace the ad-hoc 'is festivalEditionId truthy?' check with a zod schema covering every field the RPC consumes, including UUID format on the edition id and archive ids. Bad input now returns 400 with the field-level issues instead of failing later inside the RPC with an opaque error.
…, timestamp parse and artist sync The body of commit_schedule was repeating four patterns inline: - a stage_id resolution subquery on (edition, name) - a CASE WHEN ... ::TIMESTAMPTZ for nullable timestamp casts - a hand-rolled regex slug builder - the delete-then-insert-on-conflict dance for set_artists Pull each into a commit_schedule__-prefixed helper so the main body reads as the actual workflow rather than a wall of subqueries. Behaviour is unchanged — the sync helper still scopes its DELETE through sets to preserve edition isolation.
Two unrelated test infra fixes that were both pre-existing: - Vitest reads vitest.config.ts when present, which overrides the test block in vite.config.ts. The previous fix added 'supabase/**' to vite.config.ts only, so the Deno tests in supabase/functions/ kept getting picked up. Move the exclude into vitest.config.ts and drop the dead block in vite.config.ts. - The Supabase client throws at module init when the env vars are missing. Component tests that mock the query hooks still trigger that init through the import graph. Stub the two vars in src/test/setup.ts so the client can construct (it's never actually called).
Two pre-existing oxlint failures the project-wide lint surfaced: - scheduleImportService.parseScheduleCsv had an arrow-function const helper, which the func-style rule rejects. - commit-schedule.test.ts midnight-crossing test destructured data but only asserted on error.
The migration was failing on staging because (a) the artists.slug dedup suffix wasn't guaranteed unique — using just the first 6 chars of the id can still collide — and (b) stages had duplicate (edition, name) pairs in prod that blocked the new unique constraint outright. Switch both dedups to append the full id, which is guaranteed unique. Add a stages dedup mirroring the artists one. Wrap both ADD CONSTRAINT statements in DO blocks that skip if a constraint of the same name (or the equivalent stages_name_festival_edition_id_key from PR #28) already exists, so the migration is safe to re-run.
Comment on lines
231
to
233
| '/admin/festivals/$festivalSlug': typeof AdminFestivalsFestivalSlugRouteWithChildren | ||
| '/admin/festivals/import': typeof AdminFestivalsImportRoute | ||
| '/festivals/$festivalSlug/': typeof FestivalsFestivalSlugIndexRoute |
Comment on lines
+201
to
+206
| (resolvedStageId | ||
| ? (candidates.find((s) => s.stage_id === resolvedStageId) ?? null) | ||
| : null) ?? | ||
| (date | ||
| ? (candidates.find((s) => s.time_start?.startsWith(date)) ?? null) | ||
| : null) ?? |
Comment on lines
+61
to
+66
| SELECT LOWER( | ||
| REGEXP_REPLACE( | ||
| REGEXP_REPLACE(TRIM(p_name), '[^a-zA-Z0-9\s]', '', 'g'), | ||
| '\s+', '-', 'g' | ||
| ) | ||
| ); |
Comment on lines
+193
to
+196
| p_festival_edition_id, | ||
| v_set_elem->>'name', | ||
| commit_schedule__slugify(v_set_elem->>'name'), | ||
| NULLIF(v_set_elem->>'description', ''), |
Comment on lines
+162
to
+167
| name = v_set_elem->>'name', | ||
| description = NULLIF(v_set_elem->>'description', ''), | ||
| stage_id = commit_schedule__resolve_stage_id( | ||
| p_festival_edition_id, v_set_elem->>'stageName' | ||
| ), | ||
| time_start = commit_schedule__parse_ts(v_set_elem->>'timeStart'), |
Comment on lines
+63
to
+66
| } catch (error) { | ||
| console.error("diff-schedule error:", error); | ||
| return new Response(JSON.stringify({ error: error.message }), { | ||
| status: 500, |
Comment on lines
+87
to
+90
| } catch (error) { | ||
| console.error("commit-schedule error:", error); | ||
| return new Response(JSON.stringify({ error: error.message }), { | ||
| status: 500, |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces the old client-side CSV import with a server-side ingestion system.
Two Supabase Edge Functions (
diff-schedule,commit-schedule) handle the diff and atomic commit via a Postgres RPC. The frontend wizard walks admins through upload → conflict resolution → commit.Key design decisions: sets matched by artist roster + stage (preserving votes), orphaned sets surfaced as explicit archive/keep conflicts, stage name mismatches resolved via map-to-existing or create-new, all writes wrapped in a single transaction with full rollback on failure.