Repository navigation
fix(expenses): dedupe double-posted receipts, Personal/Tech sides - #1607
Conversation
- sync: collapse same-receipt double-posts (vendor/amount/day/label fingerprint) and skip suppressed idempotency keys, so one receipt is always one net expense row going forward - schema: suppressed_expenses table for ledger rows posted in error - categories: Personal and Tech top-level sides; all business stays under Tech with the existing subcategories (AI, Cloud/Hosting, Domains, Software, Services, Other); Personal empty for now - dashboard: filter bar leads with All / Personal / Tech; title Expenses - 26 unit tests green
This comment has been minimized.
This comment has been minimized.
- fingerprint uses full occurredAt timestamp with NUL delimiters (no day-truncation, no adjacent-field bleed) - sync deletes dedupe losers from D1, not just the upsert batch
Code Review Completed! 🔥The code review was successfully completed based on your current configurations. Kody Guide: Usage and ConfigurationInteracting with Kody
Current Kody ConfigurationReview OptionsThe following review options are enabled or disabled:
|
|
Sentence-spacing violation in the PR-body bullet leaves only one space between “... Kody rule violation: Use two spaces between sentences in every human-facing string and agent-written paragraph |
| for (const loser of unique.dropped) { | ||
| await env.EXPENSES_DB.prepare( | ||
| "DELETE FROM expenses WHERE idempotency_key = ?" | ||
| ) | ||
| .bind(loser.idempotencyKey) | ||
| .run(); | ||
| } |
There was a problem hiding this comment.
The new dedupe cleanup in workers/expenses-site/src/sync.mjs creates one sequential await ...run() D1 round trip per loser in the per-page loop instead of using the env.EXPENSES_DB.batch(...) call for upserts two lines later, making a page with K colliding receipts cost K serialized round trips per sync run. Because losers never enter suppressed_expenses (schema.sql:30-38, which defines the table for this double-post case), every cron run resends the same K DELETE statements even after the rows are gone; add prepared loser DELETEs to the same db.batch() as the upserts and/or persist loser keys in suppressed_expenses so later runs skip them.
const rows = unique.kept.map(mapUpstreamExpense);
const deletes = unique.dropped.map((loser) =>
env.EXPENSES_DB
.prepare("DELETE FROM expenses WHERE idempotency_key = ?")
.bind(loser.idempotencyKey)
);
const statements = [
...deletes,
...buildUpsertStatements(env.EXPENSES_DB, rows, syncedAt),
];Prompt for LLM
File workers/expenses-site/src/sync.mjs:
Line 189 to 195:
The new dedupe cleanup in `workers/expenses-site/src/sync.mjs` creates one sequential `await ...run()` D1 round trip per loser in the per-page loop instead of using the `env.EXPENSES_DB.batch(...)` call for upserts two lines later, making a page with K colliding receipts cost K serialized round trips per sync run. Because losers never enter `suppressed_expenses` (`schema.sql:30-38`, which defines the table for this double-post case), every cron run resends the same K DELETE statements even after the rows are gone; add prepared loser DELETEs to the same `db.batch()` as the upserts and/or persist loser keys in `suppressed_expenses` so later runs skip them.
Suggested Code:
const rows = unique.kept.map(mapUpstreamExpense);
const deletes = unique.dropped.map((loser) =>
env.EXPENSES_DB
.prepare("DELETE FROM expenses WHERE idempotency_key = ?")
.bind(loser.idempotencyKey)
);
const statements = [
...deletes,
...buildUpsertStatements(env.EXPENSES_DB, rows, syncedAt),
];
Talk to Kody by mentioning @kody
Was this suggestion helpful? React with 👍 or 👎 to help Kody learn from this interaction.
Summary
Fixes double-posted receipts showing up as duplicate expenses on the dashboard, and reorganizes expense categorization around two top-level sides (Personal / Tech).
Functional changes
Expense sync (
workers/expenses-site/src/sync.mjs)expenseFingerprintanddedupeUpstreamRowsto collapse the same receipt when it was posted twice under different idempotency keys (e.g., the Sep 2026 bulk run filing several receipts under bothone_timeandusage). Matching is based on vendor, amount, calendar day, and label, keeping the most canonical kind.loadSuppressedKeysto read the new suppression table and skip those idempotency keys on every sync run.syncExpensesnow filters suppressed keys, dedupes the remaining rows before upserting, and reportssuppressedCountanddedupedCountalongsideupsertedandpages.Schema (
workers/expenses-site/schema.sql)suppressed_expensestable storing idempotency keys that should never render on the dashboard, with areasonandsuppressed_at. Used for ledger rows posted in error (double-posted receipts, superseded gross/discount/correction rows) corrected at the display layer.Categories (
workers/expenses-site/src/categories.mjs)SIDES(Personal, Tech) withsideForCategoryandsideLabel. All business categories map to the Tech side; thepersonalcategory maps to Personal.tech-ailabel from "Tech / AI" to "AI" and added apersonalcategory.Dashboard API (
workers/expenses-site/src/index.mjs)readDashboardRowsnow accepts side filters (personal,tech) in addition to category filters, mirroringsideForCategoryin SQL via aCASEexpression derived from category (no schema change).sidefield per row.Rendering (
workers/expenses-site/src/render.mjs)data-sideattribute for client-side filtering.Tests
syncExpenses; updated category and render tests for the new side filters and "AI" label.Review findings (per automated review, not addressed here)
The review agent reported 3 high-severity concerns in
workers/expenses-site/src/sync.mjs: the day-granularity fingerprint may merge distinct same-day charges; the dedupe filters upserts but does not delete the collapsed loser from D1; and the fingerprint concatenates fields without a delimiter, which can cause collisions.