Skip to content

feat(import): record inbound emails for readers and admins - #249

Merged
AnnatarHe merged 2 commits into
mainfrom
claude/happy-galileo-071e5o
Oct 12, 2026
Merged

AnnatarHe merged 2 commits into
mainfrom
claude/happy-galileo-071e5o

Conversation

@AnnatarHe

Copy link
Copy Markdown
Member

Email sent to /api/v3/inbound/resend left no record anywhere except OTel logs (with the sender masked) and email_bindings.last_imported_at. Readers couldn't tell whether an export worked, and admins couldn't see mail from unverified senders. This PR records every inbound email the worker handles, shows readers their own, and gives admins a page for mail no verified address sent.

What changes

New table inbound_emails (drizzle/0007_inbound_emails.sql, generated by drizzle-kit)

  • One row per Resend email_id.
  • Columns: sender, subject, received time, status, detail, DMARC result, notebook titles, highlight count, clippings created, notes created, attempts.
  • user_id is the reader whose verified address sent the email. Mail from a pending address gets no reader, because anyone can claim an address.
  • As agreed, the full normalized sender address is stored and nothing is purged. Rows are removed only on account deletion.

Recording (src/server/email-import/records.ts, import.ts, worker.ts)

  • run() fills in a facts object as it goes. importEmail() upserts the row after every attempt.
  • A thrown attempt is recorded as retrying, or failed on the last of the 5. The worker now passes the attempt number and whether it is the last one. detail keeps only the first line of the error, because Drizzle puts the query's parameters on the second line.
  • On retries, created counts add up, since saving a notebook isn't one transaction. user_id keeps its first value if a retry no longer finds the binding.
  • Recording is best effort. A failed write logs a WARN and never fails the import or makes it run again.
  • The webhook itself is unchanged and still doesn't touch the database.

Reader view

  • Settings → Email import gets a "Recent emails" section with the latest 50.
  • Each row shows a status badge, the time in UTC, sender, subject, books, counts, and a hint for problem statuses (DMARC failure, no notebook, expired, retrying, failed).
  • Error text is never sent to readers.

Admin view

  • New page /dash/<you>/admin/inbound-emails, with the same isAdmin guard and offset pagination as the existing admin page.
  • It lists rows with no reader, newest first, including the full sender, the translated reply limit or the error, DMARC, and attempt count.
  • Both admin pages get a small NavTabs nav, rendered after the guard and not from a layout.

Account deletion removes the reader's rows, plus rows with no reader that came from the reader's verified addresses.

Other

  • formatDateTime() (date and time, in UTC and labelled as such).
  • Translations in en, zhCN and ko.
  • docs/email-import.md and CLAUDE.md updated.

Deploy note

Run pnpm db:migrate before deploying. The startup schema check exits when inbound_emails is missing.

Testing

  • pnpm exec vitest run: 88 files, 679 tests pass. New and extended tests:
    • import.test.ts: a row for each outcome, retrying then succeeding, the final failure, and an import that still succeeds when the row can't be written.
    • New records.test.ts: upsert merging, clipping, list scoping.
    • database.test.ts: which rows account deletion removes.
    • New inbound-email-list.test.tsx: what readers and admins each see.
    • format-date.test.ts.
  • pnpm lint and pnpm format:check are clean. pnpm typecheck:server is clean.
  • pnpm typecheck reports only the 4 asset-module errors that main already has here, because the gitignored next-env.d.ts is missing.
  • I didn't run the app in a browser, since sign-in needs Gate. The UI is covered by the component render test.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GFQ6mgubacomHnKckCPg4A


Generated by Claude Code

Every attempt the worker makes at an inbound email now upserts one row in
`inbound_emails` (keyed by Resend's email id): sender, subject, received
time, status, DMARC result, notebook titles and what was created. A thrown
attempt is recorded as `retrying`, or `failed` on the last of the 5, with
the error's first line. Created counts add up across retries; recording is
best effort and never fails or repeats an import.

- Settings → Email import lists the reader's latest 50 emails (only mail
  their verified addresses sent), with a hint per status and no error text.
- /dash/<you>/admin/inbound-emails lists mail no verified address sent,
  with the full sender, reply limit, DMARC and attempts; both admin pages
  get a nav.
- Account deletion removes the reader's rows and the reader-less rows from
  their verified addresses.
- formatDateTime shows a time in UTC, labelled as such.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GFQ6mgubacomHnKckCPg4A
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

Main split the importer (saving into save.ts, logging into log.ts) and
added the `unpinned` DMARC result. saveNotebook now adds what each chunk
saves to the record's counts from save.ts; the local Outcome type stays
replaced by ImportOutcome.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GFQ6mgubacomHnKckCPg4A
@claude

claude Bot commented Oct 12, 2026

Copy link
Copy Markdown
Contributor

Review: record inbound emails

Overall this is a clean, well-scoped change. The best-effort recording, the first-line-only detail, keeping error text away from readers, and the NavTabs guard ordering are all sound. I read records.ts, the admin page and deletion.ts; I did not run the tests or the app.

Worth addressing

  1. Unbounded growth from strangers. Every email the worker handles now gets a permanent row, including mail from unverified senders, and "nothing is purged". The reply path is rate-limited (5/day, 30/month, 500/day overall), but recording is not. Anyone who finds the inbound address can fill inbound_emails with up to 255-char senders, 512-char subjects and a books jsonb. I know retention was agreed for the full sender address. I'd still consider a purge for user_id IS NULL rows older than N days, or at least a row cap, since these rows have no owner who could ever delete them.
  2. Forged mail can crowd out a reader's list. user_id is set from the claimed From address even when DMARC fails (the docs say this is intentional so readers can see why). Someone who knows a reader's address can send 50+ spoofed emails and push real entries out of the "latest 50". It's low severity, but you could cap failed_authentication rows shown per reader, or collapse them. Subjects are attacker-controlled; they render through React so they're escaped, which is fine.
  3. Admin pagination has no total. The next arrow shows whenever emails.length === PAGE_SIZE, so a last page that is exactly 50 rows leads to an empty page. Fetching limit + 1 rows is the usual fix. Deep OFFSET scans will also get slower as strangers' rows pile up (see 1).

Minor

  • records.ts upsert: on a redelivery of an already-finished job, attempts and status are overwritten by the new attempt, so an imported row could flip to something else. That's probably fine, but it deserves a comment or a guard such as only overwriting when excluded.attempts >= inbound_emails.attempts.
  • highlights, clippings_created, notes_created and attempts are bigint, which Drizzle returns as number or string depending on the mode. Check that the schema uses mode: 'number'. int would also be enough for these columns.
  • The user_id column has no FK. That matches the rest of the schema, so it's consistent. The cleanup in deletion.ts depends on the sender string equalling the normalized binding email, so a test with a mixed-case sender would be useful.
  • sender is varchar(255) and the raw From header is stored when no address parses. That's clipped, so it's safe.
  • The limit in listInboundEmails runs a separate count() over the same predicate on every settings page load. That's cheap thanks to the (user_id, received_at) index.

Tests

Coverage is good: every outcome, retry then success, final failure, and the best-effort write failure. A test for the redelivery case above and for the spoofed-DMARC listing would round it out.

🤖 Generated with Claude Code

@AnnatarHe
AnnatarHe merged commit 186350a into main Oct 12, 2026
2 checks passed
@AnnatarHe
AnnatarHe deleted the claude/happy-galileo-071e5o branch October 12, 2026 11:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants