Repository navigation
feat(import): import Kindle notebook exports by email - #247
Conversation
Readers can now email the HTML notebook export from the Kindle app to ClippingKK. Resend receives the mail and calls /api/v3/inbound/resend (email.received); the route checks the Svix signature and queues an import-email job, which parses the attachment and saves its highlights as the sender's clippings. - Only addresses bound and verified under Settings → Email import are imported (email_bindings, at most 10 per reader). Adding an address mails a 6-digit code; a verified address belongs to one reader. Unbound senders are logged as an error and dropped. - Kindle notes go to a new reader_notes table, linked to the highlight they follow, and are listed under Settings → Notes for now. - The Kindle notebook parser recognises highlights by their colour span, so localised exports work, and stores pageAt like the .txt importer. Highlights already imported from My Clippings.txt are skipped. - createClippings' insert moves to src/server/clippings/create.ts so the mutation and the email import share dedup, nouns, webhooks and cache. - Account deletion removes a reader's bindings and notes. Needs migration 0006 (pnpm db:migrate) before deploy, and the env vars RESEND_WEBHOOK_SECRET and INBOUND_EMAIL_ADDRESS; see docs/email-import.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CCcqmwTy6GqiC7gN4HCjMa
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
…lin-shhvxl # Conflicts: # src/server/gate/__tests__/database.test.ts
An email from an address no reader has verified used to be logged and dropped. It is still logged, and now the sender also gets a reply in the same thread with the notebook attached as a flat JSON clipping list (clippings and notes with title, author, content and pageAt), plus a short English and Chinese note on binding the address to import. From is easy to forge, so replies are capped: 5 a day and 30 a month per sender address (UTC calendar buckets in Redis, keyed by a hash of the address) and 500 a day in all. The sender's own limits are checked first, so one address can't use up everyone's allowance. An email counts once even when its job retries, and the send reuses a Resend idempotency key so a retry never sends twice. Mail that fails DMARC or carries no notebook is never answered. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CCcqmwTy6GqiC7gN4HCjMa
Review: email import of Kindle notebook exportsI read the webhook, bindings, importer and reply code. I didn't run the tests or the app. The design is careful. Svix verification, a bounded body, a deterministic job id, a counted-before-compare code check, the partial unique index on verified addresses, a Resend idempotency key on replies, and docs that state the forgery limits openly all hold up. Tests cover the parser, importer, bindings, reply and webhook. Worth addressing
Smaller points
Nits
Overall this is solid, and I'd merge after addressing (1), and (2) if the header is dependable. |
Addresses review feedback on the email import. - An email from an unverified address is now counted against the reply limits before anything is fetched, notebook or not, and over a limit the job ends without calling Resend or downloading. Strangers' mail is also limited to one HTML attachment of at most 5 MB, so junk mail can no longer keep the shared queue busy with 5 x 10 MB downloads. - claimReplyQuota reads all three counters before bumping any, so a full global allowance doesn't spend senders' own and vice versa. - The DMARC result is counted in app.email_import.authentication (pass, fail, none, ... or absent), to show whether requiring a pass for imports is safe before doing it. - A binding code that couldn't be mailed is withdrawn instead of left live. - Comments on the text-only dedup trade-offs, and the reply's origin link is escaped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CCcqmwTy6GqiC7gN4HCjMa
|
Thanks for the review. These points are addressed in e65d030: 1. Work done for unauthenticated senders
2. Missing
Smaller points
Not changed
Generated by Claude Code |
Review: Kindle notebook import by emailI read the webhook, bindings, reply and importer code. I did not run the tests or the app, and I skipped the UI, the locales and the migration. The structure is clean and the layers are well separated. The webhook verifies the Svix signature, bounds the body size and answers quickly. Binding codes are compared in constant time and the attempt is counted before the compare. The reply path is rate-limited with a retry-safe idempotency key. The notes are explicit about what the code does and doesn't protect. Security1. Imports into a reader's account only block on
2. A forged sender can burn another address's reply quota (
3. Attacker-controlled content is mailed to a third party
4. Account enumeration (
Correctness and robustness
Performance
TestsI saw coverage for bindings, import, parser, reply and webhook. I'd add:
Nits
Overall this is solid, thoughtful work. Item 1 is the one I'd want addressed before merge. The rest can be follow-ups. |
Second round of review feedback on the email import. - From is the only credential for a bound address, so when Resend reports a DMARC result, anything but a pass (none, temperror, ...) is now neither imported nor answered. That stops forgeries from domains without a DMARC policy writing into a reader's library or triggering JSON replies. Mail with no reported result still goes through, so imports keep working if the header turns out to be missing; the authentication metric shows how often that happens. - Adding an address another reader holds no longer answers EMAIL_ALREADY_BOUND up front, which let any signed-in reader probe which addresses are bound. The unique index refuses it at verify time, to someone who proved they own the address. - saveNotebook reads the reader's same-title clippings once and links notes from those rows plus the inserted ones, instead of querying again. - Tests pin the DMARC rule and the one-copy behaviour for a highlight repeated on another page; docs cover the DMARC trade-off and that a forger can spend another address's reply allowance. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CCcqmwTy6GqiC7gN4HCjMa
|
Second review addressed in 8443c0d: Security
Performance and tests
Not changed
Generated by Claude Code |
Review: Kindle notebook email importI read the server code (
Security / correctness (worth addressing)
Smaller points
TestsCoverage looks good: bindings, import, reply, webhook, parser and dialog. I'd add:
StyleIt follows CLAUDE.md: the settings UI uses lake-ui, namespaces and locales are in place, and the schema comes with its drizzle migration. I'd confirm that en, zhCN and ko have matching keys, since the repo has a test for that. This PR adds no |
…on replies Third round of review feedback on the email import. - Mail is imported or answered only on a DMARC pass. Mail Resend reports no result for is now refused too, unless EMAIL_IMPORT_ALLOW_MISSING_DMARC=1; the authentication metric shows whether that is needed after deploy. - The reply quota is charged only when a reply goes out, after the DMARC and notebook checks, so forged or notebook-less mail spends no sender's or the global allowance. Before fetching, a read-only check turns away senders already at their limit, and a separate cap of 1,000 fetches a day bounds the work done for strangers. - INBOUND_EMAIL_ADDRESS is required: the webhook answers 503 and the worker drops mail while it is unset, instead of importing mail sent to any address of the receiving domain. - The binding routes read the id from the route params instead of slicing the URL. - Tests cover the missing-DMARC switch, the unset inbound address, the quota not being spent by forged or junk mail, the fetch cap and the webhook's configuration check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CCcqmwTy6GqiC7gN4HCjMa
|
Third review addressed in 5caf273: Security / correctness
Smaller points
Tests
Locales: the repo has no Generated by Claude Code |
Review: email import of Kindle notebook exportsWell-structured PR. I read Security
Correctness
Performance
Tests and styleCoverage across the webhook, bindings, import, reply and parser looks solid. I'd add: the forged Overall a solid approach. I'd resolve item 1 before merging; the rest are suggestions. |
Fourth round of review feedback on the email import. - DMARC is read by a small RFC 8601 parser (authentication.ts) instead of a loose regex over one header: every header differing only in case is read, comments and quoted strings are ignored, only a dmarc= that starts a result counts, and any non-pass among several results wins. A sender can no longer pass with `spf=fail (dmarc=pass)`, a quoted reason or a second lower-case header of their own. - EMAIL_IMPORT_AUTHSERV_ID pins the server whose results are trusted; the docs say how to read it off a received email in Resend. - An email let through only by EMAIL_IMPORT_ALLOW_MISSING_DMARC is logged as a WARN each time. - A retried job no longer spends a second fetch of the daily cap for strangers' mail. - The Wenqu book match takes the first hit whose title matches the notebook's, ignoring case, spacing and punctuation, and otherwise leaves the clippings unmatched instead of filing them under whatever ranked first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CCcqmwTy6GqiC7gN4HCjMa
|
Fourth review addressed in 38d8c7b: 1.
Not fixable from here: if Resend's Unit tests cover each forgery above. Import tests cover the pinned id rejecting a pass from another server. 2. 3. Code brute force: agreed it's acceptable, unchanged. 4. Quota counters:
Correctness
Tests added:
A retry of an already-claimed reply not re-spending quota was already covered. Generated by Claude Code |
Review: email import of Kindle notebook exportsI read the webhook, Security
Correctness and robustness
Style and conventions
TestsCoverage is strong: bindings (PGlite), DMARC variants, the webhook signature and replay, the importer, quotas and the dialog. Gaps:
Overall this is a solid feature. I'd treat point 1 as blocking, or at least as a decision to record explicitly, and the rest as follow-ups. |
Summary
Readers can now email the HTML Notebook export from the Kindle iPhone/iPad app (subject like
iPhone Notebook export for 经略幽燕:宋辽战争军事灾难的战略分析 (曾瑞龙)) to ClippingKK.POST /api/v3/inbound/resendwith anemail.receivedevent.import-emailBullMQ job.dmarc=pass, then parses the attached HTML. If the sender is an address a reader has bound and verified, the highlights become that reader's clippings.What's in it
email_bindings)/api/v3/email-bindingsaccept the cookie session only, like the MCP token routes.authentication.ts)Authentication-Resultsby a strict RFC 8601 parser:EMAIL_IMPORT_AUTHSERV_ID(optional) ignores results stamped by any other server.dmarc=passis imported or answered. Mail with no result is dropped too, unlessEMAIL_IMPORT_ALLOW_MISSING_DMARC=1; each email let through that way logs a WARN.app.email_import.authentication.reply.ts){ clippings: [{ title, author, content, pageAt, createdAt }], notes: [{ title, author, content, pageAt }] }.reader_notes)kindle-notebook.ts, usingnode-html-parser)pageAtuses the.txtimporter's format: the page when there is one, else#location.import.ts)createClippings' insert moves tosrc/server/clippings/create.tswith no change in behaviour, so the mutation and the email import share it.mainis merged in, alongside itsmcp_tokenspurge.docs/email-import.mdcovers setup, flow, replies and limits.CLAUDE.mdgets a short section.Deploy notes
pnpm db:migratebefore deploying. Migration0006_email_importaddsemail_bindingsandreader_notes, and the startup schema check exits while they are missing.*.resend.appaddress) and add a webhook foremail.receivedpointing athttps://<APP_ORIGIN>/api/v3/inbound/resend.RESEND_WEBHOOK_SECRET(whsec_…) andINBOUND_EMAIL_ADDRESSare required. Without either, the webhook answers 503.RESEND_API_KEYfetches received mail and sends the JSON replies.EMAIL_IMPORT_ALLOW_MISSING_DMARCdefaults to0.Authentication-Resultsheader: setEMAIL_IMPORT_AUTHSERV_IDto the first word of its value.EMAIL_IMPORT_ALLOW_MISSING_DMARC=1. The docs explain the risk.Test plan
pnpm test: 86 files, 657 tests. New tests cover:Authentication-Results: comments, quoted strings, duplicate and case-variant headers, joined values, the authserv-id pin.txtdedup and the repeated-highlight case, book reuse and title matching, notes, the body fallbackpass,none,temperror, absent, the env switch and its warningpnpm lint,pnpm format:check,pnpm typecheck,pnpm typecheck:serverpnpm dev:🤖 Generated with Claude Code
https://claude.ai/code/session_01CCcqmwTy6GqiC7gN4HCjMa
Generated by Claude Code