Add Beeper Desktop chat archiving#458
Conversation
roborev: Combined Review (
|
roborev: Combined Review (
|
|
Addressed the roborev findings in dcd966a, each with a regression test:
|
roborev: Combined Review (
|
|
Round-2 findings addressed in the latest commit, both with regression tests: mid-chat checkpoints now persist buffered reply pairs together with the advanced cursor (tested across both interrupt paths), and messages whose media was removed at the source clear their stale attachment rows/pending markers on re-persist. |
roborev: Combined Review (
|
|
Round-3 findings addressed in 77ebf10: BackfillMedia now skips only on 404 (transient fetch failures are recorded as errors and keep their markers — regression-tested), and pending-attachment messages are buffered into a slice before the per-item download/write work instead of holding a read cursor open across it. |
roborev: Combined Review (
|
|
Thank you for working on this, I am not a Beeper user but this is a good opportunity to set it up! |
Happy to! I like beeper a lot, good options to run your own bridges, self-host etc. And I generally like breaking open walled-garden chat products. |
|
Round-4 findings addressed in the latest commit, both regression-tested: a 404'd source message now clears its pending media marker (no more eternal retries), and bare-ID participant cache entries are upgraded when phone/email metadata appears later in the run, preserving cross-archive dedup. |
roborev: Combined Review (
|
|
Round-5 findings addressed in the latest commit, all regression-tested: backfill-beeper-media now runs the same anchor verification as the main sync before trusting stored message IDs; the gone-message cleanup clears only pending markers and preserves downloaded attachment rows; and anchor re-arming falls back to chats known from prior sync state when a run enumerates none. |
roborev: Combined Review (
|
|
Round-6 findings addressed in the latest commit, both regression-tested: backfill-beeper-media re-arms a soft-cleared anchor before persisting its state (it becomes the newest completed baseline), and the anchor chat-probe only reports a reinstall on 404 — transient probe failures now fail the run retryably instead of prompting a remove/re-add. |
roborev: Combined Review (
|
|
Round-7 finding addressed in the latest commit: the sync-beeper CLI loop now collects per-account errors and continues with the remaining networks, rebuilds the cache regardless, and reports failures aggregated — matching the sync/sync-full convention and the scheduler path. |
roborev: Combined Review (
|
|
Round-8 findings addressed in the latest commit: the reinstall guard now keeps up to three anchor probes in distinct chats, so deleting any one anchored chat is ordinary churn while a reinstall (which invalidates all of them) still fails fast — regression-tested for both cases. Transient media-backfill fetch failures now count their kept markers as still-pending and the CLI surfaces the error count. |
roborev: Combined Review (
|
|
Round-9 finding addressed in the latest commit: both sync and media-backfill runs force a final checkpoint with their closing counters before CompleteSync, so quick runs no longer show stale/zero counts in sync diagnostics (asserted in the e2e test). Separately: the red test check is not this PR — govulncheck started flagging two new Go stdlib CVEs (GO-2026-5856, GO-2026-4970) fixed in go1.26.5, with all traces through pre-existing code (imap, api, backup, emlx). Every branch will fail that step until CI's Go is bumped from 1.26.4; happy to send a separate PR for the toolchain bump if useful. |
roborev: Combined Review (
|
|
Round-10 finding addressed in the latest commit: since chat/room IDs survive a reinstall, losing every anchor probe is no longer excused by the chats still existing — recently archived messages now arbitrate. If one resolves at the source with a matching timestamp the losses were churn (e.g. disappearing-message chats); otherwise the run fails as a reinstall before trusting any stored cursor. Covered by the existing churn/reinstall tests plus enriched media fixtures. |
roborev: Combined Review (
|
|
Round-11 finding addressed in the latest commit: phone/email-resolved participants now persist their Beeper user ID as an identifier (re-pointing any existing bare-ID row), so bare sender IDs in later runs resolve to the same person across cache resets — regression-tested in both directions (rich-then-bare and weak-then-upgraded). |
roborev: Combined Review (
|
|
Round-12 finding addressed in the latest commit: verification no longer fast-paths the zero-anchor case — with no probes, recently archived messages are verified instead (no-op for fresh accounts), so the reinstall guard cannot lapse for an account with history even if a prior re-arm failed. Regression test covers both the passing (intact install, re-arms) and blocking (re-assigned IDs) sides. |
roborev: Combined Review (
|
|
Round-13 finding addressed in the latest commit: when a rich (phone/email) resolution discovers the Beeper user ID is owned by a weak bare-ID participant, that participant's messages, reactions, mentions, and membership are merged into the rich row (constraint-safe dedup) and the fork deleted — guarded so a previous owner with real contact metadata is never absorbed. Verified on SQLite and PostgreSQL. |
roborev: Combined Review (
|
roborev: Combined Review (
|
|
Rebased onto main now that #460 is merged (the toolchain commit drops out of this diff), and pushed a holistic cleanup pass over the whole PR — a structural review for readability after the round-by-round fixes, plus the round-15 finding:
|
roborev: Combined Review (
|
|
Round-16 finding addressed in the latest commit: the merge guard now blocks only on phone-bearing prior owners, so a cross-run email-to-phone upgrade folds the email participant's history into the phone-deduped one (an email-only owner of the same Beeper user ID is the same person seen with weaker metadata). The email rung also never downgrades a phone-resolved identity. Fresh-resolver cross-run test covers the merge, the preserved email, and the no-downgrade direction. |
roborev: Combined Review (
|
|
Good stuff. I am going to try to get #464 merged first and then rebase this, bear with me |
Archive every chat network bridged through Beeper Desktop via its local REST API, with resumable synchronization, full message fidelity, media backfill, and reinstall safety. This gives msgvault users one durable archive across the chat services connected to their desktop client. Follow-up refinements folded into this commit: - Address roborev review: anchor re-arm, bounded media reads, reply/reaction durability - Address roborev round 2: checkpoint-consistent reply pairs, stale media cleanup - Address roborev round 3: backfill-media error reporting, buffered pending list - Address roborev round 4: gone-message marker cleanup, resolver cache upgrade - Address roborev round 5: backfill anchor guard, precise marker cleanup - Address roborev round 6: media runs keep the guard armed, probe errors stay transient - Address roborev round 7: sync-beeper continues past per-account failures - Address roborev round 8: multi-anchor reinstall guard, honest pending counts - Address roborev round 9: persist final sync counters on completion - Address roborev round 10: archived-sample arbiter when no anchor survives - Address roborev round 11: persist Beeper user IDs on rich participants - Address roborev round 12: the reinstall guard never lapses on zero anchors - Address roborev round 13: merge weak participant history on rich upgrade - Holistic cleanup: reshape for readability, simplify accreted logic, tighten tests - Address roborev round 16: cross-run email-to-phone upgrades merge history Generated with Codex Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
roborev: Combined Review (
|
Archives every chat network bridged through Beeper Desktop — WhatsApp, Signal, Telegram, Instagram, LinkedIn, X, Facebook, Matrix — via its local REST API. Supersedes #455 as a single consolidated PR.
Usage
Each connected network account becomes its own source (
signal,whatsapp, …), so networks stay separately filterable in the TUI and search; all rows sharemessage_type = "beeper".What's stored
Message text with FTS (voice-note transcriptions included), reactions, replies, mentions, group membership, deletion tombstones (archived content survives network deletions), media in the content-addressed attachment store, and the verbatim API JSON per message (
raw_format = beeper_json).Behavior notes
--fullre-fetches and upserts in place.hasMorenever goes false; re-served pages at history edges) are handled and regression-tested against recorded behavior.[beeper]config covers daemon scheduling, per-network include/exclude (e.g. excludewhatsappwhen using the native importer), and a media size cap;backfill-beeper-mediaretries failed downloads.Store/query changes
SetMessageEditedhelper (nothing wroteis_editedbefore) and a same-second tiebreak inGetLastSuccessfulSync.TextMessageTypesinstead of five hand-maintained literals, and the TUI snippet fallback applies to all text types (was WhatsApp-only).Verified against a live Beeper Desktop with 8 accounts: full backfill, incremental re-runs, media download, interrupt/resume.
🤖 Generated with Claude Code