Conversation
During a sync the app froze: clicks stopped registering, opening a book did nothing, and the window eventually died. Three compounding causes: - Transfer progress was pushed to the store on EVERY chunk. With 3-5 concurrent transfers that sustains a synchronous React update storm on the renderer main thread (readest hit the same bug — their Sentry READEST-2 — and ships a progress throttle). Chunk progress is now throttled to one emit per 300ms per task; task start/completion still emit immediately. - Every DB write wrapped in runWithDbRetry waited up to 12s for a running sync to finish before even attempting the write (waitForSyncToSettle), so opening a book stalled for the full timeout. The settle wait is now capped at 1.5s — genuine SQLite lock contention is already handled by the retry loop itself. - applyChanges yielded the main thread only every 100 applied records; large remote snapshots could hog it for seconds between yields. Now every 25.
…ces per-device snapshots
The per-device snapshot layout (device-{id}.json = all 11 tables, full dump
per sync) was the root cause of sync dragging the whole UI down: one giant
JSON.stringify/parse per pass, O(library × devices) payload growth (AI chat
history and reading sessions grow unbounded), and every sync re-downloading
and re-applying every peer even when nothing changed.
New cloud layout (under /readany/sync):
- index.json — per-book/thread markers {b, a, d} / {t, d}, written
read-merge-write as a union (tombstones prevent resurrection)
- books/{bookId}.json — book row + ALL its highlights/notes/bookmarks +
per-table deleted maps (per-item LWW merge)
- threads/{threadId}.json — thread metadata only
- chat/{YYYY-MM-DD}.json — chat messages by creation day; devices keep a
pulled-day cursor and only fetch missing days (plus days changed under
the cursor by late offline pushes)
- profile/{tags,book_tags,book_groups,skills}.json — single-table files
- sessions/{YYYY-MM}.json — reading sessions in monthly shards
Every request is now KB-sized and sync work is O(changed). Sync also
propagates annotation deletions per book: sync_tombstones gained a book_id
column (migration + attribution at delete time + deletedBookIds on the
legacy wire format for LAN).
- LAN sync intentionally stays on the legacy device-snapshot protocol
(runSimpleSync); cloud backends (webdav/s3) switch to runPerBookSync.
- No backward compatibility for old cloud snapshots by design: users reset
the remote folder and re-sync once.
…ss the webview heap plugin-http serializes request bodies via Array.from(new Uint8Array(body)) into a JSON number array over IPC: every multi-megabyte book upload froze the renderer main thread for seconds and stalled the whole app during sync. Implement the desktop uploadFile/downloadFile platform methods as Tauri commands that stream directly between disk and the WebDAV server (reqwest on the tokio pool, 300s timeout, optional insecure-TLS). The per-book engine already prefers these entry points, so book/cover transfers now bypass the webview entirely and downloads report progress over an IPC channel (throttled by the engine's progress gate).
- Sync concurrency is now a setting (1-6, default 2): weak gateways/NAS were returning 502 under the first full-sync burst of parallel requests. Plumbs through config -> store -> syncFiles (upload/download/migration/ remote-cleanup pools) with a settings UI row and 7-locale labels. - The native Rust WebDAV upload/download now retries transient failures (network errors, 429, 5xx) up to 3 times with exponential backoff, matching the JS client's retry policy — a single gateway blip no longer fails a whole book transfer.
…oss runs On flaky tunnels (502/connection-reset mid-run) a single failed request aborted the whole sync even though 11 books had already uploaded. Per-item failures during pull/push (books, and later phases) are now logged and skipped; index entries are only written for successful pushes, so the next sync retries exactly the missing pieces and converges. Directory creation failures are also tolerated (per-file self-heal retries later).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
问题
WebDAV 同步过程中应用整体卡死:点击书籍无响应、菜单无法操作,Windows 判定应用未响应,最终直至应用退出("闪退")。未同步时 UI 响应正常;首次全量同步(数据量最大)时最严重,且随使用时间(AI 聊天记录等数据累积)持续恶化。
根因分析(四层问题叠加)
1. 传输层:plugin-http 把文件 body 变成 JS 数字数组
@tauri-apps/plugin-http的 fetch 实现中(dist-js/index.js:68):每本 multi-MB 的书籍上传会构造一个千万级元素的 JS 数组再走 IPC 序列化,在渲染主线程上单次阻塞数秒;下载响应体同样分块经过主线程。
2. 进度回调风暴
每个传输分块触发一次
onProgress→ zustandset({progress})→ React 全量重渲染。3-5 路并发传输下每秒产生几十次重渲染,主线程被渲染任务淹没。(readest 项目在 Sentry READEST-2 中报告过完全相同的问题,并以进度节流修复——本 PR 采用同一方案。)3. DB 写等待
runWithDbRetry默认waitForSyncToSettle:同步期间任何 DB 写入(包括打开书籍时的阅读记录写入)会先空等最多 12 秒才执行。这就是"同步时点击书籍打不开"的直接原因。4. 全量快照架构(根本原因)
旧引擎每台设备维护一个
device-{id}.json全量快照(11 张表所有行):JSON.stringify→ PUT → 拉取所有 peer 全量 → 逐行应用。主线程 JSON 工作量与传输量随数据量线性恶化,永不收敛。修复方案
阶段 1:响应性止血
waitForSyncToSettle上限 12s → 1.5s(真正的 SQLite 锁冲突由既有重试机制处理);applyChanges每 25 行让出一次主线程。阶段 2:按书同步引擎(核心重构,对齐 readest 的成熟模式)
新远端布局(
/readany/sync/下):index.json{b: 书updated_at, a: 注解标记, d: 删除墓碑},各设备"读取-合并-写入"联合更新books/{bookId}.jsonthreads/{id}.jsonchat/{YYYY-MM-DD}.jsonprofile/{table}.jsonsessions/{YYYY-MM}.json效果:每次请求负载从 MB 级降到 KB 级,同步工作量从 O(全库×设备数) 变为 O(变更数),主线程上的 JSON 工作与库大小解耦。
原生流式传输(消灭问题 1)
桌面端实现此前缺失的
uploadFile/downloadFile平台方法:新增 Rust 命令webdav_upload_file/webdav_download_file,用 reqwest 在 tokio 线程池中直接在磁盘与服务器之间流式读写——大文件字节完全不进入 webview 进程,下载经 IPC 通道上报进度(由阶段 1 的节流闸门控制 UI 频率)。容错收敛(弱网/不稳定服务器)
在真实环境中发现服务器偶发 502/连接重置会让整轮同步中止。现在单条目失败仅记录并跳过(书籍推拉、目录创建、分片合并各自独立容错),index 只记录成功条目——下一轮同步依据 marker 自动补齐缺失部分,多轮收敛;index 写入失败则如实报错(该文件是成员关系的唯一事实来源,不能静默丢失)。
并发可配置
真实环境还出现弱网关在首同步并发压力下返回 502。新增"同步并发数"设置(1-6,默认 2),贯通 config → store → 文件同步各阶段池,附 7 种语言的设置 UI 文案。
验证
兼容性说明(重要)
/readany/sync/旧device-*.json并重新同步一次(一次性操作);sync_tombstones新增可空book_id列(启动时自动迁移),其余本地结构零改动。注:本次代码改动及PR信息由AI生成。