diff --git a/packages/rs-platform-wallet-storage/SCHEMA.md b/packages/rs-platform-wallet-storage/SCHEMA.md index c05725583f..a47dcec901 100644 --- a/packages/rs-platform-wallet-storage/SCHEMA.md +++ b/packages/rs-platform-wallet-storage/SCHEMA.md @@ -11,9 +11,9 @@ chain. ## What it stores — and the boundary The persister stores **public** wallet-state material (UTXOs, transactions, -account registrations, address pools, identities, identity public keys, -contacts, asset locks, token balances, DashPay overlays, and -platform-address sync snapshots) in a SQLite database managed by +account registrations, identities, identity public keys, contacts, asset +locks, token balances, DashPay overlays, and platform-address sync +snapshots) in a SQLite database managed by [refinery](https://crates.io/crates/refinery) migrations. **No secrets are stored here.** Mnemonics, seeds, and raw private keys never @@ -37,20 +37,18 @@ Any `meta_*` row whose parent object does not exist — because it was never cre A future garbage-collection pass is expected to reap orphan metadata — rows with no live parent object older than approximately one week — but no such GC is implemented yet. Callers should not rely on orphan metadata persisting forever, nor assume it will be cleaned up promptly. `meta_global` is intentionally parentless and always survives. -The 23 tables are split into five domain diagrams below. `WALLETS` is the root anchor and appears in each diagram. For full column listings see the [Tables](#tables) section. +The 21 tables are split into five domain diagrams below. `WALLETS` is the root anchor and appears in each diagram. For full column listings see the [Tables](#tables) section. ## Diagram 1 — Core / L1 (Bitcoin/Dash layer) -Account registrations, address-pool snapshots, transactions, UTXOs, instant locks, derived addresses, and SPV sync state. +Account registrations, transactions, UTXOs, instant locks, and SPV sync state. ```mermaid erDiagram WALLETS ||--o{ ACCOUNT_REGISTRATIONS : "registers" - WALLETS ||--o{ ACCOUNT_ADDRESS_POOLS : "snapshots" WALLETS ||--o{ CORE_TRANSACTIONS : "records" WALLETS ||--o{ CORE_UTXOS : "owns" WALLETS ||--o{ CORE_INSTANT_LOCKS : "holds" - WALLETS ||--o{ CORE_DERIVED_ADDRESSES : "derives" WALLETS ||--o| CORE_SYNC_STATE : "tracks" CORE_TRANSACTIONS ||--o{ CORE_UTXOS : "spends" @@ -62,19 +60,11 @@ erDiagram ACCOUNT_REGISTRATIONS { BLOB wallet_id PK - TEXT account_type PK "standard | coinjoin | identity_registration | ..." + TEXT account_type PK "standard_bip44 | standard_bip32 | coinjoin | identity_registration | ..." INTEGER account_index PK BLOB account_xpub_bytes "bincode-encoded AccountRegistrationEntry" } - ACCOUNT_ADDRESS_POOLS { - BLOB wallet_id PK - TEXT account_type PK - INTEGER account_index PK - TEXT pool_type PK "external | internal | absent | absent_hardened" - BLOB snapshot_blob "bincode-encoded AccountAddressPoolEntry" - } - CORE_TRANSACTIONS { BLOB wallet_id PK BLOB txid PK "32-byte Txid" @@ -102,15 +92,6 @@ erDiagram BLOB islock_blob "bincode-encoded InstantLock" } - CORE_DERIVED_ADDRESSES { - BLOB wallet_id PK - TEXT account_type PK - TEXT address PK "bech32 / Base58 address string" - INTEGER account_index - TEXT derivation_path "pool_type/derivation_index" - INTEGER used "0 | 1" - } - CORE_SYNC_STATE { BLOB wallet_id PK "one row per wallet" INTEGER last_processed_height "NULL until first block processed" @@ -364,14 +345,6 @@ lookups without blob decoding. - PK: `(wallet_id, account_type, account_index)`. - FK: `wallet_id → wallets(wallet_id) ON DELETE CASCADE`. -### `account_address_pools` - -Address-pool snapshot per `(wallet, account, pool_type)`. `pool_type` is -one of `external`, `internal`, `absent`, `absent_hardened`. - -- PK: `(wallet_id, account_type, account_index, pool_type)`. -- FK: `wallet_id → wallets(wallet_id) ON DELETE CASCADE`. - ### `core_transactions` One row per transaction the wallet has seen. `height`, `block_hash`, and @@ -401,15 +374,6 @@ finalized. Rows are removed when the transaction becomes confirmed. - PK: `(wallet_id, txid)`. - FK: `wallet_id → wallets(wallet_id) ON DELETE CASCADE`. -### `core_derived_addresses` - -Address-to-account-index map. Written before UTXOs in the same -transaction so the UTXO writer can resolve `account_index` by address. - -- PK: `(wallet_id, account_type, address)`. -- FK: `wallet_id → wallets(wallet_id) ON DELETE CASCADE`. -- Index: `idx_core_derived_addresses_addr(wallet_id, address)`. - ### `core_sync_state` One row per wallet, holding monotonically-advancing SPV sync watermarks. @@ -590,27 +554,24 @@ before the address exists. ## Enum-domain CHECK constraints -Seven TEXT columns carry a `CHECK (col IN (...))` across five enum -domains — `account_type` is reused in three tables. The IN-list is built -at migration time from `pub(crate) const *_LABELS` arrays declared next -to each writer function. Four domains mirror an upstream Rust enum; the -fifth (`contacts.state`) is a synthetic lifecycle label naming which -`ContactChangeSet` slot a row came from: +Four TEXT columns carry a `CHECK (col IN (...))` across four enum +domains. The IN-list is built at migration time from +`pub(crate) const *_LABELS` arrays declared next to each writer function. +Three domains mirror an upstream Rust enum; the fourth (`contacts.state`) +is a synthetic lifecycle label naming which `ContactChangeSet` slot a row +came from: | Table | Column | Source-of-truth const | |---|---|---| | `wallets` | `network` | `sqlite::schema::wallets::NETWORK_LABELS` | | `account_registrations` | `account_type` | `sqlite::schema::accounts::ACCOUNT_TYPE_LABELS` | -| `account_address_pools` | `account_type` | `sqlite::schema::accounts::ACCOUNT_TYPE_LABELS` | -| `account_address_pools` | `pool_type` | `sqlite::schema::accounts::POOL_TYPE_LABELS` | -| `core_derived_addresses` | `account_type` | `sqlite::schema::accounts::ACCOUNT_TYPE_LABELS` | | `asset_locks` | `status` | `sqlite::schema::asset_locks::ASSET_LOCK_STATUS_LABELS` | | `contacts` | `state` | `sqlite::schema::contacts::CONTACT_STATE_LABELS` | The const arrays are the single source of truth shared by the writer mapping functions (`network_to_str`, `account_type_db_label`, -`pool_type_db_label`, `status_str`, `contact_state_db_label`) and the -migration's CHECK clauses. +`status_str`, `contact_state_db_label`) and the migration's CHECK +clauses. Per-module `*_labels_match_enum` unit tests enforce set-equality between each const and the writer's codomain — drift (a renamed/added upstream variant) fails the test rather than landing as silent garbage @@ -619,10 +580,9 @@ in this document; the source files are canonical. ### Upstream-enum coupling -Three of the persisted enums live in the external `rust-dashcore` -crate (`key_wallet::Network`, `key_wallet::account::AccountType`, -`key_wallet::managed_account::address_pool::AddressPoolType`); the -fourth (`platform_wallet::wallet::asset_lock::tracked::AssetLockStatus`) +Two of the persisted enums live in the external `rust-dashcore` +crate (`key_wallet::Network`, `key_wallet::account::AccountType`); the +third (`platform_wallet::wallet::asset_lock::tracked::AssetLockStatus`) is in-tree and carries a `# Schema coupling` rustdoc block. Because the upstream definitions cannot be edited from this repository, @@ -639,7 +599,7 @@ mechanisms working together: TODO(rust-dashcore): once the upstream `key_wallet` crate is vendored or the project gains push access there, mirror the in-tree -`AssetLockStatus` `# Schema coupling` doc block on the three upstream +`AssetLockStatus` `# Schema coupling` doc block on the two upstream enums so a developer editing them upstream sees the constraint without having to grep this repo. @@ -683,4 +643,4 @@ having to grep this repo. | Version | File | Description | |---|---|---| -| V001 | `V001__initial.rs` | Full schema: all 23 tables (including the six `meta_*` per-object metadata tables), every index, and six triggers (`setnull_core_utxos_on_tx_delete` + the five `meta_*` soft-cascade triggers) | +| V001 | `V001__initial.rs` | Full schema: all 21 tables (including the six `meta_*` per-object metadata tables), every index, and six triggers (`setnull_core_utxos_on_tx_delete` + the five `meta_*` soft-cascade triggers) | diff --git a/packages/rs-platform-wallet-storage/migrations/V001__initial.rs b/packages/rs-platform-wallet-storage/migrations/V001__initial.rs index 7caaaf2519..30a0d8b851 100644 --- a/packages/rs-platform-wallet-storage/migrations/V001__initial.rs +++ b/packages/rs-platform-wallet-storage/migrations/V001__initial.rs @@ -29,8 +29,8 @@ //! read back) at every connection open via `open_conn` //! (`src/sqlite/conn.rs`). //! -//! Enum-shaped TEXT columns (`network`, `account_type`, `pool_type`, -//! `status`, `state`) carry a `CHECK (col IN (...))` clause whose +//! Enum-shaped TEXT columns (`network`, `account_type`, `status`, +//! `state`) carry a `CHECK (col IN (...))` clause whose //! IN-list is built from the `*_LABELS` const arrays in //! `crate::sqlite::schema::{wallets, accounts, asset_locks, //! contacts}`. The consts are the single source of truth shared with @@ -51,7 +51,6 @@ pub fn migration() -> String { let network_check = build_check_in(crate::sqlite::schema::wallets::NETWORK_LABELS); let account_type_check = build_check_in(crate::sqlite::schema::accounts::ACCOUNT_TYPE_LABELS); - let pool_type_check = build_check_in(crate::sqlite::schema::accounts::POOL_TYPE_LABELS); let asset_lock_status_check = build_check_in(crate::sqlite::schema::asset_locks::ASSET_LOCK_STATUS_LABELS); let contact_state_check = @@ -82,16 +81,6 @@ CREATE TABLE account_registrations ( FOREIGN KEY (wallet_id) REFERENCES wallets(wallet_id) ON DELETE CASCADE ); -CREATE TABLE account_address_pools ( - wallet_id BLOB NOT NULL, - account_type TEXT NOT NULL CHECK (account_type IN {account_type_check}), - account_index INTEGER NOT NULL, - pool_type TEXT NOT NULL CHECK (pool_type IN {pool_type_check}), - snapshot_blob BLOB NOT NULL, - PRIMARY KEY (wallet_id, account_type, account_index, pool_type), - FOREIGN KEY (wallet_id) REFERENCES wallets(wallet_id) ON DELETE CASCADE -); - CREATE TABLE core_transactions ( wallet_id BLOB NOT NULL, txid BLOB NOT NULL, @@ -143,19 +132,6 @@ CREATE TABLE core_instant_locks ( FOREIGN KEY (wallet_id) REFERENCES wallets(wallet_id) ON DELETE CASCADE ); -CREATE TABLE core_derived_addresses ( - wallet_id BLOB NOT NULL, - account_type TEXT NOT NULL CHECK (account_type IN {account_type_check}), - account_index INTEGER NOT NULL, - address TEXT NOT NULL, - derivation_path TEXT NOT NULL, - used INTEGER NOT NULL, - PRIMARY KEY (wallet_id, account_type, address), - FOREIGN KEY (wallet_id) REFERENCES wallets(wallet_id) ON DELETE CASCADE -); - -CREATE INDEX idx_core_derived_addresses_addr ON core_derived_addresses(wallet_id, address); - CREATE TABLE core_sync_state ( wallet_id BLOB NOT NULL PRIMARY KEY, last_processed_height INTEGER, diff --git a/packages/rs-platform-wallet-storage/src/sqlite/error.rs b/packages/rs-platform-wallet-storage/src/sqlite/error.rs index 17cd68daa3..4864709501 100644 --- a/packages/rs-platform-wallet-storage/src/sqlite/error.rs +++ b/packages/rs-platform-wallet-storage/src/sqlite/error.rs @@ -218,13 +218,6 @@ pub enum WalletStorageError { limit_bytes: usize, }, - /// An unspent UTXO named an address absent from - /// `core_derived_addresses`, so its account index can't be resolved; - /// persisting it would mis-file live funds, so the write is refused. - /// Spent-only placeholder rows tolerate a missing mapping. - #[error("unspent utxo address {address} is not in core_derived_addresses")] - UtxoAddressNotDerived { address: String }, - /// `PRAGMA foreign_keys = ON` was issued on open but the read-back /// reported the constraint enforcement is still off — the linked /// SQLite build silently ignores the pragma (no FK support compiled @@ -372,7 +365,6 @@ impl WalletStorageError { | Self::IdentityEntryIdMismatch | Self::AssetLockEntryMismatch { .. } | Self::BlobTooLarge { .. } - | Self::UtxoAddressNotDerived { .. } | Self::IntegerOverflow { .. } => false, } } @@ -449,7 +441,6 @@ impl WalletStorageError { Self::IdentityEntryIdMismatch => "identity_entry_id_mismatch", Self::AssetLockEntryMismatch { .. } => "asset_lock_entry_mismatch", Self::BlobTooLarge { .. } => "blob_too_large", - Self::UtxoAddressNotDerived { .. } => "utxo_address_not_derived", Self::IntegerOverflow { .. } => "integer_overflow", } } diff --git a/packages/rs-platform-wallet-storage/src/sqlite/persister.rs b/packages/rs-platform-wallet-storage/src/sqlite/persister.rs index 07b62afd97..9369fce02d 100644 --- a/packages/rs-platform-wallet-storage/src/sqlite/persister.rs +++ b/packages/rs-platform-wallet-storage/src/sqlite/persister.rs @@ -1053,9 +1053,10 @@ fn apply_changeset_to_tx( if !cs.account_registrations.is_empty() { schema::accounts::apply_registrations(tx, wallet_id, &cs.account_registrations)?; } - if !cs.account_address_pools.is_empty() { - schema::accounts::apply_pools(tx, wallet_id, &cs.account_address_pools)?; - } + // `account_address_pools` is intentionally NOT applied: UTXO attribution + // is hardcoded to the default account (index 0) in `core_state`, so the + // pool snapshot is no longer a storage input. The changeset field is kept + // for API stability and still feeds non-storage consumers. if let Some(core) = cs.core.as_ref() { schema::core_state::apply(tx, wallet_id, core)?; } diff --git a/packages/rs-platform-wallet-storage/src/sqlite/schema/accounts.rs b/packages/rs-platform-wallet-storage/src/sqlite/schema/accounts.rs index 0d929a6f57..7a71ad881e 100644 --- a/packages/rs-platform-wallet-storage/src/sqlite/schema/accounts.rs +++ b/packages/rs-platform-wallet-storage/src/sqlite/schema/accounts.rs @@ -1,22 +1,21 @@ -//! `account_registrations` + `account_address_pools` writers + readers -//! (platform-payment registrations and the keyless account-manifest -//! reader). +//! `account_registrations` writer + keyless reader (platform-payment +//! registrations and the rehydration account-manifest oracle). use std::collections::BTreeMap; use key_wallet::bip32::ExtendedPubKey; use rusqlite::{params, Connection, Transaction}; -use platform_wallet::changeset::{AccountAddressPoolEntry, AccountRegistrationEntry}; +use platform_wallet::changeset::AccountRegistrationEntry; use platform_wallet::wallet::platform_wallet::WalletId; use crate::sqlite::error::WalletStorageError; use crate::sqlite::schema::blob; use crate::sqlite::schema::blob::impl_persistable_blob; -// PUBLIC material only: account-manifest types (account xpubs / pool -// snapshots) reaching `_blob` columns. -impl_persistable_blob!(AccountRegistrationEntry, AccountAddressPoolEntry); +// PUBLIC material only: the account-registration xpub manifest reaching +// the `account_xpub_bytes` blob column. +impl_persistable_blob!(AccountRegistrationEntry); /// Decoded `platform_payment` account registration: the DIP-17 account /// index and its extended public key, recovered from the bincode-serde @@ -125,37 +124,6 @@ pub fn apply_registrations( Ok(()) } -pub fn apply_pools( - tx: &Transaction<'_>, - wallet_id: &WalletId, - entries: &[AccountAddressPoolEntry], -) -> Result<(), WalletStorageError> { - if entries.is_empty() { - return Ok(()); - } - let mut stmt = tx.prepare_cached( - "INSERT INTO account_address_pools \ - (wallet_id, account_type, account_index, pool_type, snapshot_blob) \ - VALUES (?1, ?2, ?3, ?4, ?5) \ - ON CONFLICT(wallet_id, account_type, account_index, pool_type) DO UPDATE SET \ - snapshot_blob = excluded.snapshot_blob", - )?; - for entry in entries { - let account_type = account_type_db_label(&entry.account_type); - let account_index = account_index(&entry.account_type); - let pool_type = pool_type_db_label(&entry.pool_type); - let payload = blob::encode(entry)?; - stmt.execute(params![ - wallet_id.as_slice(), - account_type, - i64::from(account_index), - pool_type, - payload, - ])?; - } - Ok(()) -} - /// Read every `account_registrations` row for `wallet_id` into a keyless /// [`AccountRegistrationEntry`] manifest — the rehydration account-set oracle /// (which accounts to re-derive + the per-account xpubs the wrong-account gate @@ -181,14 +149,18 @@ pub fn load_state( Ok(out) } -/// Source of truth for the `account_type` TEXT domain across -/// `account_registrations`, `account_address_pools`, and -/// `core_derived_addresses`, mirroring [`key_wallet::account::AccountType`]. -/// `migrations/V001__initial.rs` interpolates it into each table's +/// Source of truth for the `account_registrations.account_type` TEXT domain, +/// mirroring [`key_wallet::account::AccountType`]. +/// `migrations/V001__initial.rs` interpolates it into the table's /// `CHECK (account_type IN (...))`; `account_type_labels_match_enum` keeps it /// in sync with [`account_type_db_label`]. +/// +/// `Standard` maps to two distinct labels by `StandardAccountType` variant +/// (`"standard_bip44"` / `"standard_bip32"`) so BIP44 and BIP32 standard +/// accounts with the same index never collide on their shared PK columns. pub(crate) const ACCOUNT_TYPE_LABELS: &[&str] = &[ - "standard", + "standard_bip44", + "standard_bip32", "coinjoin", "identity_registration", "identity_topup", @@ -205,23 +177,23 @@ pub(crate) const ACCOUNT_TYPE_LABELS: &[&str] = &[ "platform_payment", ]; -/// Single source of truth for the `account_address_pools.pool_type` -/// TEXT-column domain. -/// -/// Mirrors every variant of -/// [`key_wallet::managed_account::address_pool::AddressPoolType`] -/// (writer side: [`pool_type_db_label`]). See [`ACCOUNT_TYPE_LABELS`] -/// for the broader rationale and the parity-test contract. -pub(crate) const POOL_TYPE_LABELS: &[&str] = &["external", "internal", "absent", "absent_hardened"]; - /// Stable database label for an `AccountType` variant (the `Debug` impl is not -/// a stable format; this match is the contract). Variants sharing a label are -/// distinguished by the companion `account_index` column. An added upstream -/// variant fails this match's exhaustiveness check at compile time. +/// a stable format; this match is the contract). An added upstream variant +/// fails this match's exhaustiveness check at compile time. +/// +/// `Standard` maps to two distinct labels by `StandardAccountType` so BIP44 +/// and BIP32 accounts with the same `index` never collapse onto the same PK. pub(crate) fn account_type_db_label(at: &key_wallet::account::AccountType) -> &'static str { - use key_wallet::account::AccountType; + use key_wallet::account::{AccountType, StandardAccountType}; match at { - AccountType::Standard { .. } => "standard", + AccountType::Standard { + standard_account_type: StandardAccountType::BIP44Account, + .. + } => "standard_bip44", + AccountType::Standard { + standard_account_type: StandardAccountType::BIP32Account, + .. + } => "standard_bip32", AccountType::CoinJoin { .. } => "coinjoin", AccountType::IdentityRegistration => "identity_registration", AccountType::IdentityTopUp { .. } => "identity_topup", @@ -239,22 +211,8 @@ pub(crate) fn account_type_db_label(at: &key_wallet::account::AccountType) -> &' } } -/// Stable database label for an `AddressPoolType` variant. -pub(crate) fn pool_type_db_label( - pool: &key_wallet::managed_account::address_pool::AddressPoolType, -) -> &'static str { - use key_wallet::managed_account::address_pool::AddressPoolType; - match pool { - AddressPoolType::External => "external", - AddressPoolType::Internal => "internal", - AddressPoolType::Absent => "absent", - AddressPoolType::AbsentHardened => "absent_hardened", - } -} - /// Numeric account index embedded in an `AccountType`, persisted in the -/// `account_index` column of `account_registrations`, `account_address_pools`, -/// and `core_derived_addresses`. +/// `account_registrations.account_index` column. pub(crate) fn account_index(at: &key_wallet::account::AccountType) -> u32 { use key_wallet::account::AccountType; match at { @@ -282,7 +240,9 @@ mod tests { use std::collections::HashSet; /// Every [`key_wallet::account::AccountType`] variant; the wildcard-free - /// match below fails to compile if upstream adds one. + /// match below fails to compile if upstream adds one. `Standard` appears + /// twice — once per `StandardAccountType` — because both map to distinct + /// labels. fn all_account_type_variants() -> Vec { use key_wallet::account::{AccountType, StandardAccountType}; let variants = vec![ @@ -290,6 +250,10 @@ mod tests { index: 0, standard_account_type: StandardAccountType::BIP44Account, }, + AccountType::Standard { + index: 0, + standard_account_type: StandardAccountType::BIP32Account, + }, AccountType::CoinJoin { index: 0 }, AccountType::IdentityRegistration, AccountType::IdentityTopUp { @@ -340,25 +304,6 @@ mod tests { variants } - fn all_pool_type_variants() -> Vec { - use key_wallet::managed_account::address_pool::AddressPoolType; - let variants = vec![ - AddressPoolType::External, - AddressPoolType::Internal, - AddressPoolType::Absent, - AddressPoolType::AbsentHardened, - ]; - for v in &variants { - match v { - AddressPoolType::External - | AddressPoolType::Internal - | AddressPoolType::Absent - | AddressPoolType::AbsentHardened => {} - } - } - variants - } - #[test] fn account_type_labels_match_enum() { let from_writer: HashSet<&'static str> = all_account_type_variants() @@ -372,18 +317,4 @@ mod tests { from_const, from_writer ); } - - #[test] - fn pool_type_labels_match_enum() { - let from_writer: HashSet<&'static str> = all_pool_type_variants() - .iter() - .map(pool_type_db_label) - .collect(); - let from_const: HashSet<&'static str> = POOL_TYPE_LABELS.iter().copied().collect(); - assert_eq!( - from_writer, from_const, - "POOL_TYPE_LABELS ({:?}) drifted from pool_type_db_label codomain ({:?})", - from_const, from_writer - ); - } } diff --git a/packages/rs-platform-wallet-storage/src/sqlite/schema/core_state.rs b/packages/rs-platform-wallet-storage/src/sqlite/schema/core_state.rs index 1479e37530..8da392a5c7 100644 --- a/packages/rs-platform-wallet-storage/src/sqlite/schema/core_state.rs +++ b/packages/rs-platform-wallet-storage/src/sqlite/schema/core_state.rs @@ -54,39 +54,15 @@ pub fn apply( ])?; } } - // Derived addresses are written before UTXOs (same tx) so the UTXO - // writer's address→account_index lookup sees the fresh rows. - if !cs.addresses_derived.is_empty() { - let mut stmt = tx.prepare_cached( - "INSERT INTO core_derived_addresses \ - (wallet_id, account_type, account_index, address, derivation_path, used) \ - VALUES (?1, ?2, ?3, ?4, ?5, ?6) \ - ON CONFLICT(wallet_id, account_type, address) DO UPDATE SET \ - account_index = excluded.account_index, \ - derivation_path = excluded.derivation_path", - )?; - for da in &cs.addresses_derived { - let account_type = - crate::sqlite::schema::accounts::account_type_db_label(&da.account_type); - let account_index = crate::sqlite::schema::accounts::account_index(&da.account_type); - let pool_type = crate::sqlite::schema::accounts::pool_type_db_label(&da.pool_type); - let address = da.address.to_string(); - let path = format!("{}/{}", pool_type, da.derivation_index); - stmt.execute(params![ - wallet_id.as_slice(), - account_type, - i64::from(account_index), - address, - path, - false - ])?; - } - } + // `addresses_derived` is intentionally NOT persisted here. The iOS + // address registry is fed by the FFI `addresses_derived` callback (fired + // before the UTXO changeset in the same round), and UTXO attribution is + // hardcoded to the default account (index 0); the storage layer keeps no + // derived-address lookup table. if !cs.new_utxos.is_empty() { let mut stmt = tx.prepare_cached(UPSERT_UTXO_SQL)?; - let mut lookup_stmt = tx.prepare_cached(ACCOUNT_INDEX_BY_ADDRESS_SQL)?; for utxo in &cs.new_utxos { - execute_upsert_utxo(&mut stmt, &mut lookup_stmt, wallet_id, utxo, false)?; + execute_upsert_utxo(&mut stmt, wallet_id, utxo, false)?; } } if !cs.spent_utxos.is_empty() { @@ -96,7 +72,6 @@ pub fn apply( "UPDATE core_utxos SET spent = 1 WHERE wallet_id = ?1 AND outpoint = ?2", )?; let mut upsert_stmt = tx.prepare_cached(UPSERT_UTXO_SQL)?; - let mut lookup_stmt = tx.prepare_cached(ACCOUNT_INDEX_BY_ADDRESS_SQL)?; for utxo in &cs.spent_utxos { let op = blob::encode_outpoint(&utxo.outpoint)?; let exists: bool = exists_stmt @@ -106,10 +81,11 @@ pub fn apply( if exists { mark_spent_stmt.execute(params![wallet_id.as_slice(), &op[..]])?; } else { - // Spent-only synthetic row: best-effort account_index. A wrong - // index is inert since spent rows are excluded from + // Spent-only synthetic row for a UTXO we never saw unspent. + // account_index is the hardcoded default like every row, and + // inert anyway since spent rows are excluded from // `list_unspent_utxos`. - execute_upsert_utxo(&mut upsert_stmt, &mut lookup_stmt, wallet_id, utxo, true)?; + execute_upsert_utxo(&mut upsert_stmt, wallet_id, utxo, true)?; } } } @@ -134,14 +110,6 @@ pub fn apply( Ok(()) } -/// Resolve a UTXO's owning account index via the `core_derived_addresses` map. -/// An address can be derived under multiple `account_type`s, so `ORDER BY` with -/// `LIMIT 1` makes the choice deterministic (SQLite would otherwise pick an -/// arbitrary matching row). -const ACCOUNT_INDEX_BY_ADDRESS_SQL: &str = "SELECT account_index FROM core_derived_addresses \ - WHERE wallet_id = ?1 AND address = ?2 \ - ORDER BY account_type, account_index LIMIT 1"; - const UPSERT_UTXO_SQL: &str = "INSERT INTO core_utxos \ (wallet_id, outpoint, value, script, height, account_index, spent, spent_in_txid) \ VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, NULL) \ @@ -152,45 +120,30 @@ const UPSERT_UTXO_SQL: &str = "INSERT INTO core_utxos \ account_index = excluded.account_index, \ spent = excluded.spent"; +/// Account index written for every `core_utxos` row. The product uses only +/// the default account (index 0); a non-default funds account causes +/// `core_bridge::warn_if_non_default_account` to emit a `warn!` log but +/// the record is still persisted under index 0 (dropping it would +/// undercount the balance and lose funds). The one reader +/// (`list_unspent_utxos` per-account grouping) groups everything under 0. +const CORE_UTXO_ACCOUNT_INDEX: i64 = 0; + +/// Upsert one `core_utxos` row. `account_index` is the hardcoded default +/// ([`CORE_UTXO_ACCOUNT_INDEX`]); `spent` marks spent-only synthetic rows. fn execute_upsert_utxo( stmt: &mut rusqlite::CachedStatement<'_>, - lookup_stmt: &mut rusqlite::CachedStatement<'_>, wallet_id: &WalletId, utxo: &Utxo, spent: bool, ) -> Result<(), WalletStorageError> { let op = blob::encode_outpoint(&utxo.outpoint)?; - let address = utxo.address.to_string(); - // `Utxo` carries no account index; recover it from the derived-address map. - let looked_up: Option = lookup_stmt - .query_row(params![wallet_id.as_slice(), &address], |row| row.get(0)) - .optional()?; - let account_index: i64 = match looked_up { - Some(idx) => idx, - // Refuse an unspent UTXO whose address we never derived — it would - // mis-bucket live money under account 0 and never re-derive. The - // spent-only path below tolerates the fallback (a wrong index is inert). - None if !spent => { - return Err(WalletStorageError::UtxoAddressNotDerived { - address: address.clone(), - }); - } - None => { - tracing::debug!( - wallet_id = %hex::encode(wallet_id), - address = %address, - "spent-only UTXO address not found in core_derived_addresses; using account_index 0 placeholder" - ); - 0 - } - }; stmt.execute(params![ wallet_id.as_slice(), &op[..], crate::sqlite::util::safe_cast::u64_to_i64("core_utxos.value", utxo.value())?, utxo.txout.script_pubkey.as_bytes(), i64::from(utxo.height), - account_index, + CORE_UTXO_ACCOUNT_INDEX, spent, ])?; Ok(()) diff --git a/packages/rs-platform-wallet-storage/tests/sqlite_account_zero_attribution.rs b/packages/rs-platform-wallet-storage/tests/sqlite_account_zero_attribution.rs new file mode 100644 index 0000000000..d604c48aa8 --- /dev/null +++ b/packages/rs-platform-wallet-storage/tests/sqlite_account_zero_attribution.rs @@ -0,0 +1,121 @@ +#![allow(clippy::field_reassign_with_default)] + +//! Genesis-rescan regression for the hardcoded `account_index = 0` design +//! (PR #3828). +//! +//! Before: a UTXO landing on a freshly-derived gap-limit-edge address could +//! race address-derivation persistence and be mis-attributed or dropped, so +//! the bridge smuggled a full pool snapshot in-band to resolve it. Now UTXO +//! attribution is hardcoded to the default account (index 0) at the storage +//! writer — no in-band snapshot, no address→account lookup table. This test +//! pins that a UTXO on a real gap-limit-edge address persists directly with +//! `account_index == 0`, contributes the exact balance, and never aborts +//! the flush. + +mod common; + +use common::{ensure_wallet_meta, fresh_persister, wid}; + +use key_wallet::wallet::initialization::WalletAccountCreationOptions; +use key_wallet::wallet::managed_wallet_info::ManagedWalletInfo; +use key_wallet::wallet::Wallet; +use key_wallet::AddressInfo; +use platform_wallet::changeset::{ + CoreChangeSet, PlatformWalletChangeSet, PlatformWalletPersistence, +}; +use platform_wallet::wallet::platform_wallet::WalletId; +use platform_wallet_storage::sqlite::schema::core_state; + +/// The LAST address in the wallet's Standard BIP44 external pool — the +/// gap-limit-edge address, the one most likely to be a fresh extension and +/// thus the worst case for the retired attribution race. +fn gap_limit_edge_address(seed_byte: u8) -> AddressInfo { + use key_wallet::account::AccountType; + use key_wallet::managed_account::address_pool::AddressPoolType; + + let wallet = Wallet::from_seed_bytes( + [seed_byte; 64], + key_wallet::Network::Testnet, + WalletAccountCreationOptions::Default, + ) + .unwrap(); + let info = ManagedWalletInfo::from_wallet(&wallet, 0); + + for managed in info.all_managed_accounts() { + let account_type = managed.managed_account_type().to_account_type(); + if !matches!(account_type, AccountType::Standard { index: 0, .. }) { + continue; + } + for pool in managed.managed_account_type().address_pools() { + if pool.pool_type != AddressPoolType::External || pool.addresses.is_empty() { + continue; + } + let infos: Vec = pool.addresses.values().cloned().collect(); + return infos.last().cloned().unwrap(); + } + } + panic!("wallet must expose a non-empty Standard BIP44 external pool"); +} + +fn utxo_at(addr: &dashcore::Address, vout: u32, value: u64) -> key_wallet::Utxo { + use dashcore::hashes::Hash; + key_wallet::Utxo { + outpoint: dashcore::OutPoint { + txid: dashcore::Txid::from_byte_array([0x7E; 32]), + vout, + }, + txout: dashcore::TxOut { + value, + script_pubkey: addr.script_pubkey(), + }, + address: addr.clone(), + height: 7, + is_coinbase: false, + is_confirmed: true, + is_instantlocked: false, + is_locked: false, + is_trusted: false, + } +} + +/// A UTXO on a freshly-derived gap-limit-edge address persists directly with +/// the hardcoded `account_index == 0`: no snapshot, no lookup, no flush +/// abort, and the unspent balance is exact. +#[test] +fn utxo_on_fresh_gap_limit_address_persists_under_account_zero() { + let (persister, _tmp, _path) = fresh_persister(); + let w: WalletId = wid(0xD1); + ensure_wallet_meta(&persister, &w); + + let edge = gap_limit_edge_address(0x55); + let addr = edge.address.clone(); + + persister + .store( + w, + PlatformWalletChangeSet { + core: Some(CoreChangeSet { + new_utxos: vec![utxo_at(&addr, 0, 777_000)], + ..Default::default() + }), + ..Default::default() + }, + ) + .expect("a UTXO on a fresh gap-limit address must persist, not abort"); + + let conn = persister.lock_conn_for_test(); + let by_account = core_state::list_unspent_utxos(&conn, &w).unwrap(); + + let total: usize = by_account.values().map(|v| v.len()).sum(); + assert_eq!(total, 1, "the edge-address UTXO must persist"); + + let rows = by_account + .get(&0) + .expect("UTXO must be attributed to the default account (index 0)"); + assert_eq!( + rows.len(), + 1, + "exactly the one edge-address UTXO under account 0" + ); + assert_eq!(rows[0].value, 777_000, "value preserved"); +} diff --git a/packages/rs-platform-wallet-storage/tests/sqlite_accounts_reader.rs b/packages/rs-platform-wallet-storage/tests/sqlite_accounts_reader.rs index a351cc0c7c..5839b6e097 100644 --- a/packages/rs-platform-wallet-storage/tests/sqlite_accounts_reader.rs +++ b/packages/rs-platform-wallet-storage/tests/sqlite_accounts_reader.rs @@ -108,7 +108,7 @@ fn a1_corrupt_blob_is_hard_error() { conn.execute( "INSERT INTO account_registrations \ (wallet_id, account_type, account_index, account_xpub_bytes) \ - VALUES (?1, 'standard', 0, X'00')", + VALUES (?1, 'standard_bip44', 0, X'00')", rusqlite::params![w.as_slice()], ) .unwrap(); diff --git a/packages/rs-platform-wallet-storage/tests/sqlite_check_constraints.rs b/packages/rs-platform-wallet-storage/tests/sqlite_check_constraints.rs index 8321468b14..68eadff1d1 100644 --- a/packages/rs-platform-wallet-storage/tests/sqlite_check_constraints.rs +++ b/packages/rs-platform-wallet-storage/tests/sqlite_check_constraints.rs @@ -1,11 +1,7 @@ //! Smoke tests for the enum-domain `CHECK` constraints. The schema has -//! seven such TEXT columns across five domains (`account_type` is reused -//! by `account_registrations`, `account_address_pools`, and -//! `core_derived_addresses`). These tests exercise one column per -//! upstream-enum domain: `wallets.network`, -//! `account_registrations.account_type`, `account_address_pools.pool_type`, -//! and `asset_locks.status`. The synthetic `contacts.state` domain is not -//! exercised here. +//! four such TEXT columns across four domains: `wallets.network`, +//! `account_registrations.account_type`, `asset_locks.status`, and the +//! synthetic `contacts.state`. These tests exercise each directly. //! //! The per-module parity unit tests in `src/sqlite/schema/*` cover the //! Rust↔const-array equality. These tests cover the runtime half: a @@ -71,30 +67,6 @@ fn check_rejects_bad_account_type_on_registrations() { assert_constraint_check(res, "account_registrations.account_type"); } -#[test] -fn check_rejects_bad_pool_type() { - let (persister, _tmp, _path) = fresh_persister(); - let conn = persister.lock_conn_for_test(); - conn.execute( - "INSERT INTO wallets (wallet_id, network, birth_height) VALUES (?1, ?2, ?3)", - params![wid(3).as_slice(), "testnet", 0i64], - ) - .expect("seed wallets"); - let res = conn.execute( - "INSERT INTO account_address_pools \ - (wallet_id, account_type, account_index, pool_type, snapshot_blob) \ - VALUES (?1, ?2, ?3, ?4, ?5)", - params![ - wid(3).as_slice(), - "standard", - 0i64, - "not_a_pool", - &[0u8; 4][..] - ], - ); - assert_constraint_check(res, "account_address_pools.pool_type"); -} - #[test] fn check_rejects_bad_asset_lock_status() { let (persister, _tmp, _path) = fresh_persister(); @@ -137,3 +109,58 @@ fn check_accepts_every_known_label_network() { .unwrap_or_else(|e| panic!("network={label} should be accepted: {e}")); } } + +#[test] +fn check_rejects_bad_contact_state() { + let (persister, _tmp, _path) = fresh_persister(); + let conn = persister.lock_conn_for_test(); + // Seed a valid parent wallet so the insert trips the state CHECK, not the FK. + conn.execute( + "INSERT INTO wallets (wallet_id, network, birth_height) VALUES (?1, ?2, ?3)", + params![wid(7).as_slice(), "testnet", 0i64], + ) + .expect("seed wallets"); + let res = conn.execute( + "INSERT INTO contacts (wallet_id, owner_id, contact_id, state) \ + VALUES (?1, ?2, ?3, ?4)", + params![ + wid(7).as_slice(), + &[0xAAu8; 32][..], + &[0xBBu8; 32][..], + "not_a_contact_state" + ], + ); + assert_constraint_check(res, "contacts.state"); +} + +#[test] +fn check_accepts_every_known_contact_state_label() { + let (persister, _tmp, _path) = fresh_persister(); + let conn = persister.lock_conn_for_test(); + conn.execute( + "INSERT INTO wallets (wallet_id, network, birth_height) VALUES (?1, ?2, ?3)", + params![wid(8).as_slice(), "testnet", 0i64], + ) + .expect("seed wallets"); + // Mirrors `sqlite::schema::contacts::CONTACT_STATE_LABELS`; hardcoded + // because that const is `pub(crate)` and unreachable from this separate + // integration-test crate (same constraint as the network test above). + // The per-module `contact_state_labels_match_enum` unit test guards the + // const itself against drift, so a label added there without updating + // this list surfaces in that test, not as a silent gap here. + for (i, label) in ["sent", "received", "established"].iter().enumerate() { + // Same wallet+owner, distinct contact_id per label to keep the + // composite PK (wallet_id, owner_id, contact_id) unique. + conn.execute( + "INSERT INTO contacts (wallet_id, owner_id, contact_id, state) \ + VALUES (?1, ?2, ?3, ?4)", + params![ + wid(8).as_slice(), + &[0xC0u8; 32][..], + &[i as u8; 32][..], + *label + ], + ) + .unwrap_or_else(|e| panic!("contact state={label} should be accepted: {e}")); + } +} diff --git a/packages/rs-platform-wallet-storage/tests/sqlite_core_state_reader.rs b/packages/rs-platform-wallet-storage/tests/sqlite_core_state_reader.rs index 8c39013416..1e619ee46b 100644 --- a/packages/rs-platform-wallet-storage/tests/sqlite_core_state_reader.rs +++ b/packages/rs-platform-wallet-storage/tests/sqlite_core_state_reader.rs @@ -15,11 +15,24 @@ use key_wallet::wallet::managed_wallet_info::ManagedWalletInfo; use key_wallet::wallet::Wallet; use key_wallet::Utxo; use platform_wallet::changeset::{ - CoreChangeSet, PlatformWalletChangeSet, PlatformWalletPersistence, + AccountRegistrationEntry, CoreChangeSet, PlatformWalletChangeSet, PlatformWalletPersistence, }; use platform_wallet_storage::sqlite::schema::core_state; use platform_wallet_storage::WalletStorageError; +/// Keyless account manifest the rehydration path resolves xpubs from. +fn manifest_for(wallet: &Wallet) -> Vec { + wallet + .accounts + .all_accounts() + .into_iter() + .map(|a| AccountRegistrationEntry { + account_type: a.account_type, + account_xpub: a.account_xpub, + }) + .collect() +} + fn reopen(path: &std::path::Path) -> platform_wallet_storage::SqlitePersister { platform_wallet_storage::SqlitePersister::open( platform_wallet_storage::SqlitePersisterConfig::new(path), @@ -63,31 +76,6 @@ fn wallet_and_utxo(seed: [u8; 64], value: u64, height: u32, vout: u32) -> (Walle (w, utxo) } -/// The `core_derived_addresses` row a real scan records before a UTXO on -/// `address` lands. The strict UTXO writer refuses an unspent UTXO whose -/// address was never derived, so every test paying a wallet address must -/// seed the matching derivation. The writer keys its lookup on -/// `(wallet_id, address)` only, so account_type/index/pubkey here are -/// inert placeholders — the address is the load-bearing field. -fn derived_for(address: &dashcore::Address) -> platform_wallet::DerivedAddress { - // Compressed secp256k1 generator point — a valid placeholder pubkey. - const PUBKEY_G: [u8; 33] = [ - 0x02, 0x79, 0xbe, 0x66, 0x7e, 0xf9, 0xdc, 0xbb, 0xac, 0x55, 0xa0, 0x62, 0x95, 0xce, 0x87, - 0x0b, 0x07, 0x02, 0x9b, 0xfc, 0xdb, 0x2d, 0xce, 0x28, 0xd9, 0x59, 0xf2, 0x81, 0x5b, 0x16, - 0xf8, 0x17, 0x98, - ]; - platform_wallet::DerivedAddress { - account_type: key_wallet::account::AccountType::Standard { - index: 0, - standard_account_type: key_wallet::account::StandardAccountType::BIP44Account, - }, - pool_type: key_wallet::managed_account::address_pool::AddressPoolType::External, - derivation_index: 0, - address: address.clone(), - public_key: dashcore::PublicKey::from_slice(&PUBKEY_G).expect("valid compressed pubkey"), - } -} - /// A non-zero balance survives store → drop → reopen → load, guarding /// against a silent-zero-balance reconstruction. #[test] @@ -101,7 +89,6 @@ fn rt2_nonzero_balance_survives_reopen() { let cs = PlatformWalletChangeSet { core: Some(CoreChangeSet { - addresses_derived: vec![derived_for(&utxo.address)], new_utxos: vec![utxo.clone()], last_processed_height: Some(200), synced_height: Some(200), @@ -128,8 +115,12 @@ fn rt2_nonzero_balance_survives_reopen() { // rehydration path) and assert the wallet balance is the persisted // amount — NOT a silent zero. let mut info = ManagedWalletInfo::from_wallet(&wallet, 1); - platform_wallet::manager::rehydrate::apply_persisted_core_state(&mut info, &core) - .expect("BIP44 reconstruction must not error"); + platform_wallet::manager::rehydrate::apply_persisted_core_state( + &mut info, + &manifest_for(&wallet), + &core, + ) + .expect("BIP44 reconstruction must not error"); let bal = WalletInfoInterface::balance(&info); let total = bal.confirmed() + bal.unconfirmed() + bal.immature() + bal.locked(); assert_eq!( @@ -156,7 +147,6 @@ fn b2_spent_utxo_excluded() { w, PlatformWalletChangeSet { core: Some(CoreChangeSet { - addresses_derived: vec![derived_for(&u_unspent.address)], new_utxos: vec![u_unspent.clone()], spent_utxos: vec![u_spent.clone()], ..Default::default() @@ -267,8 +257,7 @@ fn f2_no_bip44_wallet_nonzero_balance_survives_reopen() { w, PlatformWalletChangeSet { core: Some(CoreChangeSet { - addresses_derived: vec![derived_for(&utxo.address)], - new_utxos: vec![utxo], + new_utxos: vec![utxo.clone()], last_processed_height: Some(60), synced_height: Some(60), ..Default::default() @@ -286,8 +275,12 @@ fn f2_no_bip44_wallet_nonzero_balance_survives_reopen() { assert_eq!(core.new_utxos.len(), 1); let mut info = ManagedWalletInfo::from_wallet(&wallet, 1); - platform_wallet::manager::rehydrate::apply_persisted_core_state(&mut info, &core) - .expect("CoinJoin-only reconstruction must not error"); + platform_wallet::manager::rehydrate::apply_persisted_core_state( + &mut info, + &manifest_for(&wallet), + &core, + ) + .expect("CoinJoin-only reconstruction must not error"); let bal = WalletInfoInterface::balance(&info); let total = bal.confirmed() + bal.unconfirmed() + bal.immature() + bal.locked(); assert_eq!( diff --git a/packages/rs-platform-wallet-storage/tests/sqlite_dashpay_overlay_contract.rs b/packages/rs-platform-wallet-storage/tests/sqlite_dashpay_overlay_contract.rs index 4a2d11f8ec..dd235e1e4d 100644 --- a/packages/rs-platform-wallet-storage/tests/sqlite_dashpay_overlay_contract.rs +++ b/packages/rs-platform-wallet-storage/tests/sqlite_dashpay_overlay_contract.rs @@ -72,6 +72,7 @@ fn overlay_only_write_does_not_corrupt_load() { let mut core_cs = PlatformWalletChangeSet::default(); core_cs.wallet_metadata = Some(WalletMetadataEntry { network: key_wallet::Network::Testnet, + wallet_group_id: [0u8; 32], birth_height: 0, }); core_cs.core = Some(CoreChangeSet { diff --git a/packages/rs-platform-wallet-storage/tests/sqlite_delete_partial_commit_window.rs b/packages/rs-platform-wallet-storage/tests/sqlite_delete_partial_commit_window.rs index f8ee46d83e..1d34a2b12d 100644 --- a/packages/rs-platform-wallet-storage/tests/sqlite_delete_partial_commit_window.rs +++ b/packages/rs-platform-wallet-storage/tests/sqlite_delete_partial_commit_window.rs @@ -23,6 +23,7 @@ fn full_changeset(synced: u32) -> PlatformWalletChangeSet { let mut cs = PlatformWalletChangeSet::default(); cs.wallet_metadata = Some(WalletMetadataEntry { network: Network::Testnet, + wallet_group_id: [0u8; 32], birth_height: 0, }); cs.core = Some(CoreChangeSet { diff --git a/packages/rs-platform-wallet-storage/tests/sqlite_delete_real_apply_failure.rs b/packages/rs-platform-wallet-storage/tests/sqlite_delete_real_apply_failure.rs index 7bc531f2fd..a0d0cf8a25 100644 --- a/packages/rs-platform-wallet-storage/tests/sqlite_delete_real_apply_failure.rs +++ b/packages/rs-platform-wallet-storage/tests/sqlite_delete_real_apply_failure.rs @@ -22,6 +22,7 @@ fn full_changeset(synced: u32) -> PlatformWalletChangeSet { let mut cs = PlatformWalletChangeSet::default(); cs.wallet_metadata = Some(WalletMetadataEntry { network: Network::Testnet, + wallet_group_id: [0u8; 32], birth_height: 0, }); cs.core = Some(CoreChangeSet { diff --git a/packages/rs-platform-wallet-storage/tests/sqlite_error_classification.rs b/packages/rs-platform-wallet-storage/tests/sqlite_error_classification.rs index 21e909fdf0..9c1e5fdb0d 100644 --- a/packages/rs-platform-wallet-storage/tests/sqlite_error_classification.rs +++ b/packages/rs-platform-wallet-storage/tests/sqlite_error_classification.rs @@ -157,9 +157,6 @@ fn samples() -> Vec { WalletStorageError::InvalidWalletIdLength { actual: 10 }, WalletStorageError::ConfigInvalid { reason: "bad knob" }, WalletStorageError::IdentityEntryIdMismatch, - WalletStorageError::UtxoAddressNotDerived { - address: "yMockAddress".into(), - }, // BincodeEncode / BincodeDecode / HashDecode / ConsensusCodec // need real upstream errors; omitted but covered by their arms. WalletStorageError::BlobDecode { @@ -245,7 +242,6 @@ fn tc_p2_005_is_transient_table() { (false, "asset_lock_entry_mismatch") } WalletStorageError::BlobTooLarge { .. } => (false, "blob_too_large"), - WalletStorageError::UtxoAddressNotDerived { .. } => (false, "utxo_address_not_derived"), WalletStorageError::ForeignKeysNotEnforced => (false, "foreign_keys_not_enforced"), WalletStorageError::JournalModeNotApplied { .. } => (false, "journal_mode_not_applied"), WalletStorageError::SchemaHistoryMalformed { .. } => { diff --git a/packages/rs-platform-wallet-storage/tests/sqlite_fk_changeset_ordering.rs b/packages/rs-platform-wallet-storage/tests/sqlite_fk_changeset_ordering.rs index 546cf59d64..867a6f9ce3 100644 --- a/packages/rs-platform-wallet-storage/tests/sqlite_fk_changeset_ordering.rs +++ b/packages/rs-platform-wallet-storage/tests/sqlite_fk_changeset_ordering.rs @@ -249,6 +249,7 @@ fn wallets_anchor_and_children_in_same_changeset_commits() { PlatformWalletChangeSet { wallet_metadata: Some(WalletMetadataEntry { network: Network::Testnet, + wallet_group_id: [0u8; 32], birth_height: 0, }), identities: Some(identities_changeset(identity, Some(w))), diff --git a/packages/rs-platform-wallet-storage/tests/sqlite_load_wiring.rs b/packages/rs-platform-wallet-storage/tests/sqlite_load_wiring.rs index e5aa4b65ee..ff4d91744f 100644 --- a/packages/rs-platform-wallet-storage/tests/sqlite_load_wiring.rs +++ b/packages/rs-platform-wallet-storage/tests/sqlite_load_wiring.rs @@ -22,31 +22,6 @@ fn reopen(path: &std::path::Path) -> SqlitePersister { SqlitePersister::open(SqlitePersisterConfig::new(path)).expect("reopen") } -/// The `core_derived_addresses` row a real scan records before a UTXO on -/// `address` lands. The strict UTXO writer refuses an unspent UTXO whose -/// address was never derived, so a stored UTXO must carry its matching -/// derivation. The writer keys its lookup on `(wallet_id, address)` only -/// and the read side re-attributes by topology, so the account fields -/// here are inert placeholders — the address is the load-bearing field. -fn derived_for(address: &dashcore::Address) -> platform_wallet::DerivedAddress { - // Compressed secp256k1 generator point — a valid placeholder pubkey. - const PUBKEY_G: [u8; 33] = [ - 0x02, 0x79, 0xbe, 0x66, 0x7e, 0xf9, 0xdc, 0xbb, 0xac, 0x55, 0xa0, 0x62, 0x95, 0xce, 0x87, - 0x0b, 0x07, 0x02, 0x9b, 0xfc, 0xdb, 0x2d, 0xce, 0x28, 0xd9, 0x59, 0xf2, 0x81, 0x5b, 0x16, - 0xf8, 0x17, 0x98, - ]; - platform_wallet::DerivedAddress { - account_type: key_wallet::account::AccountType::Standard { - index: 0, - standard_account_type: key_wallet::account::StandardAccountType::BIP44Account, - }, - pool_type: key_wallet::managed_account::address_pool::AddressPoolType::External, - derivation_index: 0, - address: address.clone(), - public_key: dashcore::PublicKey::from_slice(&PUBKEY_G).expect("valid compressed pubkey"), - } -} - /// A registered wallet with UTXOs round-trips into the keyless `wallets` /// payload — manifest, network, birth height, core state. #[test] @@ -80,6 +55,7 @@ fn c1_load_populates_keyless_wallet_payload() { let reg = PlatformWalletChangeSet { wallet_metadata: Some(WalletMetadataEntry { network: key_wallet::Network::Testnet, + wallet_group_id: [0u8; 32], birth_height: 7, }), account_registrations: manifest.clone(), @@ -113,8 +89,7 @@ fn c1_load_populates_keyless_wallet_payload() { w, PlatformWalletChangeSet { core: Some(CoreChangeSet { - addresses_derived: vec![derived_for(&utxo.address)], - new_utxos: vec![utxo], + new_utxos: vec![utxo.clone()], last_processed_height: Some(50), synced_height: Some(50), ..Default::default() diff --git a/packages/rs-platform-wallet-storage/tests/sqlite_migrations.rs b/packages/rs-platform-wallet-storage/tests/sqlite_migrations.rs index 628d790c09..fb402eeb2f 100644 --- a/packages/rs-platform-wallet-storage/tests/sqlite_migrations.rs +++ b/packages/rs-platform-wallet-storage/tests/sqlite_migrations.rs @@ -86,12 +86,7 @@ fn tc027_smoke_insert_every_table() { // Labels must match the writer-side canonical strings — see the // CHECK constraint sourced from `ACCOUNT_TYPE_LABELS` in // `sqlite::schema::accounts`. - "INSERT INTO account_registrations (wallet_id, account_type, account_index, account_xpub_bytes) VALUES (?1, 'standard', 0, X'00')", - &[&wallet_id.as_slice()], - ), - ( - "account_address_pools", - "INSERT INTO account_address_pools (wallet_id, account_type, account_index, pool_type, snapshot_blob) VALUES (?1, 'standard', 0, 'external', X'00')", + "INSERT INTO account_registrations (wallet_id, account_type, account_index, account_xpub_bytes) VALUES (?1, 'standard_bip44', 0, X'00')", &[&wallet_id.as_slice()], ), ( @@ -109,11 +104,6 @@ fn tc027_smoke_insert_every_table() { "INSERT INTO core_instant_locks (wallet_id, txid, islock_blob) VALUES (?1, ?2, X'00')", &[&wallet_id.as_slice(), &txid], ), - ( - "core_derived_addresses", - "INSERT INTO core_derived_addresses (wallet_id, account_type, account_index, address, derivation_path, used) VALUES (?1, 'standard', 0, 'addr', '', 0)", - &[&wallet_id.as_slice()], - ), ( "core_sync_state", "INSERT INTO core_sync_state (wallet_id, last_processed_height, synced_height) VALUES (?1, NULL, NULL)", diff --git a/packages/rs-platform-wallet-storage/tests/sqlite_object_metadata.rs b/packages/rs-platform-wallet-storage/tests/sqlite_object_metadata.rs index a2f748545d..1efd76e224 100644 --- a/packages/rs-platform-wallet-storage/tests/sqlite_object_metadata.rs +++ b/packages/rs-platform-wallet-storage/tests/sqlite_object_metadata.rs @@ -1016,12 +1016,10 @@ fn delete_wallet_leaves_no_surviving_rows() { let txid = vec![0x01u8; 32]; let outpoint = vec![0x02u8; 36]; let stmts: &[(&str, &[&dyn rusqlite::ToSql])] = &[ - ("INSERT INTO account_registrations (wallet_id, account_type, account_index, account_xpub_bytes) VALUES (?1, 'standard', 0, X'00')", &[&a.as_slice()]), - ("INSERT INTO account_address_pools (wallet_id, account_type, account_index, pool_type, snapshot_blob) VALUES (?1, 'standard', 0, 'external', X'00')", &[&a.as_slice()]), + ("INSERT INTO account_registrations (wallet_id, account_type, account_index, account_xpub_bytes) VALUES (?1, 'standard_bip44', 0, X'00')", &[&a.as_slice()]), ("INSERT INTO core_transactions (wallet_id, txid, finalized, record_blob) VALUES (?1, ?2, 0, X'00')", &[&a.as_slice(), &txid]), ("INSERT INTO core_utxos (wallet_id, outpoint, value, script, account_index, spent) VALUES (?1, ?2, 0, X'00', 0, 0)", &[&a.as_slice(), &outpoint]), ("INSERT INTO core_instant_locks (wallet_id, txid, islock_blob) VALUES (?1, ?2, X'00')", &[&a.as_slice(), &txid]), - ("INSERT INTO core_derived_addresses (wallet_id, account_type, account_index, address, derivation_path, used) VALUES (?1, 'standard', 0, 'addr', '', 0)", &[&a.as_slice()]), ("INSERT INTO core_sync_state (wallet_id, last_processed_height, synced_height) VALUES (?1, 1, 1)", &[&a.as_slice()]), ("INSERT INTO identity_keys (wallet_id, identity_id, key_id, public_key_blob, public_key_hash, derivation_blob) VALUES (?1, ?2, 0, X'00', X'00', NULL)", &[&a.as_slice(), &idy.as_slice()]), ("INSERT INTO platform_address_sync (wallet_id, sync_height, sync_timestamp, last_known_recent_block) VALUES (?1, 0, 0, 0)", &[&a.as_slice()]), @@ -1188,11 +1186,9 @@ fn delete_wallet_leaves_no_surviving_rows() { let wallet_scoped = [ "wallets", "account_registrations", - "account_address_pools", "core_transactions", "core_utxos", "core_instant_locks", - "core_derived_addresses", "core_sync_state", "identities", "contacts", diff --git a/packages/rs-platform-wallet-storage/tests/sqlite_structural_hardening.rs b/packages/rs-platform-wallet-storage/tests/sqlite_structural_hardening.rs index 4c6d6ed4fc..9c79b36baf 100644 --- a/packages/rs-platform-wallet-storage/tests/sqlite_structural_hardening.rs +++ b/packages/rs-platform-wallet-storage/tests/sqlite_structural_hardening.rs @@ -116,38 +116,27 @@ fn make_utxo(addr: &Address, vout: u32, value: u64) -> Utxo { Utxo::new(outpoint, txout, addr.clone(), 10, false) } -/// UTXOs resolve their real `account_index` from the derived-address -/// map written earlier in the same transaction, instead of a hardcoded -/// 0. +/// Every `core_utxos` row is written with the hardcoded default +/// `account_index = 0` (the product uses only the default account; a +/// non-default account causes `core_bridge::warn_if_non_default_account` +/// to log a `warn!` but still persists the record under index 0 to avoid +/// fund loss), so the per-account grouping reader buckets every unspent +/// UTXO — at any address — under account 0. #[test] -fn multi_account_utxos_bucket_to_real_account() { +fn utxos_bucket_under_default_account_index_zero() { use platform_wallet_storage::sqlite::schema::core_state; let (persister, _tmp, _path) = fresh_persister(); let w: WalletId = wid(0xC7); ensure_wallet_meta(&persister, &w); - let addr_acct5 = p2pkh(0x05); - let addr_acct9 = p2pkh(0x09); + let addr_a = p2pkh(0x05); + let addr_b = p2pkh(0x09); { let mut conn = persister.lock_conn_for_test(); - // Pre-seed the derived-address map with two distinct accounts. - for (acct, addr) in [(5u32, &addr_acct5), (9u32, &addr_acct9)] { - conn.execute( - "INSERT INTO core_derived_addresses \ - (wallet_id, account_type, account_index, address, derivation_path, used) \ - VALUES (?1, 'standard', ?2, ?3, '0/0', 0)", - params![w.as_slice(), acct as i64, addr.to_string()], - ) - .unwrap(); - } - let cs = CoreChangeSet { - new_utxos: vec![ - make_utxo(&addr_acct5, 0, 1000), - make_utxo(&addr_acct9, 1, 2000), - ], + new_utxos: vec![make_utxo(&addr_a, 0, 1000), make_utxo(&addr_b, 1, 2000)], ..Default::default() }; let tx = conn.transaction().unwrap(); @@ -158,47 +147,20 @@ fn multi_account_utxos_bucket_to_real_account() { let conn = persister.lock_conn_for_test(); let by_account = core_state::list_unspent_utxos(&conn, &w).unwrap(); assert_eq!( - by_account.get(&5).map(|v| v.len()), - Some(1), - "account 5 should hold exactly one UTXO" + by_account.len(), + 1, + "all UTXOs bucket under a single (default) account" ); assert_eq!( - by_account.get(&9).map(|v| v.len()), - Some(1), - "account 9 should hold exactly one UTXO" - ); -} - -/// A NEW unspent UTXO whose address is absent from -/// `core_derived_addresses` cannot resolve an owning account, so the -/// write is refused with the typed `UtxoAddressNotDerived` instead of -/// silently mis-filing live funds under account 0. -#[test] -fn unspent_utxo_on_undeclared_address_is_rejected() { - use platform_wallet_storage::sqlite::schema::core_state; - - let (persister, _tmp, _path) = fresh_persister(); - let w: WalletId = wid(0xC8); - ensure_wallet_meta(&persister, &w); - - let addr_unknown = p2pkh(0xEE); - let mut conn = persister.lock_conn_for_test(); - let cs = CoreChangeSet { - new_utxos: vec![make_utxo(&addr_unknown, 0, 3000)], - ..Default::default() - }; - let tx = conn.transaction().unwrap(); - let err = core_state::apply(&tx, &w, &cs) - .expect_err("unspent UTXO on an undeclared address must error"); - assert!( - matches!(err, WalletStorageError::UtxoAddressNotDerived { .. }), - "expected UtxoAddressNotDerived, got {err:?}" + by_account.get(&0).map(|v| v.len()), + Some(2), + "both UTXOs are attributed to the default account (index 0)" ); } -/// A spent-only placeholder UTXO whose address was never derived still -/// persists with the account-0 fallback — spent rows are excluded from -/// the unspent set, so the placeholder index is inert. +/// A spent-only placeholder UTXO (no prior unspent row to mark) persists +/// with the hardcoded account 0 — spent rows are excluded from the unspent +/// set, so the index is inert. #[test] fn spent_only_utxo_on_undeclared_address_uses_zero_fallback() { use platform_wallet_storage::sqlite::schema::core_state; diff --git a/packages/rs-platform-wallet-storage/tests/sqlite_wallet_db_identity.rs b/packages/rs-platform-wallet-storage/tests/sqlite_wallet_db_identity.rs index 61ea6b8fc3..ae3ef2ce92 100644 --- a/packages/rs-platform-wallet-storage/tests/sqlite_wallet_db_identity.rs +++ b/packages/rs-platform-wallet-storage/tests/sqlite_wallet_db_identity.rs @@ -60,6 +60,7 @@ fn fresh_wallet_db() -> (tempfile::TempDir, std::path::PathBuf) { let mut cs = PlatformWalletChangeSet::default(); cs.wallet_metadata = Some(WalletMetadataEntry { network: key_wallet::Network::Testnet, + wallet_group_id: [0u8; 32], birth_height: 0, }); cs.core = Some(CoreChangeSet { diff --git a/packages/rs-platform-wallet/src/changeset/changeset.rs b/packages/rs-platform-wallet/src/changeset/changeset.rs index b4c18917c4..cc8b705df9 100644 --- a/packages/rs-platform-wallet/src/changeset/changeset.rs +++ b/packages/rs-platform-wallet/src/changeset/changeset.rs @@ -963,8 +963,12 @@ pub struct PlatformWalletChangeSet { /// the merge policy (plain `Vec::extend`, dedup is the apply-side /// caller's job). pub account_registrations: Vec, - /// Address-pool snapshots emitted at wallet create (initial - /// gap-limit population) and on any pool extension / "used" flip. + /// Full address-pool snapshots: emitted once at wallet registration. + /// Incremental derivations are delivered via `core.addresses_derived` + /// (the `WalletEvent` bus / FFI path); no per-block in-band pool + /// snapshot is written. The storage persister intentionally ignores this + /// field (UTXO attribution is hardcoded to account 0); non-storage + /// consumers (e.g. the iOS FFI address registry) may still read it. /// See [`AccountAddressPoolEntry`] for the merge policy. pub account_address_pools: Vec, /// Shielded sub-wallet deltas: per-subwallet decrypted notes, diff --git a/packages/rs-platform-wallet/src/changeset/core_bridge.rs b/packages/rs-platform-wallet/src/changeset/core_bridge.rs index 46945667ef..a4f2cb32c3 100644 --- a/packages/rs-platform-wallet/src/changeset/core_bridge.rs +++ b/packages/rs-platform-wallet/src/changeset/core_bridge.rs @@ -41,6 +41,34 @@ use crate::changeset::changeset::{CoreChangeSet, PlatformWalletChangeSet}; use crate::changeset::traits::PlatformWalletPersistence; use crate::wallet::platform_wallet::PlatformWalletInfo; +/// Single-account observation. The storage writer hardcodes +/// `core_utxos.account_index = 0` (the product uses only the default +/// account, and that column drives only cosmetic per-account grouping). A +/// UTXO-bearing record owned by a non-default funds account is STILL +/// persisted under index 0 — never skipped, because skipping it would +/// undercount the wallet balance and lose funds. We only `warn!` so the +/// approximate grouping is visible. Identity/provider account types carry +/// no funds index (`AccountType::index() == None`) and never emit +/// `Received`/`Change` UTXOs, so they never warn. +fn warn_if_non_default_account(record: &TransactionRecord) { + if let Some(index) = non_default_account_index(record) { + tracing::warn!( + account_index = index, + txid = %record.txid, + "non-default account UTXO persisted under account_index 0; \ + per-account grouping is approximate" + ); + } +} + +/// The record's funds account index when it is a *non-default* (index != 0) +/// funds account, else `None`. Identity/provider account types carry no +/// funds index (`index() == None`) and never emit `Received`/`Change` +/// UTXOs, so they yield `None`. +fn non_default_account_index(record: &TransactionRecord) -> Option { + record.account_type.index().filter(|&index| index != 0) +} + /// Spawn the wallet-event subscriber task. /// /// Subscribes to `wallet_manager.subscribe_events()` from inside the @@ -129,18 +157,17 @@ async fn build_core_changeset( addresses_derived, .. } => { + // Persist regardless of account; warn on a non-default account. + warn_if_non_default_account(record); // Derive UTXO deltas before moving the record into `records` // so the per-record borrows are still live. CoreChangeSet { new_utxos: derive_new_utxos(record), spent_utxos: derive_spent_utxos(record), records: vec![(**record).clone()], - // Mirror the upstream-emitted derived addresses - // through to the persister so newly-extended pool - // rows are written transactionally with the tx that - // triggered the extension. See - // `CoreChangeSet.addresses_derived` for the cascade- - // link rationale. + // Forward the upstream-emitted derived addresses to the + // persister; the FFI layer feeds the iOS address registry + // from this delta. See `CoreChangeSet.addresses_derived`. addresses_derived: addresses_derived.clone(), ..CoreChangeSet::default() } @@ -171,11 +198,28 @@ async fn build_core_changeset( .. } => { let mut cs = CoreChangeSet::default(); - // Inserted records bring fresh UTXOs and may consume previous ones. + // Inserted records bring fresh UTXOs and may consume previous + // ones — always project. Non-default-account records are tallied + // and surfaced in a single aggregated warn after the loop (rather + // than one warn per record) to keep a busy block quiet. + let mut non_default_count = 0usize; + let mut non_default_sample: Option = None; for r in inserted { + if non_default_account_index(r).is_some() { + non_default_count += 1; + non_default_sample.get_or_insert(r.txid); + } cs.new_utxos.extend(derive_new_utxos(r)); cs.spent_utxos.extend(derive_spent_utxos(r)); } + if non_default_count > 0 { + tracing::warn!( + non_default_count, + sample_txid = ?non_default_sample, + "non-default account UTXO(s) persisted under account_index 0; \ + per-account grouping is approximate" + ); + } // Updated records (re-confirmation, IS-lock applied to a known // mempool tx, etc.) don't usually change UTXO topology — the // record's content does change though, so re-emit it. @@ -357,3 +401,120 @@ impl CoreChangeSet { && self.addresses_derived.is_empty() } } + +#[cfg(test)] +mod tests { + use std::collections::BTreeMap; + + use dashcore::blockdata::transaction::Transaction; + use dashcore::hashes::Hash; + use key_wallet::account::{AccountType, StandardAccountType}; + use key_wallet::managed_account::transaction_record::{ + OutputDetail, TransactionDirection, TransactionRecord, + }; + use key_wallet::transaction_checking::{BlockInfo, TransactionContext, TransactionType}; + use key_wallet::WalletCoreBalance; + + use super::*; + + fn standard(index: u32) -> AccountType { + AccountType::Standard { + index, + standard_account_type: StandardAccountType::BIP44Account, + } + } + + /// A throwaway testnet P2PKH address keyed off `seed`. + fn p2pkh(seed: u8) -> dashcore::Address { + use dashcore::address::Payload; + use dashcore::PubkeyHash; + dashcore::Address::new( + dashcore::Network::Testnet, + Payload::PubkeyHash(PubkeyHash::from_byte_array([seed; 20])), + ) + } + + /// A confirmed `TransactionRecord` owned by `account_type` carrying a + /// single `Received` output worth `value` at `addr`, so + /// `derive_new_utxos` yields exactly one UTXO. + fn record_with_received_output( + account_type: AccountType, + addr: &dashcore::Address, + value: u64, + ) -> TransactionRecord { + let tx = Transaction { + version: 3, + lock_time: 0, + input: vec![], + output: vec![dashcore::TxOut { + value, + script_pubkey: addr.script_pubkey(), + }], + special_transaction_payload: None, + }; + TransactionRecord::new( + tx, + account_type, + TransactionContext::InChainLockedBlock(BlockInfo::new( + 42, + dashcore::BlockHash::from_byte_array([3u8; 32]), + 1_735_689_600, + )), + TransactionType::Standard, + TransactionDirection::Incoming, + Vec::new(), + vec![OutputDetail { + index: 0, + role: OutputRole::Received, + address: Some(addr.clone()), + value, + }], + value as i64, + ) + } + + /// Project a `TransactionDetected` for `record` through the real bridge + /// path. `balance`/`account_balances` are unused by the projection. + async fn changeset_for(record: TransactionRecord) -> CoreChangeSet { + let wm = Arc::new(RwLock::new(WalletManager::::new( + key_wallet::Network::Testnet, + ))); + let event = WalletEvent::TransactionDetected { + wallet_id: [0u8; 32], + record: Box::new(record), + balance: WalletCoreBalance::default(), + account_balances: BTreeMap::new(), + addresses_derived: Vec::new(), + }; + build_core_changeset(&wm, &event).await + } + + /// A default-account (index 0) UTXO is projected into the changeset. + #[tokio::test] + async fn default_account_utxo_persists() { + let addr = p2pkh(0x11); + let cs = changeset_for(record_with_received_output(standard(0), &addr, 500_000)).await; + assert_eq!( + cs.new_utxos.len(), + 1, + "the default-account UTXO must be projected" + ); + assert_eq!(cs.new_utxos[0].value(), 500_000); + } + + /// REGRESSION (fund-loss): a non-default-account (index != 0) UTXO is + /// STILL projected — never dropped. Storage persists it under + /// `account_index 0`; the only cost is approximate per-account grouping + /// (a `warn!` is logged). Dropping it would undercount the balance. + #[tokio::test] + async fn non_default_account_utxo_persists_under_zero() { + let addr = p2pkh(0x22); + let cs = changeset_for(record_with_received_output(standard(7), &addr, 900_000)).await; + assert_eq!( + cs.new_utxos.len(), + 1, + "a non-default-account UTXO must NOT be dropped" + ); + assert_eq!(cs.new_utxos[0].value(), 900_000, "funds preserved"); + } +} diff --git a/packages/rs-platform-wallet/src/error.rs b/packages/rs-platform-wallet/src/error.rs index b6217c9839..9514f2db54 100644 --- a/packages/rs-platform-wallet/src/error.rs +++ b/packages/rs-platform-wallet/src/error.rs @@ -26,6 +26,22 @@ pub enum PlatformWalletError { utxo_count: usize, }, + /// The deep-index discovery probes did not mirror the account's real + /// address pools 1:1 during rehydration, so applying probe depths by + /// position would index the wrong pool. Fail-closed instead of risking + /// a misattributed derivation — the probes are built directly from the + /// same `address_pools()` enumeration, so a mismatch is a structural + /// invariant break, not user-reachable. + #[error( + "rehydration pool/probe mismatch: expected {expected} address pool(s) to mirror the discovery probes, found {found}" + )] + RehydrationPoolMismatch { + /// Number of discovery probes built from `address_pools()`. + expected: usize, + /// Number of real address pools from `address_pools_mut()`. + found: usize, + }, + #[error("Wallet not found: {0}")] WalletNotFound(String), diff --git a/packages/rs-platform-wallet/src/manager/load.rs b/packages/rs-platform-wallet/src/manager/load.rs index 1d8baf163b..6f4c118b03 100644 --- a/packages/rs-platform-wallet/src/manager/load.rs +++ b/packages/rs-platform-wallet/src/manager/load.rs @@ -120,9 +120,11 @@ impl PlatformWalletManager

{ // UTXOs but no funds account hard-fails here rather than // reconstructing a silent zero balance. let mut wallet_info = ManagedWalletInfo::from_wallet(&wallet, birth_height); - if let Err(e) = - super::rehydrate::apply_persisted_core_state(&mut wallet_info, &core_state) - { + if let Err(e) = super::rehydrate::apply_persisted_core_state( + &mut wallet_info, + &account_manifest, + &core_state, + ) { load_error = Some(e); break 'load; } diff --git a/packages/rs-platform-wallet/src/manager/platform_address_sync.rs b/packages/rs-platform-wallet/src/manager/platform_address_sync.rs index e1a229806c..30885258ba 100644 --- a/packages/rs-platform-wallet/src/manager/platform_address_sync.rs +++ b/packages/rs-platform-wallet/src/manager/platform_address_sync.rs @@ -97,6 +97,14 @@ pub struct PlatformAddressSyncManager { event_manager: Arc, /// Cancel token for the background loop, if running. background_cancel: StdMutex>, + /// Monotonically increasing generation counter. Bumped on every + /// `start()` so the exiting thread can tell whether its + /// generation is still the active one before clearing + /// `background_cancel`. Without this, a `stop()` → `start()` + /// overlap lets the prior thread's cleanup strip the new + /// generation's token, leaving the new loop running but + /// untrackable via `is_running()`. + background_generation: AtomicU64, interval_secs: AtomicU64, is_syncing: AtomicBool, /// Set by [`quiesce`](Self::quiesce) to gate new passes while it @@ -125,6 +133,7 @@ impl PlatformAddressSyncManager { wallets, event_manager, background_cancel: StdMutex::new(None), + background_generation: AtomicU64::new(0), interval_secs: AtomicU64::new(DEFAULT_SYNC_INTERVAL_SECS), is_syncing: AtomicBool::new(false), quiescing: AtomicBool::new(false), @@ -201,6 +210,7 @@ impl PlatformAddressSyncManager { } let cancel = CancellationToken::new(); *guard = Some(cancel.clone()); + let my_gen = self.background_generation.fetch_add(1, Ordering::AcqRel) + 1; drop(guard); let handle = tokio::runtime::Handle::current(); @@ -223,8 +233,17 @@ impl PlatformAddressSyncManager { } } + // Clear `background_cancel` only if the active + // generation is still ours. Acquire the lock + // FIRST, then check — `start()` bumps the + // generation while holding this same lock, so + // once we hold it the generation is final w.r.t. + // any concurrent token swap (no TOCTOU between + // the check and the clear). if let Ok(mut guard) = this.background_cancel.lock() { - *guard = None; + if this.background_generation.load(Ordering::Acquire) == my_gen { + *guard = None; + } } }); }) @@ -505,4 +524,56 @@ mod tests { assert_eq!(counter.completions.load(AtomicOrdering::SeqCst), 0); assert!(!mgr.is_syncing()); } + + /// Restart-in-place regression for the generation guard: a tight + /// `start()` → `stop()` → `start()` must leave the manager *running* + /// on the new generation. The cancelled gen-1 loop races to clear + /// `background_cancel` as it exits; the generation guard must stop it + /// from stripping gen-2's freshly installed token — otherwise the new + /// loop keeps running but becomes invisible to `is_running()` / + /// `stop()`. + /// + /// Determinism: the only wait is a *bounded* poll. With the guard in + /// place `is_running()` is true for the whole window, so the test + /// never fails spuriously on correct code. A regression flips it false + /// within milliseconds once the stale loop clears the slot, which the + /// poll catches. Needs the multi-thread flavor because `start()` + /// drives its loop via `Handle::current().block_on` on a dedicated OS + /// thread, which would deadlock a single-threaded test runtime. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn restart_in_place_keeps_running_after_stale_loop_exits() { + let (mgr, _counter) = make_manager(); + + // Gen 1. Wait (bounded) for the first pass to land — a real + // lifecycle signal that the loop is now parked in its interval + // sleep, so its cleanup is still pending when we stop+restart. + Arc::clone(&mgr).start(); + let mut waited = 0; + while mgr.last_sync_unix_seconds().is_none() { + assert!(waited < 200, "gen-1's first sync pass never completed"); + tokio::time::sleep(Duration::from_millis(10)).await; + waited += 1; + } + + // Tight stop→start with no await between: the just-cancelled gen-1 + // loop cannot reach its cleanup before gen 2 is installed, so the + // race window the guard protects is reliably open. + mgr.stop(); + Arc::clone(&mgr).start(); + + // Give the stale gen-1 loop ample time to run its (guarded) + // cleanup. `is_running()` must stay true throughout. + for _ in 0..100 { + assert!( + mgr.is_running(), + "stale gen-1 loop cleared gen-2's cancel token — generation guard regressed" + ); + tokio::time::sleep(Duration::from_millis(10)).await; + } + + // The surviving loop is the tracked one: a single `stop()` fully + // reflects it, so there is no orphaned unreflectable duplicate. + mgr.stop(); + assert!(!mgr.is_running(), "stop() must reflect the live loop"); + } } diff --git a/packages/rs-platform-wallet/src/manager/rehydrate.rs b/packages/rs-platform-wallet/src/manager/rehydrate.rs index d5fca89a20..44527d1ee2 100644 --- a/packages/rs-platform-wallet/src/manager/rehydrate.rs +++ b/packages/rs-platform-wallet/src/manager/rehydrate.rs @@ -89,6 +89,18 @@ pub(super) fn build_watch_only_wallet( /// Apply the keyless persisted core-state projection onto a /// freshly-minted `ManagedWalletInfo` skeleton. /// +/// # Parameters +/// +/// - `wallet_info`: the skeleton to hydrate in place. +/// - `manifest`: keyless account manifest (one entry per registered +/// account). Each entry carries an `account_type` → `account_xpub` +/// mapping used by [`extend_pools_for_restored_utxos`] to derive +/// addresses for restored UTXOs. If an account's `account_type` is +/// absent from the manifest, deep-index derivation is skipped for that +/// account (no xpub → no derivation possible); already-derived in-window +/// addresses are still marked used. +/// - `core`: the persisted core-state changeset to apply. +/// /// # Reconstructed (safety-critical-correct) /// /// - **Wallet balance** (`wallet_info.balance`, the no-silent-zero @@ -99,6 +111,10 @@ pub(super) fn build_watch_only_wallet( /// - **UTXO set**: every unspent persisted outpoint is restored into a /// funds-bearing account of the wallet (whatever topology it has — /// BIP44, BIP32, CoinJoin, DashPay). +/// - **Address-pool depth**: each pool is forward-derived to cover +/// restored UTXOs at deep derivation indices, then the gap window is +/// refilled beyond the deepest restored index so the per-address view +/// reconciles with the wallet total. /// - **Sync watermarks**: `synced_height` / `last_processed_height`. /// /// # Deferred to the first post-load `sync` (safe re-warm) @@ -109,6 +125,20 @@ pub(super) fn build_watch_only_wallet( /// first funds-bearing account and re-attributed on the next scan. /// The *wallet total* is unaffected (it is a sum across all funds /// accounts). +/// - **Deep-index address visibility**: each chain's pool scan stops +/// after [`MAX_REHYDRATION_DERIVATION_INDEX`] or after `gap_limit` +/// consecutive non-matching indices past the deepest resolved index. +/// The horizon only advances when an unspent UTXO anchors a match, so a +/// UTXO address can be left unresolved in two distinct cases: (1) it is +/// genuinely foreign (a different account's key routed here, or corrupt), +/// and (2) it is a *legitimately-owned but deep-and-sparse* address — +/// owned by this account, yet sitting past the first `gap_limit` window +/// with no nearer unspent UTXO to walk the horizon out to it. Both cases +/// are counted and logged via `tracing::warn!` and re-warm on the next +/// full sync. The wallet *total* stays exact (every UTXO is summed +/// regardless of pool visibility); only the per-address view is +/// incomplete until that sync. This is the accepted behavior of the +/// horizon-walk algorithm — see [`extend_pools_for_restored_utxos`]. /// - **`last_applied_chain_lock`**: not a persisted column (V001) and /// never written by the core-state writer; always `None` from disk. /// SPV re-applies a fresh chainlock on the first post-restart sync. @@ -127,13 +157,18 @@ pub(super) fn build_watch_only_wallet( /// than reconstructing a silent zero balance (the no-silent-zero /// mandate). An empty UTXO set is always `Ok`. /// -/// This never logs and never touches key material. +/// This never touches key material. pub fn apply_persisted_core_state( wallet_info: &mut ManagedWalletInfo, + manifest: &[AccountRegistrationEntry], core: &crate::changeset::CoreChangeSet, ) -> Result<(), PlatformWalletError> { use key_wallet::wallet::managed_wallet_info::wallet_info_interface::WalletInfoInterface; + // Captured before the mutable account borrow below so it can flow into + // pool-extension diagnostics without re-borrowing `wallet_info`. + let wallet_id = wallet_info.wallet_id; + // Sync watermarks first so `update_balance`'s maturity check sees // the restored tip. if let Some(h) = core.last_processed_height { @@ -164,13 +199,16 @@ pub fn apply_persisted_core_state( .next() { Some(account) => { - for utxo in unspent { - account.utxos.insert(utxo.outpoint, utxo.clone()); + for utxo in &unspent { + account.utxos.insert(utxo.outpoint, (*utxo).clone()); } + // Eager derivation covers only `0..=gap_limit`; extend each + // chain to cover restored UTXOs at deeper indices. + extend_pools_for_restored_utxos(account, manifest, &unspent, wallet_id)?; } None => { return Err(PlatformWalletError::RehydrationTopologyUnsupported { - wallet_id: wallet_info.wallet_id, + wallet_id, utxo_count: core.new_utxos.len(), }); } @@ -184,6 +222,272 @@ pub fn apply_persisted_core_state( Ok(()) } +/// Upper bound on forward derivation while resolving a restored UTXO +/// address to its derivation index. Addresses that don't resolve within +/// this many indices (e.g. they belong to a different funds account whose +/// UTXOs were routed here, or are corrupt) are left for the next full +/// rescan to re-warm — generous enough to cover any realistic per-account +/// derivation depth. The common (single funds account) path terminates at +/// the true high-water mark well before this and never reaches the cap. +const MAX_REHYDRATION_DERIVATION_INDEX: u32 = 10_000; + +/// Soft threshold past which a single chain's discovery scan is treated as +/// abnormally deep and worth a `tracing::warn!`. Real funds chains anchor +/// well below this; reaching it means either a corrupt / foreign-heavy UTXO +/// set walking the horizon out, or an approach toward the hard +/// [`MAX_REHYDRATION_DERIVATION_INDEX`] ceiling — both worth surfacing. +const REHYDRATION_DEEP_SCAN_WARN_INDEX: u32 = 1_000; + +/// Extend `account`'s address pools so every resolved UTXO address is +/// derived at its exact `(chain, index)` slot and marked used, then refill +/// the gap window beyond — following the sync path's `mark_used` → +/// `maintain_gap_limit` sequence. Each chain is scanned independently, +/// stopping once no unresolved address matches within a `gap_limit`-sized +/// window past the deepest resolved index; [`MAX_REHYDRATION_DERIVATION_INDEX`] +/// is the hard ceiling. Addresses that don't resolve from this account's +/// xpub — foreign keys, multi-account mismatch, or legitimately-owned but +/// deep-and-sparse slots with no nearer unspent UTXO to anchor the horizon — +/// are counted and logged via `tracing::warn!`; they re-warm on the next +/// full sync. Every restored address the pools *do* hold (in-window or +/// deep-resolved) is marked used so a funded address is never handed out as +/// a fresh receive address. +/// +/// Tested with Standard BIP44 topology (External + Internal pools) and +/// CoinJoin topology (single External pool). The per-chain probe loop has no +/// topology-specific branches, so the non-hardened single-pool type +/// (`Absent`) follows the same code path with a different relative derivation +/// path. `AbsentHardened` pools cannot be derived from a public xpub at all — +/// hardened child derivation needs the private key — so under watch-only +/// rehydration their addresses never resolve and always defer to the next +/// sync (shared code path, but the outcome is "unresolved"). +/// +/// # Errors +/// +/// [`PlatformWalletError::RehydrationPoolMismatch`] if the discovery probes +/// don't mirror the real pools 1:1 (a structural invariant break, not +/// user-reachable). Fail-closed rather than apply a probe depth to the wrong +/// pool by position. +/// +/// Never touches key material — the xpub is the keyless account public key. +fn extend_pools_for_restored_utxos( + account: &mut key_wallet::managed_account::ManagedCoreFundsAccount, + manifest: &[AccountRegistrationEntry], + restored: &[&key_wallet::Utxo], + wallet_id: [u8; 32], +) -> Result<(), PlatformWalletError> { + use key_wallet::managed_account::address_pool::{AddressPool, KeySource}; + use key_wallet::managed_account::managed_account_trait::ManagedAccountTrait; + use std::collections::{BTreeSet, HashSet}; + + let account_type = account.managed_account_type().to_account_type(); + + // The funds account carries no key material; recover its watch-only xpub + // from the keyless manifest by account type. Without it we cannot derive + // deeper, but can still mark already-derived (in-window) addresses used. + let key_source = manifest + .iter() + .find(|e| e.account_type == account_type) + .map(|e| KeySource::Public(e.account_xpub)); + + // Probe pools mirror each real pool's chain 1:1 so the index search + // derives into throwaway state (real pools keep their own exact depth) + // and the resolved depth can be applied back by position. Re-deriving + // each probe from index 0 is an accepted, bounded one-time-load cost + // (per chain capped at MAX_REHYDRATION_DERIVATION_INDEX); rehydration + // runs once per wallet at startup, never on a hot path. + let mut probes: Vec<(AddressPool, BTreeSet)> = account + .managed_account_type() + .address_pools() + .iter() + .map(|p| { + ( + AddressPool::new_without_generation( + p.base_path.clone(), + p.pool_type, + p.gap_limit, + p.network, + ), + BTreeSet::new(), + ) + }) + .collect(); + + // Deep-index discovery (requires the xpub): resolve restored addresses the + // eager derivation didn't already cover, recording the matching index per + // chain. Each chain advances independently and stops once no unresolved + // address resolves within gap_limit indices past its deepest match + // (preventing a full scan when the UTXO set carries foreign addresses); + // MAX_REHYDRATION_DERIVATION_INDEX is the hard ceiling regardless. + if let Some(key_source) = key_source.as_ref() { + let mut unresolved: HashSet = { + let pools = account.managed_account_type().address_pools(); + restored + .iter() + .map(|u| u.address.clone()) + .filter(|addr| !pools.iter().any(|p| p.contains_address(addr))) + .collect() + }; + + for (probe, matched) in probes.iter_mut() { + if unresolved.is_empty() { + break; + } + let chain_gap = probe.gap_limit; + let mut deepest_resolved: Option = None; + let mut index: u32 = 0; + + loop { + // Horizon: gap_limit past the deepest match, or the initial + // gap_limit window when nothing has resolved yet. + let horizon = deepest_resolved + .map(|d| d.saturating_add(chain_gap)) + .unwrap_or(chain_gap); + + if index > horizon || index > MAX_REHYDRATION_DERIVATION_INDEX { + break; + } + + if let Some(addr) = ensure_derived(probe, key_source, index) { + if unresolved.remove(&addr) { + matched.insert(index); + deepest_resolved = Some(index); + } + } + + if unresolved.is_empty() { + break; + } + + index = index.saturating_add(1); + } + + // Surface an abnormally deep scan once per chain (outside the loop + // — never log inside the per-index walk). + if index > REHYDRATION_DEEP_SCAN_WARN_INDEX { + tracing::warn!( + wallet_id = %hex::encode(wallet_id), + account_type = ?account_type, + pool_type = ?probe.pool_type, + deepest_resolved = ?deepest_resolved, + scanned_to = index.saturating_sub(1), + "rehydration: chain discovery scanned abnormally deep — \ + likely a foreign-heavy or sparse UTXO set" + ); + } + } + + // Still-unresolved addresses are either foreign (a different account's + // key routed here, or corrupt) or legitimately-owned but deep-and-sparse + // (past the first gap window with no nearer unspent UTXO to anchor the + // horizon). Either way they re-warm on the next full sync; the wallet + // total is exact regardless. + if !unresolved.is_empty() { + tracing::warn!( + wallet_id = %hex::encode(wallet_id), + account_type = ?account_type, + unresolved_count = unresolved.len(), + "rehydration: UTXO address(es) unresolved for this account xpub \ + — will re-warm on next sync; balance total is exact" + ); + } + } + + // No explicit aggregate-derivation cap is needed: a funds account exposes + // a fixed, small number of chains (Standard = 2, others = 1), each already + // capped at MAX_REHYDRATION_DERIVATION_INDEX, so total derivation is bounded + // by chains × MAX with no unbounded growth — an aggregate cap would either + // equal that natural bound (no-op) or clip a legitimate deep multi-chain + // wallet. The per-chain ceiling plus the deep-scan warn above are the + // proportionate guard against a corrupt/foreign-heavy UTXO set. + + // Apply discovered depths and mark restored addresses used. `probes` is + // built directly from `address_pools()`, so it mirrors `address_pools_mut()` + // 1:1 and in chain order; verify that invariant before zipping by position. + let mut pools = account.managed_account_type_mut().address_pools_mut(); + if pools.len() != probes.len() { + return Err(PlatformWalletError::RehydrationPoolMismatch { + expected: probes.len(), + found: pools.len(), + }); + } + for (pool, (probe, matched)) in pools.iter_mut().zip(probes.iter()) { + // `iter_mut()` over `Vec<&mut AddressPool>` yields `&mut &mut _`; + // reborrow once so the pool flows into `ensure_derived` cleanly. + let pool: &mut AddressPool = pool; + debug_assert_eq!( + pool.pool_type, probe.pool_type, + "probe/pool chain order must match for by-position depth apply" + ); + + // Derive up to the deepest discovered index so its address exists in + // the real pool before we mark it used. + if let Some(&deepest) = matched.iter().next_back() { + if let Some(key_source) = key_source.as_ref() { + if ensure_derived(pool, key_source, deepest).is_none() { + tracing::warn!( + wallet_id = %hex::encode(wallet_id), + account_type = ?account_type, + pool_type = ?pool.pool_type, + index = deepest, + "rehydration: failed to derive resolved index into pool; \ + deferring its address to the next sync" + ); + } + } + } + + // Mark every restored address this pool now holds as used — covers both + // deep-resolved addresses (just derived) and in-window addresses the + // discovery scan never visits. Without this an already-derived but + // funded address keeps `used = false` and could be handed out as a fresh + // receive address. `mark_used` is a no-op for addresses not in this + // pool, so an underived (foreign / sparse) index is never marked. + let mut marked_any = false; + for u in restored { + if pool.mark_used(&u.address) { + marked_any = true; + } + } + + // Refill the gap window past the deepest used index (needs the xpub). + if marked_any { + if let Some(key_source) = key_source.as_ref() { + if let Err(e) = pool.maintain_gap_limit(key_source) { + tracing::warn!( + wallet_id = %hex::encode(wallet_id), + account_type = ?account_type, + pool_type = ?pool.pool_type, + error = %e, + "rehydration: gap-limit maintenance failed; pool window \ + may be short until the next sync" + ); + } + } + } + } + Ok(()) +} + +/// Ensure `pool` has derived through `index` (generating only the missing +/// tail), and return that index's address. `None` only on a derivation +/// error. +fn ensure_derived( + pool: &mut key_wallet::managed_account::address_pool::AddressPool, + key_source: &key_wallet::managed_account::address_pool::KeySource, + index: u32, +) -> Option { + let needs_more = match pool.highest_generated { + Some(highest) => highest < index, + None => true, + }; + if needs_more { + let start = pool.highest_generated.map(|h| h + 1).unwrap_or(0); + pool.generate_addresses(index - start + 1, key_source, true) + .ok()?; + } + pool.address_at_index(index) +} + #[cfg(test)] mod tests { use super::*; @@ -235,4 +539,755 @@ mod tests { .expect_err("empty manifest must be MissingManifest"); assert!(matches!(err, RehydrateRowError::MissingManifest)); } + + /// Regression: after restart-in-place the watch-only pools eagerly + /// cover only `0..=gap_limit`, but persisted UTXOs can sit at deeper + /// derivation indices. Rehydration must extend each chain's pool to its + /// deepest restored index so the per-address view reconciles with the + /// wallet total instead of undercounting. + /// + /// Index layout (gap_limit = 30): + /// - external idx 3: within eager window (not in `unresolved`), balance included + /// - external idx 30: first index past eager window; anchors the initial scan + /// window and extends it to idx 60 + /// - external idx 50: within extended window (50 < 60), resolved + /// - internal idx 30: within initial scan window, resolved + /// + /// Standard BIP44 topology (External + Internal pools) is exercised. + /// Asserts that maintain_gap_limit fills beyond the deepest resolved. + #[test] + fn rehydration_extends_pools_to_cover_deep_index_utxos() { + use dashcore::blockdata::transaction::txout::TxOut; + use dashcore::{OutPoint, Txid}; + use key_wallet::bip32::DerivationPath; + use key_wallet::gap_limit::DEFAULT_EXTERNAL_GAP_LIMIT; + use key_wallet::managed_account::address_pool::{AddressPool, AddressPoolType, KeySource}; + use key_wallet::managed_account::managed_account_trait::ManagedAccountTrait; + use key_wallet::wallet::managed_wallet_info::ManagedWalletInfo; + use key_wallet::{Address, Utxo}; + use std::collections::HashSet; + + let seed = [7u8; 64]; + let wallet = Wallet::from_seed_bytes( + seed, + Network::Testnet, + WalletAccountCreationOptions::Default, + ) + .unwrap(); + let manifest = manifest_for(&wallet); + + // Mint the watch-only skeleton (pools cover only the eager gap + // window) and resolve the first funds account's keyless xpub. + let mut wallet_info = ManagedWalletInfo::from_wallet(&wallet, 1); + let funds_type = wallet_info + .accounts + .all_funding_accounts() + .first() + .unwrap() + .managed_account_type() + .to_account_type(); + let xpub = manifest + .iter() + .find(|e| e.account_type == funds_type) + .map(|e| e.account_xpub) + .expect("funds account xpub"); + + // Derive addresses on each chain from the same account xpub the + // pools use; `base_path` is record-keeping only and does not affect + // the derived address, so DerivationPath::master() is fine here. + let derive = |pool_type, index: u32| -> Address { + let mut p = AddressPool::new_without_generation( + DerivationPath::master(), + pool_type, + DEFAULT_EXTERNAL_GAP_LIMIT, + Network::Testnet, + ); + p.generate_addresses(index + 1, &KeySource::Public(xpub), true) + .unwrap(); + p.address_at_index(index).unwrap() + }; + + // idx 3: within eager window (0..=29) — covered by init, NOT in + // unresolved. Contributes to balance but needs no pool extension. + let shallow_recv = derive(AddressPoolType::External, 3); + // idx 30: first past eager window; falls in initial scan window + // (horizon = gap_limit = 30 on a chain with no prior matches). + // Anchors the external probe and extends horizon to 60. + let mid_recv = derive(AddressPoolType::External, 30); + // idx 50: within the extended window (50 < 30+30=60), resolved. + let deep_recv = derive(AddressPoolType::External, 50); + // idx 30: within the internal chain's initial scan window (<=30). + let deep_change = derive(AddressPoolType::Internal, 30); + + let utxo = |addr: Address, value: u64, n: u8| Utxo { + outpoint: OutPoint { + txid: Txid::from([n; 32]), + vout: 0, + }, + txout: TxOut { + value, + script_pubkey: addr.script_pubkey(), + }, + address: addr, + height: 1, + is_coinbase: false, + is_confirmed: true, + is_instantlocked: false, + is_locked: false, + is_trusted: false, + }; + let new_utxos = vec![ + utxo(shallow_recv, 1_000, 1), + utxo(mid_recv.clone(), 10_000, 2), + utxo(deep_recv.clone(), 20_000, 3), + utxo(deep_change.clone(), 300_000, 4), + ]; + let expected_total: u64 = new_utxos.iter().map(|u| u.value()).sum(); + let core = crate::changeset::CoreChangeSet { + new_utxos, + last_processed_height: Some(1), + synced_height: Some(1), + ..Default::default() + }; + + apply_persisted_core_state(&mut wallet_info, &manifest, &core).unwrap(); + + // The wallet total is exact regardless (a sum over the UTXO set). + assert_eq!(wallet_info.balance.total(), expected_total); + + // The per-address view joins pool addresses to UTXOs; every + // resolved UTXO address must now be derived into a pool. + let funds = wallet_info + .accounts + .all_funding_accounts() + .into_iter() + .next() + .unwrap(); + let pool_addresses: HashSet

= funds + .managed_account_type() + .address_pools() + .iter() + .flat_map(|p| p.addresses.values().map(|i| i.address.clone())) + .collect(); + let visible: u64 = funds + .utxos + .values() + .filter(|u| pool_addresses.contains(&u.address)) + .map(|u| u.value()) + .sum(); + assert_eq!( + visible, expected_total, + "all UTXO addresses (including deep-index) must be derived into their pools" + ); + + // Each deep address resolves to its exact derivation slot. + let pools = funds.managed_account_type().address_pools(); + let external = pools.iter().find(|p| p.is_external()).unwrap(); + let internal = pools.iter().find(|p| p.is_internal()).unwrap(); + assert_eq!(external.address_at_index(30).as_ref(), Some(&mid_recv)); + assert_eq!(external.address_at_index(50).as_ref(), Some(&deep_recv)); + assert_eq!(internal.address_at_index(30).as_ref(), Some(&deep_change)); + + // maintain_gap_limit must refill BEYOND the deepest restored + // index so the gap window is actually exercised, not just the restore. + // Deepest external resolved = idx 50; gap window must reach >= 50+30=80. + let expected_min_gen = 50 + DEFAULT_EXTERNAL_GAP_LIMIT; + assert!( + external.highest_generated >= Some(expected_min_gen), + "maintain_gap_limit must extend external pool to >= {} (got {:?})", + expected_min_gen, + external.highest_generated, + ); + } + + /// A UTXO whose address is not derivable from this account's + /// xpub (foreign key, multi-account mismatch) must not cause a panic or + /// hang. The total balance is exact (the UTXO is in the set regardless), + /// but the foreign address is absent from the pool so per-address + /// visibility is reduced. `tracing::warn!` fires for the unresolved count. + #[test] + fn rehydration_unresolvable_address_is_deferred_not_panics() { + use dashcore::blockdata::transaction::txout::TxOut; + use dashcore::{OutPoint, Txid}; + use key_wallet::bip32::DerivationPath; + use key_wallet::gap_limit::DEFAULT_EXTERNAL_GAP_LIMIT; + use key_wallet::managed_account::address_pool::{AddressPool, AddressPoolType, KeySource}; + use key_wallet::managed_account::managed_account_trait::ManagedAccountTrait; + use key_wallet::wallet::managed_wallet_info::ManagedWalletInfo; + use key_wallet::{Address, Utxo}; + use std::collections::HashSet; + + let seed = [13u8; 64]; + let wallet = Wallet::from_seed_bytes( + seed, + Network::Testnet, + WalletAccountCreationOptions::Default, + ) + .unwrap(); + let manifest = manifest_for(&wallet); + let mut wallet_info = ManagedWalletInfo::from_wallet(&wallet, 1); + + let funds_type = wallet_info + .accounts + .all_funding_accounts() + .first() + .unwrap() + .managed_account_type() + .to_account_type(); + let xpub = manifest + .iter() + .find(|e| e.account_type == funds_type) + .map(|e| e.account_xpub) + .expect("funds account xpub"); + + // Normal UTXO at external index 3 (within eager window, pool-visible). + let normal_addr = { + let mut p = AddressPool::new_without_generation( + DerivationPath::master(), + AddressPoolType::External, + DEFAULT_EXTERNAL_GAP_LIMIT, + Network::Testnet, + ); + p.generate_addresses(4, &KeySource::Public(xpub), true) + .unwrap(); + p.address_at_index(3).unwrap() + }; + + // Foreign address: derive from a completely different wallet seed so + // it cannot be resolved from this wallet's xpub. + let foreign_addr = { + let fw = Wallet::from_seed_bytes( + [99u8; 64], + Network::Testnet, + WalletAccountCreationOptions::Default, + ) + .unwrap(); + let fw_info = ManagedWalletInfo::from_wallet(&fw, 1); + fw_info + .accounts + .all_funding_accounts() + .into_iter() + .next() + .unwrap() + .managed_account_type() + .address_pools() + .first() + .unwrap() + .address_at_index(0) + .unwrap() + }; + assert_ne!( + normal_addr, foreign_addr, + "test fixture: foreign address must differ from normal" + ); + + let utxo = |addr: Address, value: u64, n: u8| Utxo { + outpoint: OutPoint { + txid: Txid::from([n; 32]), + vout: 0, + }, + txout: TxOut { + value, + script_pubkey: addr.script_pubkey(), + }, + address: addr, + height: 1, + is_coinbase: false, + is_confirmed: true, + is_instantlocked: false, + is_locked: false, + is_trusted: false, + }; + + let normal_val = 100_000u64; + let foreign_val = 200_000u64; + let expected_total = normal_val + foreign_val; + + let core = crate::changeset::CoreChangeSet { + new_utxos: vec![ + utxo(normal_addr, normal_val, 1), + utxo(foreign_addr, foreign_val, 2), + ], + last_processed_height: Some(1), + synced_height: Some(1), + ..Default::default() + }; + + // Must not panic. tracing::warn! fires for the unresolved count. + apply_persisted_core_state(&mut wallet_info, &manifest, &core).unwrap(); + + // Total balance is exact — foreign UTXO is in the set regardless. + assert_eq!( + wallet_info.balance.total(), + expected_total, + "total must include foreign UTXO even though it is unresolved" + ); + + // Per-address visible: only the normal UTXO is in the pool. + let funds = wallet_info + .accounts + .all_funding_accounts() + .into_iter() + .next() + .unwrap(); + let pool_addresses: HashSet
= funds + .managed_account_type() + .address_pools() + .iter() + .flat_map(|p| p.addresses.values().map(|i| i.address.clone())) + .collect(); + let visible: u64 = funds + .utxos + .values() + .filter(|u| pool_addresses.contains(&u.address)) + .map(|u| u.value()) + .sum(); + assert_eq!( + visible, normal_val, + "only the non-foreign UTXO is pool-visible; foreign deferred to re-warm" + ); + assert!( + visible < expected_total, + "foreign UTXO is deferred — per-address visible < total" + ); + } + + /// CoinJoin topology (single External pool, no Internal chain). + /// Verifies that `extend_pools_for_restored_utxos` handles a single-pool + /// account at a deep derivation index (idx 30, just past the eager window). + #[test] + fn rehydration_coinjoin_single_pool_deep_index() { + use dashcore::blockdata::transaction::txout::TxOut; + use dashcore::{OutPoint, Txid}; + use key_wallet::managed_account::address_pool::{AddressPool, KeySource}; + use key_wallet::managed_account::managed_account_trait::ManagedAccountTrait; + use key_wallet::wallet::managed_wallet_info::ManagedWalletInfo; + use key_wallet::Utxo; + use std::collections::BTreeSet; + + // CoinJoin-only wallet: no BIP44, one CoinJoin account at index 0. + let mut cj_set = BTreeSet::new(); + cj_set.insert(0u32); + let opts = WalletAccountCreationOptions::SpecificAccounts( + BTreeSet::new(), + BTreeSet::new(), + cj_set, + BTreeSet::new(), + BTreeSet::new(), + None, + ); + let seed = [11u8; 64]; + let wallet = Wallet::from_seed_bytes(seed, Network::Testnet, opts).unwrap(); + assert!( + !wallet.accounts.coinjoin_accounts.is_empty(), + "fixture must have a CoinJoin account" + ); + + let manifest = manifest_for(&wallet); + let mut wallet_info = ManagedWalletInfo::from_wallet(&wallet, 1); + + // Extract pool metadata before the mutable borrow of wallet_info. + let (funds_type, pool_base_path, pool_type_val, pool_gap_limit) = { + let funds = wallet_info + .accounts + .all_funding_accounts() + .into_iter() + .next() + .expect("CoinJoin account must be the only funds account"); + let ft = funds.managed_account_type().to_account_type(); + let pools = funds.managed_account_type().address_pools(); + // CoinJoin has a single pool (External). + assert_eq!( + pools.len(), + 1, + "CoinJoin topology: must have exactly one pool" + ); + let p = &pools[0]; + (ft, p.base_path.clone(), p.pool_type, p.gap_limit) + }; + + let xpub = manifest + .iter() + .find(|e| e.account_type == funds_type) + .map(|e| e.account_xpub) + .expect("CoinJoin xpub must be in manifest"); + + // Derive the CoinJoin address at index 30 (first past the eager + // window 0..=29) using the real pool's base_path and pool_type. + let mut probe = AddressPool::new_without_generation( + pool_base_path, + pool_type_val, + pool_gap_limit, + Network::Testnet, + ); + probe + .generate_addresses(31, &KeySource::Public(xpub), true) + .unwrap(); + let deep_cj_addr = probe.address_at_index(30).unwrap(); + + let utxo_val = 7_777u64; + let utxo = Utxo { + outpoint: OutPoint { + txid: Txid::from([7u8; 32]), + vout: 0, + }, + txout: TxOut { + value: utxo_val, + script_pubkey: deep_cj_addr.script_pubkey(), + }, + address: deep_cj_addr.clone(), + height: 1, + is_coinbase: false, + is_confirmed: true, + is_instantlocked: false, + is_locked: false, + is_trusted: false, + }; + + let core = crate::changeset::CoreChangeSet { + new_utxos: vec![utxo], + last_processed_height: Some(1), + synced_height: Some(1), + ..Default::default() + }; + + apply_persisted_core_state(&mut wallet_info, &manifest, &core).unwrap(); + + // Balance is exact. + assert_eq!( + wallet_info.balance.total(), + utxo_val, + "CoinJoin deep-index balance must be exact" + ); + + // The CoinJoin pool was extended to include the deep-index address. + let funds_post = wallet_info + .accounts + .all_funding_accounts() + .into_iter() + .next() + .unwrap(); + let cj_pool = &funds_post.managed_account_type().address_pools()[0]; + assert_eq!( + cj_pool.address_at_index(30).as_ref(), + Some(&deep_cj_addr), + "CoinJoin pool must be extended to cover deep-index address at idx 30" + ); + } + + /// In-window restored UTXO: an address already covered by the eager + /// derivation (idx 3, inside `0..=gap_limit-1`) must still be marked + /// `used` during rehydration. The discovery scan never visits in-window + /// addresses, so without an explicit mark pass a funded address would keep + /// `used = false` and could later be handed out as a fresh receive address. + #[test] + fn rehydration_marks_in_window_restored_address_used() { + use dashcore::blockdata::transaction::txout::TxOut; + use dashcore::{OutPoint, Txid}; + use key_wallet::bip32::DerivationPath; + use key_wallet::gap_limit::DEFAULT_EXTERNAL_GAP_LIMIT; + use key_wallet::managed_account::address_pool::{AddressPool, AddressPoolType, KeySource}; + use key_wallet::managed_account::managed_account_trait::ManagedAccountTrait; + use key_wallet::wallet::managed_wallet_info::ManagedWalletInfo; + use key_wallet::{Address, Utxo}; + + let wallet = Wallet::from_seed_bytes( + [5u8; 64], + Network::Testnet, + WalletAccountCreationOptions::Default, + ) + .unwrap(); + let manifest = manifest_for(&wallet); + let mut wallet_info = ManagedWalletInfo::from_wallet(&wallet, 1); + + let funds_type = wallet_info + .accounts + .all_funding_accounts() + .first() + .unwrap() + .managed_account_type() + .to_account_type(); + let xpub = manifest + .iter() + .find(|e| e.account_type == funds_type) + .map(|e| e.account_xpub) + .expect("funds account xpub"); + + // External idx 3 — inside the eager window, so NOT in the discovery set. + let in_window: Address = { + let mut p = AddressPool::new_without_generation( + DerivationPath::master(), + AddressPoolType::External, + DEFAULT_EXTERNAL_GAP_LIMIT, + Network::Testnet, + ); + p.generate_addresses(4, &KeySource::Public(xpub), true) + .unwrap(); + p.address_at_index(3).unwrap() + }; + + let core = crate::changeset::CoreChangeSet { + new_utxos: vec![Utxo { + outpoint: OutPoint { + txid: Txid::from([1u8; 32]), + vout: 0, + }, + txout: TxOut { + value: 12_345, + script_pubkey: in_window.script_pubkey(), + }, + address: in_window.clone(), + height: 1, + is_coinbase: false, + is_confirmed: true, + is_instantlocked: false, + is_locked: false, + is_trusted: false, + }], + last_processed_height: Some(1), + synced_height: Some(1), + ..Default::default() + }; + + apply_persisted_core_state(&mut wallet_info, &manifest, &core).unwrap(); + + let funds = wallet_info + .accounts + .all_funding_accounts() + .into_iter() + .next() + .unwrap(); + let pools = funds.managed_account_type().address_pools(); + let external = pools.iter().find(|p| p.is_external()).unwrap(); + let info = external + .address_info(&in_window) + .expect("in-window address must be present in the pool"); + assert!( + info.used, + "in-window restored UTXO address must be marked used" + ); + assert!( + external.used_indices.contains(&3), + "used_indices must record the in-window slot" + ); + assert_eq!( + external.highest_used, + Some(3), + "highest_used must reflect the in-window slot" + ); + } + + /// Documented limitation (solution b): a legitimately-owned but + /// deep-and-sparse UTXO — external idx 45 with nothing unspent at idx + /// <= 30 — is left unresolved because the discovery horizon (gap_limit + /// past the deepest match) never advances far enough to reach it. The + /// wallet total stays exact; only the per-address view is incomplete + /// until the next sync (a `tracing::warn!` records the deferral). + #[test] + fn rehydration_deep_sparse_utxo_left_unresolved_total_exact() { + use dashcore::blockdata::transaction::txout::TxOut; + use dashcore::{OutPoint, Txid}; + use key_wallet::bip32::DerivationPath; + use key_wallet::gap_limit::DEFAULT_EXTERNAL_GAP_LIMIT; + use key_wallet::managed_account::address_pool::{AddressPool, AddressPoolType, KeySource}; + use key_wallet::managed_account::managed_account_trait::ManagedAccountTrait; + use key_wallet::wallet::managed_wallet_info::ManagedWalletInfo; + use key_wallet::{Address, Utxo}; + use std::collections::HashSet; + + let wallet = Wallet::from_seed_bytes( + [21u8; 64], + Network::Testnet, + WalletAccountCreationOptions::Default, + ) + .unwrap(); + let manifest = manifest_for(&wallet); + let mut wallet_info = ManagedWalletInfo::from_wallet(&wallet, 1); + + let funds_type = wallet_info + .accounts + .all_funding_accounts() + .first() + .unwrap() + .managed_account_type() + .to_account_type(); + let xpub = manifest + .iter() + .find(|e| e.account_type == funds_type) + .map(|e| e.account_xpub) + .expect("funds account xpub"); + + // External idx 45 — past the eager window AND past the initial scan + // window (horizon = gap_limit = 30 with no nearer match to extend it). + let sparse_deep: Address = { + let mut p = AddressPool::new_without_generation( + DerivationPath::master(), + AddressPoolType::External, + DEFAULT_EXTERNAL_GAP_LIMIT, + Network::Testnet, + ); + p.generate_addresses(46, &KeySource::Public(xpub), true) + .unwrap(); + p.address_at_index(45).unwrap() + }; + + let value = 500_000u64; + let core = crate::changeset::CoreChangeSet { + new_utxos: vec![Utxo { + outpoint: OutPoint { + txid: Txid::from([4u8; 32]), + vout: 0, + }, + txout: TxOut { + value, + script_pubkey: sparse_deep.script_pubkey(), + }, + address: sparse_deep.clone(), + height: 1, + is_coinbase: false, + is_confirmed: true, + is_instantlocked: false, + is_locked: false, + is_trusted: false, + }], + last_processed_height: Some(1), + synced_height: Some(1), + ..Default::default() + }; + + apply_persisted_core_state(&mut wallet_info, &manifest, &core).unwrap(); + + // The wallet total is exact regardless (a sum over the UTXO set). + assert_eq!(wallet_info.balance.total(), value); + + let funds = wallet_info + .accounts + .all_funding_accounts() + .into_iter() + .next() + .unwrap(); + let pools = funds.managed_account_type().address_pools(); + let external = pools.iter().find(|p| p.is_external()).unwrap(); + assert!( + !external.contains_address(&sparse_deep), + "deep-sparse idx 45 must be left unresolved (absent from the pool)" + ); + + // Per-address view: the deep-sparse UTXO is not pool-visible yet. + let pool_addresses: HashSet
= pools + .iter() + .flat_map(|p| p.addresses.values().map(|i| i.address.clone())) + .collect(); + let visible: u64 = funds + .utxos + .values() + .filter(|u| pool_addresses.contains(&u.address)) + .map(|u| u.value()) + .sum(); + assert_eq!( + visible, 0, + "the deep-sparse UTXO is deferred — not pool-visible until next sync" + ); + assert!(visible < value, "per-address visible < exact total"); + } + + /// Topology guard: a wallet with persisted UTXOs but NO funds-bearing + /// account cannot hold them — fail closed with + /// `RehydrationTopologyUnsupported` (reporting the persisted count) rather + /// than reconstruct a silent zero balance. + #[test] + fn rehydration_utxos_without_funds_account_errors() { + use dashcore::address::Payload; + use dashcore::blockdata::transaction::txout::TxOut; + use dashcore::hashes::Hash; + use dashcore::{OutPoint, PubkeyHash, Txid}; + use key_wallet::account::AccountType; + use key_wallet::wallet::managed_wallet_info::ManagedWalletInfo; + use key_wallet::{Address, Utxo}; + use std::collections::BTreeSet; + + // Keys-only wallet: a single IdentityRegistration account, no funds. + let opts = WalletAccountCreationOptions::SpecificAccounts( + BTreeSet::new(), + BTreeSet::new(), + BTreeSet::new(), + BTreeSet::new(), + BTreeSet::new(), + Some(vec![AccountType::IdentityRegistration]), + ); + let wallet = Wallet::from_seed_bytes([23u8; 64], Network::Testnet, opts).unwrap(); + let manifest = manifest_for(&wallet); + let mut wallet_info = ManagedWalletInfo::from_wallet(&wallet, 1); + assert!( + wallet_info.accounts.all_funding_accounts().is_empty(), + "fixture must have NO funds-bearing account" + ); + + let addr = Address::new( + Network::Testnet, + Payload::PubkeyHash(PubkeyHash::from_byte_array([9u8; 20])), + ); + let core = crate::changeset::CoreChangeSet { + new_utxos: vec![Utxo { + outpoint: OutPoint { + txid: Txid::from([2u8; 32]), + vout: 0, + }, + txout: TxOut { + value: 800_000, + script_pubkey: addr.script_pubkey(), + }, + address: addr, + height: 1, + is_coinbase: false, + is_confirmed: true, + is_instantlocked: false, + is_locked: false, + is_trusted: false, + }], + last_processed_height: Some(1), + synced_height: Some(1), + ..Default::default() + }; + + let err = apply_persisted_core_state(&mut wallet_info, &manifest, &core) + .expect_err("must fail closed when no funds account can hold the UTXOs"); + match err { + PlatformWalletError::RehydrationTopologyUnsupported { utxo_count, .. } => { + assert_eq!(utxo_count, 1, "utxo_count must match the persisted set"); + } + other => panic!("expected RehydrationTopologyUnsupported, got {other:?}"), + } + } + + /// Companion to the topology guard: the same keys-only wallet with an + /// EMPTY persisted UTXO set is `Ok` — there is nothing to hold, so the + /// guard does not trip. + #[test] + fn rehydration_no_funds_account_empty_utxos_ok() { + use key_wallet::account::AccountType; + use key_wallet::wallet::managed_wallet_info::ManagedWalletInfo; + use std::collections::BTreeSet; + + let opts = WalletAccountCreationOptions::SpecificAccounts( + BTreeSet::new(), + BTreeSet::new(), + BTreeSet::new(), + BTreeSet::new(), + BTreeSet::new(), + Some(vec![AccountType::IdentityRegistration]), + ); + let wallet = Wallet::from_seed_bytes([24u8; 64], Network::Testnet, opts).unwrap(); + let manifest = manifest_for(&wallet); + let mut wallet_info = ManagedWalletInfo::from_wallet(&wallet, 1); + assert!(wallet_info.accounts.all_funding_accounts().is_empty()); + + let core = crate::changeset::CoreChangeSet { + last_processed_height: Some(1), + synced_height: Some(1), + ..Default::default() + }; + apply_persisted_core_state(&mut wallet_info, &manifest, &core) + .expect("empty UTXO set must be Ok even with no funds account"); + } } diff --git a/packages/rs-platform-wallet/src/manager/shielded_sync.rs b/packages/rs-platform-wallet/src/manager/shielded_sync.rs index 482674b432..e7892e8dff 100644 --- a/packages/rs-platform-wallet/src/manager/shielded_sync.rs +++ b/packages/rs-platform-wallet/src/manager/shielded_sync.rs @@ -261,14 +261,15 @@ impl ShieldedSyncManager { } } - // Only clear `background_cancel` if the active - // generation is still ours. Without this guard a - // tight `stop()` → `start()` reschedule has the - // exiting thread overwrite the *new* generation's - // token, leaving the new loop running but - // unreflectable via `is_running()` / `stop()`. - if this.background_generation.load(Ordering::Acquire) == my_gen { - if let Ok(mut guard) = this.background_cancel.lock() { + // Clear `background_cancel` only if the active + // generation is still ours. Acquire the lock + // FIRST, then check — `start()` bumps the + // generation while holding this same lock, so + // once we hold it the generation is final w.r.t. + // any concurrent token swap (no TOCTOU between + // the check and the clear). + if let Ok(mut guard) = this.background_cancel.lock() { + if this.background_generation.load(Ordering::Acquire) == my_gen { *guard = None; } } @@ -475,3 +476,75 @@ impl std::fmt::Debug for ShieldedSyncManager { .finish() } } + +#[cfg(test)] +mod tests { + use super::*; + + use crate::events::PlatformEventHandler; + + /// Build a manager with an empty coordinator slot and a no-handler + /// event manager. An empty slot makes `sync_now` return an empty + /// summary, but it still drives the full timestamp + completion + /// protocol and — crucially for this test — the generation-guarded + /// background loop, without needing a live `NetworkShieldedCoordinator`. + fn make_manager() -> Arc { + let event_manager = Arc::new(PlatformEventManager::new(Vec::< + Arc, + >::new())); + let coordinator_slot = Arc::new(RwLock::new(None)); + Arc::new(ShieldedSyncManager::new(event_manager, coordinator_slot)) + } + + /// Restart-in-place regression for the generation guard: a tight + /// `start()` → `stop()` → `start()` must leave the manager *running* + /// on the new generation. The cancelled gen-1 loop races to clear + /// `background_cancel` as it exits; the generation guard must stop it + /// from stripping gen-2's freshly installed token — otherwise the new + /// loop keeps running but becomes invisible to `is_running()` / + /// `stop()`. + /// + /// Determinism: the only wait is a *bounded* poll. With the guard in + /// place `is_running()` is true for the whole window, so the test + /// never fails spuriously on correct code. A regression flips it false + /// within milliseconds once the stale loop clears the slot, which the + /// poll catches. Needs the multi-thread flavor because `start()` + /// drives its loop via `Handle::current().block_on` on a dedicated OS + /// thread, which would deadlock a single-threaded test runtime. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn restart_in_place_keeps_running_after_stale_loop_exits() { + let mgr = make_manager(); + + // Gen 1. Wait (bounded) for the first pass to land — a real + // lifecycle signal that the loop is now parked in its interval + // sleep, so its cleanup is still pending when we stop+restart. + Arc::clone(&mgr).start(); + let mut waited = 0; + while mgr.last_sync_unix_seconds().is_none() { + assert!(waited < 200, "gen-1's first sync pass never completed"); + tokio::time::sleep(Duration::from_millis(10)).await; + waited += 1; + } + + // Tight stop→start with no await between: the just-cancelled gen-1 + // loop cannot reach its cleanup before gen 2 is installed, so the + // race window the guard protects is reliably open. + mgr.stop(); + Arc::clone(&mgr).start(); + + // Give the stale gen-1 loop ample time to run its (guarded) + // cleanup. `is_running()` must stay true throughout. + for _ in 0..100 { + assert!( + mgr.is_running(), + "stale gen-1 loop cleared gen-2's cancel token — generation guard regressed" + ); + tokio::time::sleep(Duration::from_millis(10)).await; + } + + // The surviving loop is the tracked one: a single `stop()` fully + // reflects it, so there is no orphaned unreflectable duplicate. + mgr.stop(); + assert!(!mgr.is_running(), "stop() must reflect the live loop"); + } +}