refactor(chain,core)!: replace `CanonicalIter` with sans-IO `CanonicalTask` + `ChainQuery` trait by oleonardolima · Pull Request #2038 · bitcoindevkit/bdk

oleonardolima · 2025-09-18T05:59:57Z

Description

Replaces the iterator-based CanonicalIter with a two-phase sans-IO canonicalization pipeline, and introduces a generic ChainQuery trait in bdk_core to decouple canonicalization from chain sources.

Old API:

// Direct coupling between canonicalization logic and ChainOracle
let view = tx_graph.canonical_view(&chain, chain_tip, params)?;

New API:

// Option A: Two-phase (full control)
let canonical_txs = chain.canonicalize(tx_graph.canonical_task(tip, params));
let view = chain.canonicalize(canonical_txs.view_task(&tx_graph));

// Option B: Convenience method
let view = chain.canonical_view(&tx_graph, tip, params);

Phase 1: `CanonicalTask`

Determines which transactions are canonical by processing them in stages:

Assumed txs — transactions assumed canonical via CanonicalParams
Anchored txs — transactions anchored in the best chain (descending height)
Seen txs — unconfirmed transactions by descending last-seen time
Remaining txs — leftover anchored transactions not in the best chain

Produces a CanonicalTxs<A> containing each canonical transaction with its CanonicalReason.

Phase 2: `CanonicalViewTask`

Resolves CanonicalReasons into concrete ChainPositions (confirmed height or unconfirmed with last-seen), producing the final CanonicalView<A>.

Both phases implement the ChainQuery trait, so any chain source can drive them via the same next_query/resolve_query loop.

Key structural changes

ChainQuery trait added to bdk_core — a generic sans-IO interface (next_query → resolve_query → finish) for any algorithm that needs to verify blocks against a chain source.
ChainOracle trait removed — replaced by ChainQuery. LocalChain::canonicalize() now drives any ChainQuery implementor.
Canonical<A, P> generic container — CanonicalTxs<A> (phase 1 output) and CanonicalView<A> (phase 2 output) are type aliases over Canonical<A, P>.
Module split — canonical_view.rs split into canonical.rs (types: Canonical, CanonicalTx, CanonicalTxOut) and canonical_view_task.rs (phase 2 task). canonical_iter.rs replaced by canonical_task.rs.

Notes to the reviewers

The changes are split into multiple commits for easier review. Also depends on #2029.

Changelog notice

  ### Added
  - `bdk_core::ChainQuery` trait — generic sans-IO interface for chain verification queries
  - `bdk_core::ChainRequest` / `ChainResponse` type aliases
  - `CanonicalTask` — phase 1 sans-IO canonicalization (determines canonical txs)
  - `CanonicalViewTask` — phase 2 sans-IO canonicalization (resolves chain positions)
  - `Canonical<A, P>` generic container with `CanonicalTxs<A>` and `CanonicalView<A>` aliases
  - `LocalChain::canonicalize()` — drives any `ChainQuery` implementor
  - `LocalChain::canonical_view()` — convenience method for full two-phase canonicalization

  ### Changed
  - **Breaking:** Replace `TxGraph::canonical_iter()` / `TxGraph::canonical_view()` with `TxGraph::canonical_task()`
  - **Breaking:** Canonicalization now uses a two-phase sans-IO process via `ChainQuery`
  - **Breaking:** `ChainQuery`, `ChainRequest`, `ChainResponse` have no generics (use `BlockId` directly)
  - **Breaking:** Chain tip moved from `ChainRequest` to `ChainQuery::tip()`

  ### Removed
  - **Breaking:** `ChainOracle` trait and all implementations
  - **Breaking:** `CanonicalIter` type and `canonical_iter` module
  - **Breaking:** `TxGraph::try_canonical_view()` and `TxGraph::canonical_view()` methods
  - **Breaking:** `CanonicalView::new()` public constructor

Checklists

All Submissions:

I followed the contribution guidelines

New Features:

I've added tests for the new feature
I've added docs for the new feature

evanlinjin

Great work.

This is my initial round of reviews.

Are you planning to introduce topological ordering in a separate PR?

evanlinjin · 2025-10-02T06:43:24Z

+    let chain_tip = chain.tip().block_id();
+    let task = graph.canonicalization_task(chain_tip, Default::default());
+    let canonical_view = chain.canonicalize(task);


What do you think about the following naming:

CanonicalizationTask -> CanonicalResolver.

TxGraph::canonicalization_task -> TxGraph::resolver.

LocalChain::canonicalize -> LocalChain::resolve.

I've been thinking about this, I agree with the CanonicalResolver, though the TxGraph::resolver and LocalChain::resolve seems a bit off.

What do you think about (?):

CanonicalResolver

TxGraph::canonical_resolver

LocalChain::canonical_resolve

Alright, this one is a bit outdated.

As of 031de40 we have:

CanonicalizationTask -> CanonicalTask; and also new CanonicalViewTask.

TxGraph::canonicalization_task -> TxGraph::canonical_task.

LocalChain::canonicalize is still the same, though we now also have LocalChain::canonical_view.

I'm fine with these names for now, though if there's no consensus on those we can discuss/change in a follow-up PR.

ValuedMammal · 2026-04-06T14:48:27Z

I can take another look at it.

ValuedMammal · 2026-04-07T23:46:12Z

While reviewing I found it easier to structure the commits like this.

feat: Introduce CanonicalTask

Introduces CanonicalTask, a new type that encapsulates block-anchor verification as an explicit query/response loop. The task walks the transaction graph in priority-ordered stages (assumed, anchored, seen, leftover) and emits batches of BlockIds for the caller to verify against a chain oracle, accepting responses via resolve_query. Once driven to completion, finish() returns a CanonicalTxs which can be converted to a CanonicalView via .view().

feat!: Update usage of CanonicalTask

Migrates the entire codebase to the new CanonicalTask-based API, removing the now-superseded canonical_iter.rs and canonical_view.rs modules. Introduces canonical.rs to house the unified Canonical<A, T, P> type (aliased as CanonicalTxs and CanonicalView) along with the CanonicalReason-to-ChainPosition resolution in .view(). All consumers — tests, benchmarks, examples, and chain-sync integrations — are updated accordingly.

refactor!: Remove ChainOracle

Removes the ChainOracle trait, which is no longer needed now that canonicalization is driven through CanonicalTask's explicit query/response interface rather than a generic trait. LocalChain's is_block_in_chain and get_chain_tip methods are retained as plain inherent methods (returning Option instead of Result<Option, Infallible>), simplifying the API surface without losing any functionality.

evanlinjin · 2026-04-10T03:49:51Z

+                                    match TxDescendants::new_exclude_root(
+                                        self.tx_graph,
+                                        *txid,
+                                        |_, desc_txid| -> Option<(Txid, &A)> {
+                                            // assert the descendant visited is canonical
+                                            self.canonical_txs
+                                                .contains_key(&desc_txid)
+                                                .then(|| {
+                                                    self.direct_anchors
+                                                        .get(&desc_txid)
+                                                        .map(|anchor| (desc_txid, anchor))
+                                                })
+                                                .flatten()
+                                        },
+                                    )


This will stop when a descendant doesn't have a direct anchor which is incorrect as a direct anchor will follow after.

What should happen

The closure should return everything (no filter).

.next should stop once it hits a direct anchor.

What should be figured out in a follow up

Although TxDescendants does a BFS (which means we will find the "shallowest" confirmed anchor) - this does not guarantee it will be the "earliest confirmed" anchor (which is the ideal value to have).

evanlinjin · 2026-04-27T10:53:56Z

@oleonardolima Thanks for taking the PR this far. To push this forward, I would like to take over.

oleonardolima · 2026-04-27T13:20:08Z

@oleonardolima Thanks for taking the PR this far. To push this forward, I would like to take over.

@evanlinjin Alright, I'll take a look into your pushed commits.

evanlinjin

ACK a5d91e9

However, I agree with @ValuedMammal that the commits are a bit messy.

oleonardolima · 2026-05-12T13:18:10Z

ACK a5d91e9

However, I agree with @ValuedMammal that the commits are a bit messy.

Alright, so the only thing missing is fixing the commit history. I'll handle it and ping VM's for review.

It introduces the new `CanonicalizationTask` that's implements the canonicalization algorithm through a request/response pattern. Also, it introduces the new `ChainQuery` trait in `bdk_core`, which provides an interface for blockchain source/oracle query-based operations. Allowing sans-IO patterns for algorithm that needs a blockchain oracle, without the need for directly implement/handle I/O. Adds new API methods into `LocalChain`: `canonicalize` and `canonical_view`, adding same features as the existing `CanonicalIter` and it's APIs. Co-Authored-By: Claude <noreply@anthropic.com>

Updates the codebase to use the new convenience `LocalChain::canonical_view` method in order to generate the `CanonicalView`. Internally the convenience method follows the `sans-IO` approach, separating the canonicalization algorithm from i/o operations, and it's used as follows: 1. Create a new `CanonicalizationTask` with a `TxGraph`, by calling: `graph.canonicalization_task(params)` 2. Execute the canonicalization process with a chain oracle (e.g `LocalChain`, which implements `ChainOracle` trait), by calling: `chain.canonicalize(task, chain_tip)`

The codebase has been updated to the new `LocalChain::canonical_view` method. It's now safe to remove the `CanonicalIter` it's the old APIs relying on it, eg. `try_canonical_view`.

…`Canonical<A, P>` Separate concerns by splitting `CanonicalizationTask` into two phases: 1. `CanonicalTask` determines which transactions are canonical and why (`CanonicalReason`), outputting `CanonicalTxs<A>`. 2. `CanonicalViewTask` resolves reasons into `ChainPosition`s (confirmed vs unconfirmed), outputting `CanonicalView<A>`. Make `Canonical<A, P>`, `CanonicalTx<P>`, and `FullTxOut<P>` generic over the position type so the same structs serve both phases. Add `LocalChain::canonical_view()` convenience method for the common two-step pipeline. Renames: - `CanonicalizationTask` -> `CanonicalTask` - `CanonicalizationParams` -> `CanonicalParams` - `canonicalization_task()` -> `canonical_task()` - `FullTxOut::chain_position` -> `FullTxOut::pos` Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

…view_task` It moves the shared types: `CanonicalTx`, `Canonical`, `CanonicalView`, `CanonicalTxs` and other convenience methods into `canonical.rs`. Keep the phase-2 task (`CanonicalViewTask` in `canonical_view_task.rs`. Also, rename `FullTxOut` to `CanonicalTxOut`, and move it to `canonical.rs`. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

codecov · 2026-05-20T00:16:22Z

Codecov Report

❌ Patch coverage is 89.15187% with 55 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.65%. Comparing base (e095266) to head (815fbcb).
⚠️ Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
crates/chain/src/canonical.rs	80.18%	19 Missing and 2 partials ⚠️
crates/chain/src/canonical_task.rs	94.71%	9 Missing and 3 partials ⚠️
crates/chain/src/canonical_view_task.rs	91.80%	6 Missing and 4 partials ⚠️
crates/chain/src/indexed_tx_graph.rs	0.00%	8 Missing ⚠️
crates/chain/src/local_chain.rs	88.88%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2038      +/-   ##
==========================================
+ Coverage   77.69%   78.65%   +0.96%     
==========================================
  Files          29       30       +1     
  Lines        5801     5909     +108     
  Branches      271      279       +8     
==========================================
+ Hits         4507     4648     +141     
+ Misses       1223     1185      -38     
- Partials       71       76       +5

Flag	Coverage Δ
rust	`78.65% <89.15%> (+0.96%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

oleonardolima · 2026-05-20T00:16:42Z

@evanlinjin @ValuedMammal I've pushed the new commit history, it's now only 7 commits. I think it's the best we can get without adding more complexity to some commits.

I've tried to reduce it into fewer commits, however it got even messier as some commits had too much of a change/complexity.

At least now, each commit builds/runs successfully and truly builds on top of each other.

--

NOTE:

I think the commits b0e72fe and d45160d could be squashed together, but left as it's for better clarity.
I think can be dropped 728454d and inserted in a follow-up PR.

evanlinjin

@oleonardolima I'll just go ahead and change this!

evanlinjin · 2026-05-20T07:11:08Z

+                                anchor,
+                                transitively: None,
+                            },
+                            None => match descendant {


descendant == None means that this transaction may be the one marked as "assumed" and does not mean "this has no descendants". Therefore, we should remove this match statement.

- add new `test_canonical_view_task.rs` to handle different scenarios of chain position resolution. - fixes the assumed canonical txs chain position resolution, especially for transitively assumed canonical transactions, where there's an anchored/confirmed descendant.

evanlinjin · 2026-05-20T09:00:11Z

+}
+
+#[test]
+fn test_assumed_canonical_scenarios() {


This test is a bit odd.

We have only a single scenario so the loop is dead weight.

Also, the loop is not reusable because of the hardcoded values within it.

Agreed, I was aiming to have different scenarios but didn't add those, as thought it was redundant.

Also, I vote to drop this commit on this PR and put it strictly in another PR fixing the canonicalization algorithm, it'd be easier to backport it.

nymius

Most of the suggestions are nits related to first impressions from the code and code comprehension.
The only significant change I would do here, as @ValuedMammal suggested before, is the Kahn's algorithm addition.
The Canonical struct is claiming something that is not true (see linked tests) if not paired with this algorithm.
There are two chances here, to remove the claim or actually implement the sort.
I'm inclined for the second, as the implementation I've implemented with LLM help is not as complex as I thought: nymius@dbfabad

It also simplifies the code in canonical task, as some of the order related fields are unnecessary after implementing the sort algorithm.

I've also compared the implementation there with the implementation from 1a77475.
I've ported some of those test to the current implementation in nymius@25049a9.
I don't use the chain position to disambiguate order positions nor create depth based tiers in the order because I'm not aware of use cases that require this disposition.

nymius · 2026-05-12T21:22:56Z

+    /// Contruct a new [`CanonicalReason`] from the original which is transitive to `descendant`.
+    ///
+    /// This signals that either the [`ObservedIn`] or [`Anchor`] value belongs to the transaction's


This is confusing, what's the direction of the transitivity? ancestor ---> descendant or descendant ---> ancestor?

nymius · 2026-05-19T13:39:08Z

+    /// Transactions that will supersede all other transactions.
+    ///
+    /// In case of conflicting transactions within `assume_canonical`, transactions that appear
+    /// later in the list (have higher index) have precedence.


have higher index is misleading, which is the index? the Txid (I wouldn't call it an index)

nymius · 2026-05-19T13:59:21Z

+                    Entry::Vacant(entry) => entry,
+                };
+
+                // Any conflicts with a canonical tx can be added to `not_canonical`. Descendants


Can we add a comment header here? i.e.:

// Prune conflicts // // Any conflicts....

nymius · 2026-05-19T14:29:04Z

+
+                if self.not_canonical.contains(&this_txid) {
+                    // Early exit if self-double-spend is detected.
+                    detected_self_double_spend = true;


When a self double spend is detected, shouldn't the code stop straight away instead of "running until finished"?

nymius · 2026-05-20T18:26:00Z

+
+/// A canonical transaction output with position and spend information.
+///
+/// The position type `P` is generic — it can be [`ChainPosition`] for resolved views,


Same than above: this character "—" is ambiguous: U+2014 (em dash). U+002D (hyphen minus) is the one carried by most keyboards.

nymius · 2026-05-20T18:31:53Z

+    ///
+    /// This signals that either the [`ObservedIn`] or [`Anchor`] value belongs to the transaction's
+    /// descendant, but is transitively relevant.
+    pub fn to_transitive(&self, descendant: Txid) -> Self {


transitivity is a directed relation. Without some clues, it may be hard to understand what's the direction here. I would change to_transitive to by_descendant or something similar that shows the tx is canonical because its descendant is. So it is clear the canonical reason is transmitted from the descendant to the ancestor.

How about rename both the field and method to be via_descendant?

nymius · 2026-05-20T18:31:54Z

+        }
+    }
+
+    /// Construct a new [`CanonicalReason`] from the original which is transitive to `descendant`.


This is confusing, what's the direction of the transitivity? ancestor ---> descendant or descendant ---> ancestor`?

nymius · 2026-05-20T22:16:35Z

+///   [`CanonicalTxs`])
+///
 /// The view maintains:
 /// - An ordered list of canonical transactions in topological-spending order


This is wrong, the current order is not topological (parent before descendant) nor topological-reversed (descendant before parent).
This test shows how it can be broken with a 4-tx diamond shaped graph.
This can be simplified, the following test breaks the topological assumption again with a 3-tx chain shaped graph.

ValuedMammal · 2026-05-21T14:57:56Z

Thank you @nymius

oleonardolima · 2026-05-21T21:22:08Z

Most of the suggestions are nits related to first impressions from the code and code comprehension. The only significant change I would do here, as @ValuedMammal suggested before, is the Kahn's algorithm addition. The Canonical struct is claiming something that is not true (see linked tests) if not paired with this algorithm. There are two chances here, to remove the claim or actually implement the sort.

@nymius thanks for the review, I'll take a look on the comments. In my opinion, this PR's already got complex enough by itself, I'd suggest adding the topological order support as a follow-up, as it was intended by #2201, I'd say the same regarding some canonicalization algorithm fixes that I've added alongside here.

nymius · 2026-05-22T11:39:14Z

@nymius thanks for the review, I'll take a look on the comments. In my opinion, this PR's already got complex enough by itself, I'd suggest adding the topological order support as a follow-up, as it was intended by #2201, I'd say the same regarding some canonicalization algorithm fixes that I've added alongside

I saw the topological order issue is on the same milestone than this one, so I'm ok with this, as long they go together in the same release. I would still remove any topological order claims from docs before that happens.

github-project-automation Bot added this to BDK Chain Sep 18, 2025

evanlinjin reviewed Sep 18, 2025

View reviewed changes

Comment thread crates/chain/src/canonical_task.rs Outdated

LagginTimes assigned oleonardolima Sep 18, 2025

oleonardolima added this to the Wallet 3.0.0 milestone Sep 18, 2025

notmandatory moved this to In Progress in BDK Chain Sep 18, 2025

oleonardolima mentioned this pull request Sep 19, 2025

refactor(chain)!: remove ChainOracle dependency from CanonicalIter #2034

Closed

6 tasks

oleonardolima force-pushed the refactor/canonical-iter-api branch 6 times, most recently from d851ba6 to c02636d Compare September 23, 2025 00:54

oleonardolima added module-blockchain api A breaking API change labels Sep 23, 2025

oleonardolima force-pushed the refactor/canonical-iter-api branch from c02636d to 78c0538 Compare September 23, 2025 01:08

oleonardolima requested review from LLFourn and LagginTimes September 23, 2025 01:09

oleonardolima mentioned this pull request Sep 24, 2025

refactor(chain)!: update ChainOracle and TxGraph methods to being… #2035

Closed

6 tasks

oleonardolima force-pushed the refactor/canonical-iter-api branch 3 times, most recently from 677e25a to 9e27ab1 Compare September 29, 2025 01:47

evanlinjin reviewed Oct 2, 2025

View reviewed changes

Comment thread crates/chain/src/canonical.rs

oleonardolima marked this pull request as ready for review October 2, 2025 06:18

evanlinjin reviewed Oct 2, 2025

View reviewed changes

oleonardolima force-pushed the refactor/canonical-iter-api branch from 9e27ab1 to f6c8b02 Compare October 3, 2025 00:33

evanlinjin reviewed Oct 7, 2025

View reviewed changes

Comment thread crates/core/src/chain_query.rs Outdated

Comment thread crates/chain/src/canonical_task.rs Outdated

Comment thread crates/core/src/chain_query.rs Outdated

oleonardolima force-pushed the refactor/canonical-iter-api branch 4 times, most recently from a276c37 to b7f8fba Compare October 8, 2025 04:53

evanlinjin requested changes Apr 10, 2026

View reviewed changes

evanlinjin force-pushed the refactor/canonical-iter-api branch from 6f8ef26 to 15ce547 Compare April 27, 2026 11:23

evanlinjin force-pushed the refactor/canonical-iter-api branch from b54a71e to 8aa42d0 Compare April 28, 2026 11:44

evanlinjin approved these changes Apr 28, 2026

View reviewed changes

oleonardolima and others added 3 commits May 19, 2026 20:46

refactor(chain)!: remove CanonicalIter APIs

a63f33d

The codebase has been updated to the new `LocalChain::canonical_view` method. It's now safe to remove the `CanonicalIter` it's the old APIs relying on it, eg. `try_canonical_view`.

oleonardolima force-pushed the refactor/canonical-iter-api branch 2 times, most recently from d07a301 to 838cdfc Compare May 20, 2026 00:01

evanlinjin and others added 3 commits May 19, 2026 21:05

chore(chain,example)!: remove ChainOracle, update docs

6d4a73c

oleonardolima force-pushed the refactor/canonical-iter-api branch from 838cdfc to 728454d Compare May 20, 2026 00:07

oleonardolima moved this from In Progress to Needs Review in BDK Chain May 20, 2026

evanlinjin requested changes May 20, 2026

View reviewed changes

evanlinjin force-pushed the refactor/canonical-iter-api branch from 728454d to 815fbcb Compare May 20, 2026 08:27

evanlinjin reviewed May 20, 2026

View reviewed changes

nymius requested changes May 20, 2026

View reviewed changes

nymius mentioned this pull request May 21, 2026

Add optional topological ordering stage CanonicalViewTask #2201

Open

Conversation

oleonardolima commented Sep 18, 2025 • edited by evanlinjin Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Phase 1: CanonicalTask

Phase 2: CanonicalViewTask

Key structural changes

Notes to the reviewers

Changelog notice

Checklists

All Submissions:

New Features:

Uh oh!

Uh oh!

Uh oh!

evanlinjin left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

oleonardolima Mar 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ValuedMammal commented Apr 6, 2026

Uh oh!

ValuedMammal commented Apr 7, 2026

Uh oh!

Choose a reason for hiding this comment

What should happen

What should be figured out in a follow up

Uh oh!

Uh oh!

Uh oh!

evanlinjin commented Apr 27, 2026

Uh oh!

oleonardolima commented Apr 27, 2026

Uh oh!

evanlinjin left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

oleonardolima commented May 12, 2026

Uh oh!

codecov Bot commented May 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

oleonardolima commented May 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

evanlinjin left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

oleonardolima May 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nymius left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

oleonardolima commented Sep 18, 2025 •

edited by evanlinjin

Loading

Phase 1: `CanonicalTask`

Phase 2: `CanonicalViewTask`

oleonardolima Mar 6, 2026 •

edited

Loading

evanlinjin left a comment •

edited

Loading

codecov Bot commented May 20, 2026 •

edited

Loading

oleonardolima commented May 20, 2026 •

edited

Loading

oleonardolima May 22, 2026 •

edited

Loading