Skip to content

Make KD-tree encoder memory linear in the point dimension - #1248

Open
Filyus wants to merge 1 commit into
google:mainfrom
Filyus:kd-tree-encoder-undo-log
Open

Filyus wants to merge 1 commit into
google:mainfrom
Filyus:kd-tree-encoder-undo-log

Conversation

@Filyus

@Filyus Filyus commented Oct 10, 2026

Copy link
Copy Markdown
Contributor

DynamicIntegerPointsKdTreeEncoder allocates memory quadratic in the point dimension, up front, in its constructor. The dimension is the total number of components over all attributes. This change makes the encoder's memory linear in the dimension, the way DynamicIntegerPointsKdTreeDecoder already works since #1241. The encoded bytes are unchanged.

Encoder memory by dimension:

dimension main, always allocated this change, at most
3 2.3 KB 1.6 KB
64 1 MB 33 KB
255 17 MB 133 KB
2,048 1.1 GB 1.1 MB
6,375 10 GB 3.3 MB

What this means in practice:

  • Three equal points with 2,040 components encode to the same 3,154 bytes with about 1 MB, against 1.07 GB on main.
  • Ordinary point clouds encode with about the same work. Clouds of three components take 5% fewer instructions at most levels and 2% more at level 4, where the compiler inlines std::partition differently into the new walk. Clouds with more components take fewer, because the constructor no longer builds 32 × dimension vectors. Callgrind counts of one encode of random 16-bit points, constructor included, built with clang -O3:
level components points main this change
0 3 50,000 46.1 M 43.6 M
2 3 50,000 50.2 M 47.7 M
4 3 50,000 50.6 M 51.7 M
6 3 50,000 67.6 M 63.7 M
2 6 20,000 25.3 M 24.2 M
4 6 20,000 25.5 M 25.6 M
2 64 2,000 12.7 M 10.7 M

How: the encoder kept a full copy of the node's base and levels for every level of the tree, which is 32 × dimension levels deep. A split changes only one base value and one level. The encoder now keeps a single base and levels vector, records the old values at each split, and restores them when the walk returns to a shallower node. This is the same walk the decoder uses.

Verification:

  • 22,000 generated point clouds over all speeds, 1 to 100 components and four point shapes encode to the same bytes on main and on this change. That probe is not part of the change.
  • Nine existing PointCloudKdTreeEncodingTest cases fail if the restore is broken, so the existing tests cover it.
  • TestKdTreeEncodingOfManyComponents round-trips three equal points with 25 attributes of 255 components, a dimension of 6,375. Equal points are never separated, so the walk goes 11 × 6,375 levels deep, far past the existing tests, which is where the restore does its work. Under AddressSanitizer it peaks at 29 MB with this change and at 12.9 GB on main.
  • draco_tests passes, also under AddressSanitizer.

DynamicIntegerPointsKdTreeEncoder allocates memory quadratic in the
point dimension, up front, in its constructor: a base and a levels
vector for each of the 32 * dimension levels the tree can reach. The
dimension is the total number of components over all attributes. This
change makes the encoder's memory linear in the dimension, the way
DynamicIntegerPointsKdTreeDecoder already works. The encoded bytes are
unchanged.

A split changes one base value and one level. The encoder now keeps a
single base and levels vector, records the old values at each split,
and restores them when the walk returns to a shallower node.

TestKdTreeEncodingOfManyComponents round-trips three equal points with
25 attributes of 255 components. The walk goes 11 * 6375 levels deep,
far past what the existing tests reach, which is where the restore
does its work.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant