Skip to content

[diskann-disk] Decouple buffer memory alignment from disk block_size#984

Draft
doliawu wants to merge 1 commit intomicrosoft:mainfrom
doliawu:decouple-alignments
Draft

[diskann-disk] Decouple buffer memory alignment from disk block_size#984
doliawu wants to merge 1 commit intomicrosoft:mainfrom
doliawu:decouple-alignments

Conversation

@doliawu
Copy link
Copy Markdown
Contributor

@doliawu doliawu commented Apr 28, 2026

DiskSectorGraph previously allocated sectors_data aligned to block_size (typically 4096), conflating the on-disk stride with the reader's hardware/OS memory-placement requirement. Move the requirement where it belongs: declare it on the reader as
AlignedFileReader::MEMORY_ALIGNMENT (default 1; direct-I/O readers override to 512) and have the buffer allocator honor that.

Drop the per-request alignment check from AlignedRead::new — that constraint is the caller's responsibility (DiskSectorGraph's offsets and lengths are already block_size-aligned by construction). Making construction infallible cleans up call sites.

Add PowerOfTwo::new_or_panic for use in const trait-item declarations, since Result::unwrap is not const-stable for this E.

Refresh VertexProvider / VertexProviderFactory docs to drop stale references to ANNWrapper and JetVertexProvider.

Comment thread diskann-disk/src/search/provider/disk_provider.rs
@wuw92
Copy link
Copy Markdown
Contributor

wuw92 commented Apr 28, 2026

For awareness, this PR overlaps with @hildebrandmw's review direction on #965: #965 (review)

His two concrete asks there both touch this PR:

  1. Add PowerOfTwo / AlignedAllocator const aliases → the four PowerOfTwo::new_or_panic(...) callsites added here would collapse to those aliases, and the helper itself becomes unnecessary.
  2. Delete aligned_slice.rs, use Poly directly → the two aligned_slice(...) callsites in disk_sector_graph.rs would migrate accordingly.

Comment thread diskann-quantization/src/num.rs Outdated
DiskSectorGraph previously allocated sectors_data aligned to block_size
(typically 4096), conflating the on-disk stride with the reader's
hardware/OS memory-placement requirement. Move the requirement where
it belongs: declare it on the reader as
AlignedFileReader::MEMORY_ALIGNMENT (default PowerOfTwo::V1; direct-I/O
readers override to PowerOfTwo::V512) and have the buffer allocator
honor that.

Drop the per-request alignment check from AlignedRead::new — that
constraint is the caller's responsibility (DiskSectorGraph's offsets
and lengths are already block_size-aligned by construction). Making
construction infallible cleans up call sites.

Refresh VertexProvider / VertexProviderFactory docs to drop stale
references to ANNWrapper and JetVertexProvider.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@doliawu doliawu force-pushed the decouple-alignments branch from c2da97c to ab9e3fb Compare April 28, 2026 19:54
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Apr 28, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.42%. Comparing base (a1791ce) to head (ab9e3fb).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #984      +/-   ##
==========================================
- Coverage   89.42%   89.42%   -0.01%     
==========================================
  Files         448      448              
  Lines       83743    83713      -30     
==========================================
- Hits        74890    74857      -33     
- Misses       8853     8856       +3     
Flag Coverage Δ
miri 89.42% <100.00%> (-0.01%) ⬇️
unittests 89.26% <100.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
diskann-disk/src/search/provider/disk_provider.rs 90.89% <ø> (ø)
...kann-disk/src/search/provider/disk_sector_graph.rs 96.95% <100.00%> (-0.02%) ⬇️
...rc/search/provider/disk_vertex_provider_factory.rs 95.70% <100.00%> (ø)
...disk/src/utils/aligned_file_reader/aligned_read.rs 100.00% <100.00%> (ø)
...s/aligned_file_reader/linux_aligned_file_reader.rs 82.65% <100.00%> (ø)
...ile_reader/storage_provider_aligned_file_reader.rs 98.36% <100.00%> (ø)

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants