Skip to content

Fix poisoning leftover for vector and string ASan annotations #6285

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
amyw-msft wants to merge 9 commits into
base: main
Choose a base branch
from
Open

Fix poisoning leftover for vector and string ASan annotations #6285

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
f3cb662
Modify vector and string annotations to extend 'last valid' point to …
amyw-msft May 13, 2026
17f94e2
Properly mirror xstring/vector changes.
amyw-msft May 14, 2026
4b35e62
Fix formatting
amyw-msft May 14, 2026
7b5cd63
Fix shadowing
amyw-msft May 15, 2026
47fd8ea
Lightly refactor STL annotation logic for vector/string to attempt to…
amyw-msft Jun 3, 2026
c395821
Remove gh6276 specific test since it's handled by other tests now.
amyw-msft Jun 3, 2026
db3ced9
Remove extra spaces, reorder headers, etc.
amyw-msft Jun 4, 2026
4663fc2
Apply clang-format
amyw-msft Jun 4, 2026
d5bd1f8
Remove unused re-alignment code, rewrite comments describing how to u…
amyw-msft Jun 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
157 changes: 95 additions & 62 deletions stl/inc/vector
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -482,81 +482,101 @@ public:

private:
#ifdef _INSERT_VECTOR_ANNOTATION
_CONSTEXPR20 void _Create_annotation() const noexcept {
// Annotates the shadow memory of the valid range
auto& _My_data = _Mypair._Myval2;
_Apply_annotation(_My_data._Myfirst, _My_data._Myend, _My_data._Myend, _My_data._Mylast);

_CONSTEXPR20 bool _Should_annotate() const noexcept {
#if _HAS_CXX20
if (_STD is_constant_evaluated()) {
return false;
}
#endif // _HAS_CXX20

return _Asan_vector_should_annotate;
}

_CONSTEXPR20 void _Remove_annotation() const noexcept {
// Removes the shadow memory annotation of the range
auto& _My_data = _Mypair._Myval2;
_Apply_annotation(_My_data._Myfirst, _My_data._Myend, _My_data._Mylast, _My_data._Myend);
_CONSTEXPR20 const void* _Annotation_begin() const noexcept {
_STL_INTERNAL_CHECK(_Mypair._Myval2._Myfirst != nullptr);
return _STD _Unfancy(_Mypair._Myval2._Myfirst);
}

_CONSTEXPR20 void _Modify_annotation(const difference_type _Count) const noexcept {
// Extends/shrinks the annotated range by _Count
if (_Count == 0) { // nothing to do
// This also avoids calling _Apply_annotation() with null pointers
// when the vector has zero capacity, see GH-2464.
return;
_CONSTEXPR20 const void* _Annotation_at(const size_type _Index) const noexcept {
_STL_INTERNAL_CHECK(_Mypair._Myval2._Myfirst != nullptr);
return _STD _Unfancy(_Mypair._Myval2._Myfirst + _Index);
}
Comment on lines +496 to +504

@amyw-msft amyw-msft Jun 5, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll edit the comments in xmemory and remove any unused code.

The pointers passed in to __sanitizer_annotate_contiguous_container are permitted to be unaligned w.r.t. shadow granularity.

If the beginning is not aligned, ASan still handles the first shadow byte correctly and doesn't need extra over-poisoning logic like the end, because unlike at the end of the buffer (which would require "first X bytes are poisoned" semantics), we do have the semantics to say "first X bytes are valid".

If the end is not aligned, ASan rounds down to the nearest shadow granularity. For this reason, if the STL knows no one will poison the rest of the final shadow byte due to the allocator alignment, the STL will round up before passing the pointer (our 'over-poisoning' technique) to ensure no coverage is lost at the end of the buffer.


_CONSTEXPR20 const void* _Annotation_last() const noexcept {
_STL_INTERNAL_CHECK(_Mypair._Myval2._Mylast != nullptr);
return _STD _Unfancy(_Mypair._Myval2._Mylast);
}

_CONSTEXPR20 const void* _Annotation_last_with_difference(const difference_type _Count) const noexcept {
_STL_INTERNAL_CHECK(_Mypair._Myval2._Mylast != nullptr);
return _STD _Unfancy(_Mypair._Myval2._Mylast + _Count);
}

_CONSTEXPR20 const void* _Annotation_end() const noexcept {
_STL_INTERNAL_CHECK(_Mypair._Myval2._Myend != nullptr);
const void* const _End = _STD _Unfancy(_Mypair._Myval2._Myend);

if constexpr ((_Container_allocation_minimum_asan_alignment<vector>) >= _Asan_granularity) {
// If the allocator alignment is less than or equal to the ASan granularity, then we can
// realign the end forward to maximize the annotation coverage.
return _STD _Get_asan_aligned_after(_End);
} else {
// The allocation may not end on an ASan granularity boundary. In that case, annotating up to _End
// could affect shadow bytes for memory beyond this allocation, so restrict annotations to the largest
// aligned subrange inside the buffer. This may leave the tail of the buffer unannotated.

return _End; // Using unaligned end will snap it to previous ASan granularity boundary.
Comment on lines +520 to +529
}
}

auto& _My_data = _Mypair._Myval2;
_Apply_annotation(_My_data._Myfirst, _My_data._Myend, _My_data._Mylast, _My_data._Mylast + _Count);
_CONSTEXPR20 void _Create_annotation() const noexcept {
if constexpr (!_Disable_ASan_container_annotations_for_allocator<allocator_type>) {
if (!_Should_annotate()) {
return;
}

_CSTD __sanitizer_annotate_contiguous_container(
_Annotation_begin(), _Annotation_end(), _Annotation_end(), _Annotation_last());
}
}

static _CONSTEXPR20 void _Apply_annotation(
pointer _First_, pointer _End_, pointer _Old_last_, pointer _New_last_) noexcept {
_STL_INTERNAL_CHECK(_First_ != nullptr);
_STL_INTERNAL_CHECK(_End_ != nullptr);
_STL_INTERNAL_CHECK(_Old_last_ != nullptr);
_STL_INTERNAL_CHECK(_New_last_ != nullptr);
_CONSTEXPR20 void _Remove_annotation() const noexcept {
if constexpr (!_Disable_ASan_container_annotations_for_allocator<allocator_type>) {
if (!_Should_annotate()) {
return;
}

_CSTD __sanitizer_annotate_contiguous_container(
_Annotation_begin(), _Annotation_end(), _Annotation_last(), _Annotation_end());
}
}

_CONSTEXPR20 void _Modify_annotation(const difference_type _Count) const noexcept {
if constexpr (!_Disable_ASan_container_annotations_for_allocator<allocator_type>) {
#if _HAS_CXX20
if (_STD is_constant_evaluated()) {
if (!_Should_annotate()) {
return;
}
#endif // _HAS_CXX20

if (!_Asan_vector_should_annotate) {
if (_Count == 0) { // nothing to do
// This also avoids calling _Apply_annotation() with null pointers
// when the vector has zero capacity, see GH-2464.
return;
}

const void* const _First = _STD _Unfancy(_First_);
const void* const _End = _STD _Unfancy(_End_);
const void* const _Old_last = _STD _Unfancy(_Old_last_);
const void* const _New_last = _STD _Unfancy(_New_last_);
if constexpr ((_Container_allocation_minimum_asan_alignment<vector>) >= _Asan_granularity) {
// old state:
// [_First, _Old_last) valid
// [_Old_last, _End) poison
// new state:
// [_First, _New_last) valid
// [_New_last, asan_aligned_after(_End)) poison
_CSTD __sanitizer_annotate_contiguous_container(
_First, _STD _Get_asan_aligned_after(_End), _Old_last, _New_last);
} else {
const auto _Aligned = _STD _Get_asan_aligned_first_end(_First, _End);
if (_Aligned._First == _Aligned._End) {
// The buffer does not end at least one shadow memory section; nothing to do.
return;
}
_CSTD __sanitizer_annotate_contiguous_container(
_Annotation_begin(), _Annotation_end(), _Annotation_last(), _Annotation_last_with_difference(_Count));
}
}

const void* const _Old_fixed = _Aligned._Clamp_to_end(_Old_last);
const void* const _New_fixed = _Aligned._Clamp_to_end(_New_last);

// old state:
// [_Aligned._First, _Old_fixed) valid
// [_Old_fixed, _Aligned._End) poison
// [_Aligned._End, _End) valid
// new state:
// [_Aligned._First, _New_fixed) valid
// [_New_fixed, _Aligned._End) poison
// [_Aligned._End, _End) valid
_CSTD __sanitizer_annotate_contiguous_container(_Aligned._First, _Aligned._End, _Old_fixed, _New_fixed);
_CONSTEXPR20 void _Set_annotation(const size_type _Target_size) const noexcept {
if constexpr (!_Disable_ASan_container_annotations_for_allocator<allocator_type>) {
if (!_Should_annotate()) {
return;
}

_CSTD __sanitizer_annotate_contiguous_container(
_Annotation_begin(), _Annotation_end(), _Annotation_last(), _Annotation_at(_Target_size));
}
}

Expand All @@ -568,18 +588,31 @@ private:
constexpr explicit _Asan_extend_guard(vector* _Myvec_, size_type _Target_size_) noexcept
: _Myvec(_Myvec_), _Target_size(_Target_size_) {
_STL_INTERNAL_CHECK(_Myvec != nullptr);
auto& _My_data = _Myvec->_Mypair._Myval2;
_Apply_annotation(_My_data._Myfirst, _My_data._Myend, _My_data._Mylast, _My_data._Myfirst + _Target_size);

if constexpr (!_Disable_ASan_container_annotations_for_allocator<allocator_type>) {
if (!_Myvec->_Should_annotate()) {
return;
}

_CSTD __sanitizer_annotate_contiguous_container(_Myvec->_Annotation_begin(), _Myvec->_Annotation_end(),
_Myvec->_Annotation_last(), _Myvec->_Annotation_at(_Target_size));
}
}

_CONSTEXPR20 ~_Asan_extend_guard() {
if (!_Myvec) { // Operation succeeded, no modification to the shadow memory required.
return;
}

// Shrinks the shadow memory to the current size of the vector.
auto& _My_data = _Myvec->_Mypair._Myval2;
_Apply_annotation(_My_data._Myfirst, _My_data._Myend, _My_data._Myfirst + _Target_size, _My_data._Mylast);
if constexpr (!_Disable_ASan_container_annotations_for_allocator<allocator_type>) {
if (!_Myvec->_Should_annotate()) {
return;
}

// Shrinks the shadow memory to the current size of the vector.
_CSTD __sanitizer_annotate_contiguous_container(_Myvec->_Annotation_begin(), _Myvec->_Annotation_end(),
_Myvec->_Annotation_at(_Target_size), _Myvec->_Annotation_last());
}
}

_CONSTEXPR20 void _Release() noexcept {
Expand Down
134 changes: 78 additions & 56 deletions stl/inc/xmemory
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -813,56 +813,29 @@ _INLINE_VAR constexpr size_t _Asan_granularity_mask = _Asan_granularity - 1;
template <class>
constexpr bool _Disable_ASan_container_annotations_for_allocator = false;

struct _Asan_aligned_pointers {
const void* _First;
const void* _End;

_NODISCARD constexpr const void* _Clamp_to_end(const void* _Mid) const noexcept {
_STL_INTERNAL_CHECK(_Mid >= _First);
if (_Mid > _End) {
return _End;
} else {
return _Mid;
}
}
};

// The way that ASan shadow memory works, each eight byte block of memory ("shadow memory section")
// has a single byte to mark it as either poison or valid.
// Each section has 0 to 8 "valid" bytes followed by poison bytes, so:
// ```
// [ v v v p p p p p ]
// [ v v v p p p p p ] (encoded as 03 for 'first 3 bytes are valid')
// ```
// or
// ```
// [ v v v v v v v v ]
// [ v v v v v v v v ] (encoded as 00 for 'all bytes valid')
// ```
// are okay, but
// ```
// [ p p p p v v v v ]
// ```
// is not.
//
// This function exists to fix up `first` and `end` pointers so that one can call
// `__sanitizer_annotate_contiguous_container`:
//
// - `__sanitizer_annotate_contiguous_container` checks that `first` is aligned to an 8-byte boundary
// - if `end` is not aligned to an 8-byte boundary, `__sanitizer_annotate_contiguous_container` still poisons the
// remaining bytes in the shadow memory section.
//
// Because of the second property, we can only mark poison up to the final aligned address before the true `last`.
// Otherwise, we'd poison the memory _after_ `last` as well.
// For the first property, we can assume that everything before `first` in the shadow memory section is valid
// (since otherwise we couldn't mark `first` valid), and so we just return back the first address in
// `first`'s shadow memory section.
// is not. There are no semantics for describing the first X bytes as being poisoned.
//
// ### Example
//
// ```cpp
// struct alignas(8) cat {
// int meow; // bytes [0, 4)
// int meow; // bytes [0, 4)
// char buffer[16]; // bytes [4, 20)
// int purr; // bytes [20, 24)
// int purr; // bytes [20, 24)
// };
// ```
//
Expand All @@ -883,41 +856,90 @@ struct _Asan_aligned_pointers {
// | meow | buffer | purr |
// [ v v v v p p p p ][ p p p p p p p p ][ v v v v v v v v ]
// sm1 sm2 sm3
// Shadow: 04 fc 00
// ```
//
// We call `aligned = _Get_asan_aligned_first_end(cat.buffer, cat.buffer + 16);`, and we get back
// For this, we can call `__sanitizer_annotate_contiguous_container`.
// Since container poisoning is a set of valid bytes followed by a set of poisoned bytes, we describe this fully to ASan
// by passing a pointer to the first valid byte, the middle pointer (pointing to the first poisoned byte), and the end
// pointer. For the unaligned beginning pointer, ASan will handle the unalignment assuming the bytes to the left are
// valid. For the unaligned end pointer, since there are no 'first X bytes are poisoned' semantics, ASan will just mark
// the entire shadow memory section as valid.
//
// To generate the above shadow memory state, we would call like this:
//
// ```cpp
// aligned = {
// ._First = &cat.meow,
// ._End = cat.buffer + 12,
// };
// __sanitizer_annotate_contiguous_container(
// cat.buffer, // First
// cat.buffer + 16 // End
// cat.buffer + 16, // Old middle pointer
// cat.buffer); // New middle pointer
// ```
//
// Then, we poison as much of buffer as we can via
// Correctly describing the old middle pointer is important since ASan will not check state before the existing shadow
// memory state performing the transformation.
//
// In cases where unalignment must be supported, we can miss coverage due to ASan's 8-byte alignment requirements.
// However, most allocators on Windows will only return 8-byte aligned pointers. This gives us an opportunity to provide
// extra coverage, because if everything is 8-byte aligned, we can extend, instead of reduce, the concept of the end of
// the buffer to the next 8-byte aligned boundary. _Get_asan_aligned_after is a helper function provided for this.
//
// ### Example
//
// Let's say we took the above struct, aligned the members, and shortened the buffer to 12, and left our annotation code
// the same.
// ```cpp
// struct alignas(8) cat {
// int alignas(8) meow; // bytes [0, 4)
// char alignas(8) buffer[12]; // bytes [8, 20)
// int alignas(8) purr; // bytes [24, 28)
// };
//
// ...
//
// __sanitizer_annotate_contiguous_container(
// aligned._First,
// aligned._End,
// cat.buffer,
// aligned._Clamp_to_end(cat.buffer + 16));
// cat.buffer, // First
// cat.buffer + 12 // End
// cat.buffer + 12, // Old middle pointer
// cat.buffer // New middle pointer
// ```
//
// We are allowed to assume that `&cat.meow` is valid, since otherwise `cat.buffer + [0, 4)` could not be valid.
// We cannot poison up to `cat.buffer + 16`, since then `&purr` could not be valid.
// Thus, this results in the shadow memory state from the second example.
_NODISCARD inline _Asan_aligned_pointers _Get_asan_aligned_first_end(
const void* const _First, const void* const _End) noexcept {
return {
reinterpret_cast<const void*>(reinterpret_cast<uintptr_t>(_First) & ~_Asan_granularity_mask),
reinterpret_cast<const void*>(reinterpret_cast<uintptr_t>(_End) & ~_Asan_granularity_mask),
};
}

// When we can assume that the allocator we are using will always align allocations to the 8-byte,
// we can simply push the `_End` pointer to the end of the shadow memory section.
// This is _not_ safe in general (see _Get_asan_aligned_first_end's comment for why).
// Under this, the shadow buffer would look like this:
// ```
// | meow | pad | buffer | pad | purr | pad |
// [ v v v v v v v v ][ p p p p p p p p ][ v v v v v v v v ][ v v v v v v v v ]
// sm1 sm2 sm3 sm4
// Shadow: 00 fc 00 00
// ```
//
// Currently, there is no coverage on the last 4 bytes of the buffer due to the previous rules,
// but when everything is 8-byte aligned, we can use the fact there is padding to extend the poisoning.
//
// ```
// char *end = _Get_asan_aligned_after(cat.buffer + 12);
// __sanitizer_annotate_contiguous_container(
// cat.buffer, // First
// end, // End
// end, // Old middle pointer
// cat.buffer); // New middle pointer
// ```
//
// With this adjustment, the shadow has full coverage of the buffer, and we are not poisoning any memory that will be
// accessed, just padding.
// ```
// | meow | pad | buffer | pad | purr | pad |
// [ v v v v v v v v ][ p p p p p p p p ][ p p p p p p p p ][ v v v v v v v v ]
// sm1 sm2 sm3 sm4
// Shadow: 00 fc fc 00
// ```
//
// This example is conceptually the same for why we can apply this technique in general to any allocator guaranteed to
// only return 8-byte aligned pointers.
//
// Note that caution is warranted with this technique, particularly when differentiating between clearing the
// annotation (e.g. during destruction when we want everything valid) and setting the annotation to be fully valid
// (which would leave poisoning bytes in the padding).
//
_NODISCARD inline const void* _Get_asan_aligned_after(const void* const _End) noexcept {
return reinterpret_cast<const void*>(
(reinterpret_cast<uintptr_t>(_End) + _Asan_granularity_mask) & ~_Asan_granularity_mask);
Expand Down
Loading