[ROCm] Fix GPU memory fault in batched_unary_embeddings backward - #6261
[ROCm] Fix GPU memory fault in batched_unary_embeddings backward#6261avbokovoy wants to merge 5 commits into
Conversation
|
@q10 has imported this pull request. If you are a Meta employee, you can view this in D118492251. |
sylvesterkaczmarek
left a comment
There was a problem hiding this comment.
The eager paths now reject malformed offsets, but batched_unary_embeddings_forward_meta still floor-divides (offsets.sym_numel() - 1) / T without enforcing the new T > 0 / divisibility invariant. FakeTensor/export can therefore accept an input shape that eager now rejects and report a misleading B. Could the Meta implementation mirror the new validation?
|
|
|
Problem
batched_unary_embeddings_test.py::test_gpu crashed with a GPU memory access fault on ROCm. The permute sub-test passed
offsets of length 8 (7 segments) for a T=2 module, violating the op's offsets.numel() == T*B+1 contract. Forward silently truncated
B=3 and dropped the trailing segment; backward walked all 7 segments, deriving an out-of-range table id t=2 that indexed
grad_weight out of bounds.
Fix