mirror of
https://github.com/pytorch/pytorch.git
synced 2025-10-20 21:14:14 +08:00
[ROCm] Update to AOTriton 0.8b (#140172)
Notable new features for SDPA operators on AMD systems from AOTriton 0.8b: 1. Nestedtensor support; 2. MQA/GQA support; 3. Restore Efficient attention support for causal=True and seqlen_q != seqlen_k cases; + The kernel should use top-left alignment, bottom right alignment will be added later 4. Move gfx1100 (RX7900/W7800/W7900) out of experimental support status. However, users are strongly recommended to update to ROCM 6.2.4, notably for its firmware updates. Related unit tests are enabled as well. Notable related changes from AOTriton 0.8b: 1. AOTriton 0.8b moves the GPU kernel out of libaotriton.so to a separate directory `aotriton.images`; 2. LZMA replaces ZSTD as GPU kernel compression algorithm for better compression ratio: aotriton0.8b (.so + aotriton.images take 350MB) compared to aotriton0.7b .so: 800MB 3. The compression cannot be disabled now, and `liblzma` is hard run-time dependency. + Should not be a problem, since `lzma` is part of Python Standard Library Pull Request resolved: https://github.com/pytorch/pytorch/pull/140172 Approved by: https://github.com/jithunnair-amd, https://github.com/jeffdaily Co-authored-by: Jithun Nair <37884920+jithunnair-amd@users.noreply.github.com>
This commit is contained in:
committed by
PyTorch MergeBot
parent
0a619a212f
commit
424156c26c
@ -277,8 +277,8 @@ class TestMHADeviceType(TestCase):
|
||||
def test_native_multihead_self_attention(self, device, dtype, use_nt,
|
||||
need_weights, average_attn_weights, use_padding, pad_all, fused):
|
||||
if TEST_WITH_ROCM:
|
||||
if use_nt:
|
||||
self.skipTest("ROCM does not support nested tensors for Flash Attention for now.")
|
||||
if use_nt and use_padding and pad_all:
|
||||
self.skipTest("Large numerical errors on ROCM to investigate.")
|
||||
if use_padding and not pad_all and fused:
|
||||
self.skipTest("Large numerical errors on ROCM to investigate.")
|
||||
for need_weights in (False, not pad_all):
|
||||
|
Reference in New Issue
Block a user