[ROCm] Update to AOTriton 0.8b (#140172)

Notable new features for SDPA operators on AMD systems from AOTriton 0.8b:

1. Nestedtensor support;
2. MQA/GQA support;
3. Restore Efficient attention support for causal=True and seqlen_q != seqlen_k cases;
    + The kernel should use top-left alignment, bottom right alignment will be added later
4. Move gfx1100 (RX7900/W7800/W7900) out of experimental support status.
   However, users are strongly recommended to update to ROCM 6.2.4, notably for
   its firmware updates.

Related unit tests are enabled as well.

Notable related changes from AOTriton 0.8b:

1. AOTriton 0.8b moves the GPU kernel out of libaotriton.so to a separate directory `aotriton.images`;
2. LZMA replaces ZSTD as GPU kernel compression algorithm for better compression ratio: aotriton0.8b (.so + aotriton.images take 350MB) compared to aotriton0.7b .so: 800MB
3. The compression cannot be disabled now, and `liblzma` is hard run-time dependency.
    + Should not be a problem, since `lzma` is part of Python Standard Library

Pull Request resolved: https://github.com/pytorch/pytorch/pull/140172
Approved by: https://github.com/jithunnair-amd, https://github.com/jeffdaily

Co-authored-by: Jithun Nair <37884920+jithunnair-amd@users.noreply.github.com>
This commit is contained in:
Xinya Zhang
2024-12-06 21:45:18 +00:00
committed by PyTorch MergeBot
parent 0a619a212f
commit 424156c26c
14 changed files with 498 additions and 200 deletions

View File

@ -277,8 +277,8 @@ class TestMHADeviceType(TestCase):
def test_native_multihead_self_attention(self, device, dtype, use_nt,
need_weights, average_attn_weights, use_padding, pad_all, fused):
if TEST_WITH_ROCM:
if use_nt:
self.skipTest("ROCM does not support nested tensors for Flash Attention for now.")
if use_nt and use_padding and pad_all:
self.skipTest("Large numerical errors on ROCM to investigate.")
if use_padding and not pad_all and fused:
self.skipTest("Large numerical errors on ROCM to investigate.")
for need_weights in (False, not pad_all):