Joy Dong a5ed499f6a FlexAttention Benchmark (#139665) ...

1. Add alibi, sliding window, tahn softcap, prefixLM, and document_mask from attn_gym to benchmark.

2. Add comparison to different SDPA backends & FAv2, FAv3, FAKV.

Dependent on https://github.com/pytorch/pytorch/pull/139639

Pull Request resolved: https://github.com/pytorch/pytorch/pull/139665
Approved by: https://github.com/drisspg

2024-12-20 17:52:24 +00:00

..

attention_bias_benchmarks.py

[5/N][Easy] fix typo for usort config in pyproject.toml (kown -> known): sort torch (#127126)

2024-05-27 14:49:57 +00:00

better_transformer_vs_mha_functional.py

Fix unused Python variables outside torch/ and test/ (#136359)

2024-12-11 17:10:23 +00:00

score_mod.py

FlexAttention Benchmark (#139665)

2024-12-20 17:52:24 +00:00

sdp.py

Fix unused Python variables outside torch/ and test/ (#136359)

2024-12-11 17:10:23 +00:00

sdpa.py

[5/N][Easy] fix typo for usort config in pyproject.toml (kown -> known): sort torch (#127126)

2024-05-27 14:49:57 +00:00