frozenleaves/vllm
1
0
Fork 0
mirror of https://github.com/vllm-project/vllm.git synced 2025-10-20 14:53:52 +08:00
Code Issues Packages Projects Releases Wiki Activity

Correct alignment in the seq_len diagram. (#5592)

Browse Source 
Co-authored-by: Liqian Chen <liqian.chen@deeplang.ai>
This commit is contained in:
Charles Riggins
2024-06-18 00:05:33 +08:00
committed by GitHub
parent 9333fb8eb9
commit 9e74d9d003
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
vllm/attention/backends/flash_attn.py
View File

@ -83,7 +83,7 @@ class FlashAttentionMetadata(AttentionMetadata):
# |---------------- N iteration ---------------------|
# |- tokenA -|......................|-- newTokens ---|
# |---------- context_len ----------|
# |-------------------- seq_len ----------------------|
# |-------------------- seq_len ---------------------|
# |-- query_len ---|
# Maximum query length in the batch. None for decoding.