frozenleaves/vllm
1
0
Fork 0
mirror of https://github.com/vllm-project/vllm.git synced 2025-10-20 14:53:52 +08:00
Code Issues Packages Projects Releases Wiki Activity

[full_graph] Fix query_start_loc padding (#19321)

Browse Source 
Signed-off-by: Yinghai Lu <yinghai@thinkingmachines.ai>
This commit is contained in:
Yinghai Lu
2025-06-09 06:32:56 -07:00
committed by GitHub
parent c57c9415b1
commit 770e5dcdb8
1 changed files with 4 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
vllm/v1/worker/gpu_model_runner.py
View File

@ -655,7 +655,10 @@ class GPUModelRunner(LoRAModelRunnerMixin):
# Fill unused with -1. Needed for reshape_and_cache
self.seq_lens[num_reqs:].fill_(0)
self.query_start_loc[num_reqs + 1:].fill_(-1)
# Note: pad query_start_loc to be non-decreasing, as kernels
# like FlashAttention requires that
self.query_start_loc[num_reqs + 1:].fill_(
self.query_start_loc_cpu[num_reqs].item())
query_start_loc = self.query_start_loc[:num_reqs + 1]
seq_lens = self.seq_lens[:num_reqs]