frozenleaves/vllm
1
0
Fork 0
mirror of https://github.com/vllm-project/vllm.git synced 2025-10-20 14:53:52 +08:00
Code Issues Packages Projects Releases Wiki Activity

[Bugfix] Fix unable to run encoder model when disable_hybrid_kv_cache_manager is true (#24571)

Browse Source 
Signed-off-by: lianyibo <lianyibo1@kunlunit.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
This commit is contained in:
lianyibo
2025-09-17 01:36:58 +08:00
committed by GitHub
parent 567939953b
commit faa7a5daac
1 changed files with 4 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
vllm/v1/core/kv_cache_utils.py
View File

@ -754,6 +754,10 @@ def is_kv_cache_type_uniform(kv_cache_spec: dict[str, KVCacheSpec]) -> bool:
True if all layers have the same type, False otherwise.
"""
if not kv_cache_spec:
# Encoder-only models do not have KV cache, kv_cache_type can be
# regarded as uniform.
return True
try:
kv_cache_spec_values = list(kv_cache_spec.values())
_ = kv_cache_spec_values[0].merge(kv_cache_spec_values)