frozenleaves/vllm
1
0
Fork 0
mirror of https://github.com/vllm-project/vllm.git synced 2025-10-20 23:03:52 +08:00
Code Issues Packages Projects Releases Wiki Activity
10,583 Commits 155 Branches 89 Tags
6ac5e06f7c5d4658c9fb119826a92d9910730fb4
Commit Graph

92 Commits

Author SHA1 Message Date
Isotr0py
6ac5e06f7c [Chore] Clean up pytorch helper functions in vllm.utils (#26908) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: isotr0py <2037008807@qq.com>
 2025-10-18 09:48:22 -07:00
iAmir97
1d165d6d85 [Chore] Separate out vllm.utils.mem_utils (#27143) 
Signed-off-by: iAmir97 <Amir.balwel@embeddedllm.com>
Signed-off-by: iAmir97 <71513472+iAmir97@users.noreply.github.com>
Co-authored-by: iAmir97 <Amir.balwel@embeddedllm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-10-18 10:06:59 +00:00
Luka Govedič
bd7157a071 [torch.compile] Enable attention and allreduce fusion without custom ops enabled (#24604) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-10-17 08:10:23 -06:00
Ye (Charlotte) Qi
d32c611f45 [CI/Build] Use 127.0.0.1 instead of localhost in utils (#26750) 
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
 2025-10-14 07:04:00 +00:00
Harry Mellor
8fcaaf6a16 Update Optional[x] -> x | None and Union[x, y] to x | y (#26633) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-12 09:51:31 -07:00
Harry Mellor
d6953beb91 Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
Kunshang Ji
143844fa43 [XPU]Fix xpu spec decoding UTs, avoid using cuda graph (#25847) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2025-09-29 05:15:10 +00:00
Michael Goin
f708bd4904 [CI] Add E2E Blackwell Quantized MoE Test (#25723) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-09-26 12:23:00 -07:00
Matthew Bonanni
3468f17ebe [V0 deprecation] Remove _VLLM_V1 suffixes from attention backend names (#25489) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
 2025-09-25 17:37:50 +00:00
Woosuk Kwon
eb68c2dcd9 [CI] Revert back prepare_prompts and check_answers (#25087) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-17 11:03:16 -07:00
Nick Hill
4db4426404 [CI] Fail subprocess tests with root-cause error (#23795) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-09-10 13:53:21 -07:00
nvjullin
37241077d5 [Misc] Removed force_fp8_e4m3fnuz from FP8LinearOp (#23725) 
Signed-off-by: Julien Lin <jullin@nvidia.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-09-04 09:25:40 -04:00
Michael Goin
906e461ed6 [CI Fix] Pin deepep and pplx tags in tools/ep_kernels/, gate multigpu tests (#23568) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-08-25 18:29:00 -07:00
bigmoyan
582bbe6bd7 [Fix] correct tool_id for kimi-k2 when use tool_choice=required (#21259) 
Co-authored-by: wangzhengtao <wangzhengtao@msh.team>
 2025-08-20 12:59:54 -07:00
afeldman-nm
bf7f470b22 [V1] Logits processors extensibility (#19912) 
Signed-off-by: Andrew Feldman <afeldman@redhat.com>
Signed-off-by: Andrew Feldman <afeld2012@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Andrew Feldman <afeld2012@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-08-16 12:59:17 -07:00
yyweiss
baece8c3d2 [Frontend] Add unix domain socket support (#18097) 
Signed-off-by: <yyweiss@gmail.com>
Signed-off-by: yyw <yyweiss@gmail.com>
 2025-08-08 16:23:44 -07:00
TJian
1ee5ead5f8 [ROCm] [V1] [SpecDec] Enable Speculative Decoding on ROCm V1 Engine (#21496) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-08-07 19:13:17 -07:00
Ilya Markov
6e672daf62 Add FlashInfer allreduce RMSNorm Quant fusion (#21069) 
Signed-off-by: ilmarkov <imarkov@redhat.com>
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: ilmarkov <imarkov@redhat.com>
 2025-07-31 13:58:38 -07:00
Dmitry Rogozhkin
e760fcef22 [XPU] Use spawn with XPU multiprocessing (#20649) 
Signed-off-by: Dmitry Rogozhkin <dmitry.v.rogozhkin@intel.com>
 2025-07-09 00:34:28 -07:00
Luka Govedič
3597b06a4f [CUDA] Enable full cudagraph for FlashMLA (#18581) 
Signed-off-by: luka <luka@neuralmagic.com>
 2025-06-13 18:12:26 +00:00
Simon Mo
02f0c7b220 [Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
Nick Hill
2dbe8c0774 [Perf] API-server scaleout with many-to-many server-engine comms (#17546) 2025-05-30 08:17:00 -07:00
Jee Jee Li
822de7fb94 [Misc] Split model loader (#17712) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-05-07 12:42:26 +08:00
Harry Mellor
d6484ef3c3 Add full API docs and improve the UX of navigating them (#17485) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-03 19:42:43 -07:00
Mark McLoughlin
98d7367b61 [Metrics] Hide deprecated metrics (#15458) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-04-02 07:37:19 -07:00
Ilya Markov
b7b7676d67 [Distributed] Add custom allreduce support for ROCM (#14125) 
Signed-off-by: ilmarkov <imarkov@redhat.com>
Co-authored-by: ilmarkov <imarkov@redhat.com>
 2025-03-31 22:49:12 -07:00
Michael Goin
47e9038d23 Fix cpu offload testing for gptq/awq/ct (#15648) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-03-29 00:29:32 +08:00
Russell Bryant
b877031d80 Remove openvino support in favor of external plugin (#15339) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-03-22 14:06:39 -07:00
vllmellm
2bb0e1a799 [Bugfix][ROCm] running new process using spawn method for rocm in tests. (#14810) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-03-17 11:33:35 +00:00
Sibi
a73e183e36 [Misc] Replace os environ to monkeypatch in test suite (#14516) 
Signed-off-by: sibi <85477603+t-sibiraj@users.noreply.github.com>
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>
 2025-03-16 20:35:57 -07:00
Cyrus Leung
33f227e16b [CI/Build] Use a fixed seed to avoid flaky tests (#14480) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-08 11:30:09 +00:00
Ilya Lavrenov
8ca7a71df7 OpenVINO: added CPU-like conditions (#14338) 
Signed-off-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
 2025-03-06 22:24:49 -08:00
Harry Mellor
cf069aa8aa Update deprecated Python 3.8 typing (#13971) 2025-03-02 17:34:51 -08:00
Jongseok Park
781096e385 Expert Parallelism (EP) Support for DeepSeek V2 (#12583) 2025-02-24 07:33:20 -08:00
youkaichao
cf797aa856 [core] port pynvml into vllm codebase (#12963) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-09 15:00:00 +08:00
Russell Bryant
e489ad7a21 [Misc] Add SPDX-License-Identifier headers to python source files (#12628) 
- **Add SPDX license headers to python source files**
- **Check for SPDX headers using pre-commit**

commit 9d7ef44c3cfb72ca4c32e1c677d99259d10d4745
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:18:24 2025 -0500

    Add SPDX license headers to python source files
    
This commit adds SPDX license headers to python source files as
recommended to
the project by the Linux Foundation. These headers provide a concise way
that is
both human and machine readable for communicating license information
for each
source file. It helps avoid any ambiguity about the license of the code
and can
    also be easily used by tools to help manage license compliance.
    
The Linux Foundation runs license scans against the codebase to help
ensure
    we are in compliance with the licenses of the code we use, including
dependencies. Having these headers in place helps that tool do its job.
    
    More information can be found on the SPDX site:
    
    - https://spdx.dev/learn/handling-license-info/
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

commit 5a1cf1cb3b80759131c73f6a9dddebccac039dea
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:36:32 2025 -0500

    Check for SPDX headers using pre-commit
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

---------

Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-02-02 11:58:18 -08:00
youkaichao
bd82872211 [ci]try to fix flaky multi-step tests (#11894) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-09 14:47:29 +00:00
Joe Runde
2d1b9baa8f [Bugfix] Fix request cancellation without polling (#11190) 2024-12-17 12:26:32 -08:00
Cyrus Leung
d1e21a979b [CI/Build] Split up VLM tests (#11083) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-12 06:18:16 +08:00
Flávia Béo
aa9078fa03 Adds method to read the pooling types from model's files (#9506) 
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Max de Bayser <mbayser@br.ibm.com>
 2024-11-07 08:42:40 +00:00
Isotr0py
a5fda50a10 [CI/Build] Fix large_gpu_mark reason (#10070) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-06 08:50:37 +00:00
youkaichao
566cd27797 [torch.compile] rework test plans (#9866) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-10-31 22:20:17 -07:00
Yongzao
00d91c8a2c [CI/Build] Simplify exception trace in api server tests (#9787) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2024-10-30 14:52:05 -07:00
Alex Brooks
cc98f1e079 [CI/Build] VLM Test Consolidation (#9372) 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
 2024-10-30 09:32:17 -07:00
wangshuai09
4e2d95e372 [Hardware][ROCM] using current_platform.is_rocm (#9642) 
Signed-off-by: wangshuai09 <391746016@qq.com>
 2024-10-28 04:07:00 +00:00
Cyrus Leung
f085995a7b [CI/Build] Remove unnecessary fork_new_process (#9484) 2024-10-21 19:47:29 -07:00
Cyrus Leung
051eaf6db3 [Model] Add user-configurable task for models that support both generation and embedding (#9424) 2024-10-18 11:31:58 -07:00
Kuntai Du
81ede99ca4 [Core] Deprecating block manager v1 and make block manager v2 default (#8704) 
Removing the block manager v1. This is the initial piece of prefix-caching-centric design. In order to achieve prefix-caching-centric design, we need to simplify the code path so that we only use v2 block manager (which has much higher performance on prefix caching).
 2024-10-17 11:38:15 -05:00
sroy745
f3a507f1d3 [Core] Add an environment variable which needs to be set explicitly to allow BlockSpaceManagerV1 (#9149) 2024-10-10 14:17:17 +08:00
youkaichao
c8627cd41b [ci][test] use load dummy for testing (#9165) 2024-10-09 00:38:40 -07:00