frozenleaves/vllm
1
0
Fork 0
mirror of https://github.com/vllm-project/vllm.git synced 2025-11-04 17:34:34 +08:00
Code Issues Packages Projects Releases Wiki Activity

[Misc] small improve (#18680)

Browse Source 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
This commit is contained in:
Reid
2025-05-25 21:05:38 +08:00
committed by GitHub
parent 35be8fad62
commit 3a886bd58c
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/features/quantization/bnb.md
View File

@ -15,7 +15,7 @@ pip install bitsandbytes>=0.45.3
vLLM reads the model's config file and supports both in-flight quantization and pre-quantized checkpoint. vLLM reads the model's config file and supports both in-flight quantization and pre-quantized checkpoint.
You can find bitsandbytes quantized models on <https://huggingface.co/models?search=bitsandbytes>. You can find bitsandbytes quantized models on [Hugging Face](https://huggingface.co/models?search=bitsandbytes).
And usually, these repositories have a config.json file that includes a quantization_config section. And usually, these repositories have a config.json file that includes a quantization_config section.
## Read quantized checkpoint ## Read quantized checkpoint