frozenleaves/vllm
1
0
Fork 0
mirror of https://github.com/vllm-project/vllm.git synced 2025-10-20 14:53:52 +08:00
Code Issues Packages Projects Releases Wiki Activity

Update comment (#2934)

Browse Source
This commit is contained in:
Ronen Schaffer
2024-02-22 04:18:37 +02:00
committed by GitHub
parent 8fbd84bf78
commit d7f396486e
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
benchmarks/benchmark_serving.py
View File

@ -7,7 +7,7 @@ On the server side, run one of the following commands:
--disable-log-requests
(TGI backend)
./launch_hf_server.sh <your_model>
./launch_tgi_server.sh <your_model> <max_batch_total_tokens>
On the client side, run:
python benchmarks/benchmark_serving.py \