frozenleaves/vllm-dev
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
2 Commits 129 Branches 75 Tags
39161c98a054756cb39edb9f634c6a466410c92d
Go to file
Clone
Open with VS Code Open with VSCodium Open with Intellij IDEA
Download ZIP Download TAR.GZ Download BUNDLE
Woosuk Kwon 39161c98a0 Add OPT
2023-02-09 11:25:37 +00:00
cacheflow/worker/models
Add OPT
2023-02-09 11:25:37 +00:00
README.md
Initial commit
2023-02-09 11:24:15 +00:00

README.md

CacheFlow

Description
A high-throughput and memory-efficient inference and serving engine for LLMs
amdcudadeepseekgpthpuinferenceinferentiallamallmllm-servingllmopsmlopsmodel-servingpytorchqwenrocmtputrainiumtransformerxpu
Readme Apache-2.0 771 MiB
Languages
Python 86%
Cuda 8%
C++ 4.5%
Shell 0.7%
C 0.4%
Other 0.4%