frozenleaves/vllm
1
0
Fork 0
mirror of https://github.com/vllm-project/vllm.git synced 2025-10-20 14:53:52 +08:00
Code Issues Packages Projects Releases Wiki Activity

[Perf] Reduce peak memory usage of llama (#10339)

Browse Source 
Signed-off-by: andoorve <37849411+andoorve@users.noreply.github.com>
This commit is contained in:
Murali Andoorveedu
2024-11-14 16:38:20 -08:00
committed by GitHub
parent 4a18fd14ba
commit b2e0ad3b59
1 changed files with 2 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
vllm/model_executor/models/llama.py
View File

@ -90,8 +90,8 @@ class LlamaMLP(nn.Module):
self.act_fn = SiluAndMul()
def forward(self, x):
gate_up, _ = self.gate_up_proj(x)
x = self.act_fn(gate_up)
x, _ = self.gate_up_proj(x)
x = self.act_fn(x)
x, _ = self.down_proj(x)
return x