frozenleaves/DeepSpeed
1
0
Fork 0
mirror of https://github.com/deepspeedai/DeepSpeed.git synced 2025-10-20 15:33:51 +08:00
Code Issues Packages Projects Releases Wiki Activity

Update zeropp.md (#3821)

Browse Source 
Set reduce_scatter flag to true
This commit is contained in:
Sam Ade Jacobs
2023-06-27 08:38:52 -07:00
committed by GitHub
parent 203ac9d7ac
commit 25e500e8dd
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/_tutorials/zeropp.md
View File

@ -36,7 +36,7 @@ An example snippet of deepspeed configurations with all three ZeRO++ optimizatio
"zero_optimization": {
"stage": 3,
"reduce_bucket_size": 10000000,
"reduce_scatter": false,
"reduce_scatter": true,
"zero_quantized_weights": true,
"zero_hpz_partition_size": 16,