frozenleaves/DeepSpeed
1
0
Fork 0
mirror of https://github.com/deepspeedai/DeepSpeed.git synced 2025-10-20 23:46:02 +08:00
Code Issues Packages Projects Releases Wiki Activity

[docs] fix broken read-the-docs build (#2075)

Browse Source
This commit is contained in:
Jeff Rasley
2022-07-06 14:23:18 -07:00
committed by GitHub
parent 9305916d6b
commit 559fb8e515
3 changed files with 8 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/code-docs/source/conf.py
View File

@ -20,7 +20,7 @@ copyright = '2020, Microsoft'
author = 'Microsoft'
# The full version, including alpha/beta/rc tags
release = '0.3.0'
release = '0.6'
master_doc = 'index'

8
docs/code-docs/source/memory.rst
View File

@ -7,9 +7,9 @@ API To Estimate Memory Usage
ZeRO2:
.. autofunction:: deepspeed.runtime.zero.stage2.estimate_zero2_model_states_mem_needs_all_live
.. autofunction:: deepspeed.runtime.zero.stage_1_and_2.estimate_zero2_model_states_mem_needs_all_live
.. autofunction:: deepspeed.runtime.zero.stage2.estimate_zero2_model_states_mem_needs_all_cold
.. autofunction:: deepspeed.runtime.zero.stage_1_and_2.estimate_zero2_model_states_mem_needs_all_cold
Examples:
@ -18,7 +18,7 @@ Let's try a 3B model with just 1 node with 8 gpus, using live model:
.. code-block:: bash
python -c 'from transformers import AutoModel; \
from deepspeed.runtime.zero.stage2 import estimate_zero2_model_states_mem_needs_all_live; \
from deepspeed.runtime.zero.stage_1_and_2 import estimate_zero2_model_states_mem_needs_all_live; \
model = AutoModel.from_pretrained("t5-3b"); \
estimate_zero2_model_states_mem_needs_all_live(model, num_gpus_per_node=8, num_nodes=1)'
Estimated memory needed for params, optim states and gradients for a:
@ -34,7 +34,7 @@ faster as we don't need to load the model.
.. code-block:: bash
python -c 'from deepspeed.runtime.zero.stage2 import estimate_zero2_model_states_mem_needs_all_cold; \
python -c 'from deepspeed.runtime.zero.stage_1_and_2 import estimate_zero2_model_states_mem_needs_all_cold; \
estimate_zero2_model_states_mem_needs_all_cold(total_params=2851e6, num_gpus_per_node=8, num_nodes=1)'
Estimated memory needed for params, optim states and gradients for a:
HW: Setup with 1 node, 8 GPUs per node.

3
requirements/requirements-readthedocs.txt
View File

@ -1,5 +1,8 @@
docutils<0.18
hjson
packaging
psutil
py-cpuinfo
pydantic
torch
tqdm