frozenleaves/pytorch
1
0
Fork 0
mirror of https://github.com/pytorch/pytorch.git synced 2025-10-20 21:14:14 +08:00
Code Issues Packages Projects Releases Wiki Activity

Fix typos under torch/distributed directory (#95638)

Browse Source 
This PR fixes typos in comments and messages of `.py` files under torch/distributed directory

Pull Request resolved: https://github.com/pytorch/pytorch/pull/95638
Approved by: https://github.com/usamah1, https://github.com/H-Huang, https://github.com/kit1980
This commit is contained in:
Kazuaki Ishizaki
2023-03-27 21:13:40 +00:00
committed by PyTorch MergeBot
parent 8313becefa
commit 35fd5c548e
51 changed files with 105 additions and 105 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
torch/distributed/run.py
View File

@ -97,7 +97,7 @@ setup on different ports to avoid port conflicts (or worse, two jobs being merge
as a single job). To do this you have to run with ``--rdzv-backend=c10d``
and specify a different port by setting ``--rdzv-endpoint=localhost:$PORT_k``.
For ``--nodes=1``, its often convenient to let ``torchrun`` pick a free random
port automatically instead of manually assgining different ports for each run.
port automatically instead of manually assigning different ports for each run.
::
@ -325,7 +325,7 @@ utility
5. This module only supports homogeneous ``LOCAL_WORLD_SIZE``. That is, it is assumed that all
nodes run the same number of local workers (per role).
6. ``RANK`` is NOT stable. Between restarts, the local workers on a node can be assgined a
6. ``RANK`` is NOT stable. Between restarts, the local workers on a node can be assigned a
different range of ranks than before. NEVER hard code any assumptions about the stable-ness of
ranks or some correlation between ``RANK`` and ``LOCAL_RANK``.