frozenleaves/pytorch
1
0
Fork 0
mirror of https://github.com/pytorch/pytorch.git synced 2025-10-20 21:14:14 +08:00
Code Issues Packages Projects Releases Wiki Activity

add c10d dynamic loading mechanism and unit test (#28068)

Browse Source 
Summary:
The original behavior of pytorch c10d only supports built-in c10d backends, such as
nccl/gloo/mpi. This patch is used to extend the c10d capability to support dynamically
loading 3rd party communication libraries which are derived from ProcessGroup base class.

related RFC is in: https://github.com/pytorch/pytorch/issues/27955

Through this way, user just need specify a 3rd party c10d backend name when invoking
torch.distributed.init_process_group(). The proposed logic will try to load corresponding
c10d backend cpp extension automatically. as for how to develop a new 3rd party c10d backend
through cpp extension, pls refer to test/cpp_extensions/cpp_c10d_extension.cpp
Pull Request resolved: https://github.com/pytorch/pytorch/pull/28068

Differential Revision: D19174838

Pulled By: agolynski

fbshipit-source-id: 3409a504a43ce7260e6f9d1207c00e87471fac62
This commit is contained in:
Feng Tian
2020-04-02 15:44:40 -07:00
committed by Facebook GitHub Bot
parent 2a4ca70832
commit 762270c51f
8 changed files with 361 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
test/test_determination.py
View File

@ -92,6 +92,7 @@ class DeterminationTest(unittest.TestCase):
self.assertEqual(
self.determined_tests(["torch/utils/cpp_extension.py"]),
[
"distributed/test_distributed",
"test_cpp_extensions_aot_ninja",
"test_cpp_extensions_aot_no_ninja",
"test_determination",