Pieter Noordhuis a0263ec047 Make DistributedDataParallel use new reducer (#18953) ...

Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/18953

This removes Python side bucketing code from DistributedDataParallel
and replaces it with calls to the new C++ based bucketing and reducing
code. To confirm this is working well, we ran a test with both the
previous implementation and the new implementation, and confirmed they
are numerically equivalent.

Performance is improved by a couple percent or more, including the
single machine multiple GPU runs.

Closes #13273.

Reviewed By: mrshenli

Differential Revision: D14580911

fbshipit-source-id: 44e76f8b0b7e58dd6c91644e3df4660ca2ee4ae2

2019-04-15 12:44:38 -07:00

..

bottleneck

Implement torch.util.bottleneck (#5216)

2018-03-23 17:27:35 -04:00

cpp

Support Kwargs in C++ Function/Method calls (#19086)

2019-04-13 08:42:11 -07:00

cpp_extensions

Change is_variable() to check existence of AutogradMeta, and remove is_variable_ (#19139)

2019-04-11 14:03:33 -07:00

custom_operator

Create Object that represents a Module (#18469)

2019-04-05 18:58:52 -07:00

data

Implement kthvalue in ATen (#17544)

2019-03-01 19:00:10 -08:00

error_messages

[pep8] Fix most lint automatically with autopep8

2017-01-28 01:15:51 +01:00

expect

First class modules in the compiler, round 2 (#19167)

2019-04-11 13:55:48 -07:00

ffi/src

Add ffi utils for user C extensions

2016-09-29 09:35:56 -07:00

onnx

Fix onnx ints (#19102)

2019-04-12 12:01:14 -07:00

optim

Turn on F401: Unused import warning. (#18598)

2019-03-30 09:01:17 -07:00

test_module

Turn on F401: Unused import warning. (#18598)

2019-03-30 09:01:17 -07:00

common_cuda.py

Support replicating multi-GPU modules (#18687)

2019-04-03 14:43:07 -07:00

common_methods_invocations.py

Allow Tensor lists to show up in symbolic differentiable graphs. (#16784)

2019-04-10 18:16:20 -07:00

common_nn.py

Turn on F401: Unused import warning. (#18598)

2019-03-30 09:01:17 -07:00

common_utils.py

Added numpy conversion (#18505)

2019-04-03 07:28:24 -07:00

expecttest.py

Add support for inline expect tests. (#12825)

2018-10-22 19:29:04 -07:00

HowToWriteTestsUsingFileCheck.md

Add a document 'How to Write Tests Using FileCheck' (#19005)

2019-04-08 12:12:30 -07:00

run_test.py

Basic implementation of QRelu in C10 (#19091)

2019-04-11 08:47:56 -07:00

test_autograd.py

Modify Cholesky derivative (#19116)

2019-04-15 12:16:55 -07:00

test_c10d.py

Make DistributedDataParallel use new reducer (#18953)

2019-04-15 12:44:38 -07:00

test_cpp_extensions.py

Enable autograd to recognize the XLA backend as one providing multiple devices (#17847)

2019-03-20 13:58:36 -07:00

test_cuda_primary_ctx.py

Lazily load libcuda libnvrtc from c++ (#17317)

2019-02-22 13:51:45 -08:00

test_cuda.py

Free all blocks with outstanding events on OOM-retry (#19222)

2019-04-15 11:29:27 -07:00

test_dataloader.py

Disable flaky test_proper_exit test. (#18950)

2019-04-05 09:49:54 -07:00

test_distributed.py

Fix BN tests for >= 8 GPU test environments (#19049)

2019-04-09 08:08:05 -07:00

test_distributions.py

Modify Cholesky derivative (#19116)

2019-04-15 12:16:55 -07:00

test_docs_coverage.py

Add torch.__config__.show(), reporting detailed version of all libraries. (#18579)

2019-04-09 11:13:24 -07:00

test_expecttest.py

Add support for inline expect tests. (#12825)

2018-10-22 19:29:04 -07:00

test_indexing_cuda.py

Speed-up "advanced" indexing operations (#13420)

2018-11-27 15:23:59 -08:00

test_indexing.py

Added indexing for bool tensors and bool Indices (#18583)

2019-04-03 12:47:26 -07:00

test_jit_fuser.py

JIT Layernorm fusion (#18266)

2019-04-12 14:38:31 -07:00

test_jit.py

Make sure that any of the future versions can load and execute older models. (#19174)

2019-04-15 10:49:31 -07:00

test_mkldnn.py

Add MKL-DNN Tensor (#17748)

2019-04-08 21:41:38 -07:00

test_multiprocessing_spawn.py

Binding for prctl(PR_SET_PDEATHSIG) (#14491)

2018-11-29 20:09:19 -08:00

test_multiprocessing.py

fixes multiprocessing serialization for integer nn.Parameter (#18639)

2019-04-01 17:15:42 -07:00

test_namedtuple_return_api.py

make test_jit_fuser runnable

2019-04-09 12:36:25 -07:00

test_nccl.py

Shard all of tests based on how many tests exist. (#13160)

2018-10-26 18:20:34 -07:00

test_nn.py

Enable working ROCm tests (#19169)

2019-04-12 21:51:10 -07:00

test_numba_integration.py

Fix Windows test CI

2019-03-13 09:22:46 -07:00

test_optim.py

Adds Cyclical Learning Rate and Momentum (#18001)

2019-03-27 19:56:04 -07:00

test_quantized.py

Basic implementation of QRelu in C10 (#19091)

2019-04-11 08:47:56 -07:00

test_quantizer.py

Implement Observer pass on simple model and validate stats (#18848)

2019-04-07 09:17:14 -07:00

test_sparse.py

Fix sparse mm for ROCm (#18985)

2019-04-07 18:16:16 -07:00

test_thd_distributed.py

Turn on F401: Unused import warning. (#18598)

2019-03-30 09:01:17 -07:00

test_torch.py

Add Quantized Backend (#18546)

2019-04-12 12:55:49 -07:00

test_type_hints.py

Turn on F401: Unused import warning. (#18598)

2019-03-30 09:01:17 -07:00

test_type_info.py

Allow converting char tensor to numpy; add [fi]info.min (#15046)

2018-12-24 09:11:24 -08:00

test_utils.py

add/move a few apis in torch.hub (#18758)

2019-04-10 23:10:39 -07:00