Bruce Chang fa0db212e7 shrink_group implementation to expose ncclCommShrink API (#164518) ...

Closes #164529

To expose the new [ncclCommShrink](https://docs.nvidia.com/deeplearning/nccl/user-guide/docs/api/comms.html#ncclcommshrink) API to PyTorch.

This is useful when you need to exclude certain GPUs or nodes from a collective operation, for example in fault tolerance scenarios or when dynamically adjusting resource utilization.

For more info:  [Shrinking a communicator](https://docs.nvidia.com/deeplearning/nccl/user-guide/docs/usage/communicators.html#shrinking-a-communicator)

Pull Request resolved: https://github.com/pytorch/pytorch/pull/164518
Approved by: https://github.com/kwen2501

2025-10-19 18:00:08 +00:00

..

_composable

[FSDP2] provide public API to share cuda streams across roots (#165024)

2025-10-14 17:50:46 +00:00

_pycute

[DeviceMesh] Prefer using _layout over _mesh for all sorts of things (#165554)

2025-10-17 17:57:51 +00:00

_shard

Enable all SIM rules except disabled ones (#164645)

2025-10-17 07:27:11 +00:00

_tools

[1/N] Apply UP035 rule in tests (#163947)

2025-09-29 01:42:01 +00:00

algorithms

Enable all PIE rules on ruff (#165814)

2025-10-18 07:36:18 +00:00

bin

…

checkpoint

Enable all PIE rules on ruff (#165814)

2025-10-18 07:36:18 +00:00

elastic

Enable all PIE rules on ruff (#165814)

2025-10-18 07:36:18 +00:00

flight_recorder

[fr] Skip the dtype check for some one to all or all to one collective (#163839)

2025-09-25 16:02:06 +00:00

fsdp

Enable all PIE rules on ruff (#165814)

2025-10-18 07:36:18 +00:00

launcher

Support XPU in --nproc-per-node option to torchrun (#159474)

2025-09-12 08:32:04 +00:00

nn/jit

PEP585 update - test (#145176)

2025-01-22 04:48:28 +00:00

optim

Use torch.testing.test_close instead of torch.testing.test_allclose (#164539)

2025-10-03 14:39:10 +00:00

pipelining

Enable all SIM rules except disabled ones (#164645)

2025-10-17 07:27:11 +00:00

rpc

[BE][6/6] fix typos in test/ (test/distributed/) (#157640)

2025-07-11 14:09:37 +00:00

tensor

Enable more DTensor tests in local tensor mode and fix more integration issues (#165716)

2025-10-18 23:33:24 +00:00

_test_template.py

[C10D] Fix spelling of MultiProcContinuousTest (#160892)

2025-08-19 20:17:19 +00:00

argparse_util_test.py

…

logging_utils.py

shrink_group implementation to expose ncclCommShrink API (#164518)

2025-10-19 18:00:08 +00:00

test_aten_comm_compute_reordering.py

Overlap scheduler improvements (#165318)

2025-10-15 21:58:47 +00:00

test_backends.py

API to retrieve default distributed backend from device (#140536)

2024-11-22 11:01:53 +00:00

test_c10d_common.py

Fix invalid f-strings (#164112)

2025-09-30 04:17:13 +00:00

test_c10d_functional_native.py

Native matmul (#157743)

2025-10-14 04:22:30 +00:00

test_c10d_gloo.py

[C10d][Gloo] Enable complex datatype support in ProcessGroupGloo (#156633)

2025-09-05 21:24:36 +00:00

test_c10d_logger.py

add device generalisation support for distributed tests (#152471)

2025-06-20 07:35:42 +00:00

test_c10d_nccl.py

shrink_group implementation to expose ncclCommShrink API (#164518)

2025-10-19 18:00:08 +00:00

test_c10d_object_collectives.py

Update test_c10d_object_collectives.py with DistributedTestBase class (#145056)

2025-02-13 03:57:59 +00:00

test_c10d_ops_nccl.py

[ROCm] Enabling several UTs (#161715)

2025-09-09 15:49:21 +00:00

test_c10d_pypg.py

[c10d] block_current_stream: correctness fixes (#158757)

2025-07-21 22:23:44 +00:00

test_c10d_spawn_gloo.py

PEP585 update - test (#145176)

2025-01-22 04:48:28 +00:00

test_c10d_spawn_nccl.py

PEP585 update - test (#145176)

2025-01-22 04:48:28 +00:00

test_c10d_spawn_ucc.py

PEP585 update - test (#145176)

2025-01-22 04:48:28 +00:00

test_c10d_spawn.py

Add __main__ guards to distributed tests (#154628)

2025-06-04 14:39:57 +00:00

test_c10d_ucc.py

[BE][PYFMT] migrate PYFMT for test/[a-h]*/ to ruff format (#144555)

2025-06-24 04:53:54 +00:00

test_collective_utils.py

[C10D] add _summarize_ranks util (#160284)

2025-08-28 00:17:53 +00:00

test_composability.py

[PP] move FSDP reduce scatters to end of step (#165106)

2025-10-12 13:28:02 +00:00

test_compute_comm_reordering.py

Native matmul (#157743)

2025-10-14 04:22:30 +00:00

test_control_collectives.py

[BE][PYFMT] migrate PYFMT for test/[a-h]*/ to ruff format (#144555)

2025-06-24 04:53:54 +00:00

test_cupy_as_tensor.py

[CI][CUDA] Add periodic b200 distributed job (#159323)

2025-10-16 21:54:04 +00:00

test_data_parallel.py

[ROCm] Enable several distributed UTs (#164390)

2025-10-03 19:52:51 +00:00

test_device_mesh.py

Enable all PIE rules on ruff (#165814)

2025-10-18 07:36:18 +00:00

test_dist2.py

Enable more DTensor tests in local tensor mode and fix more integration issues (#165716)

2025-10-18 23:33:24 +00:00

test_distributed_spawn.py

Remove NO_MULTIPROCESSING_SPAWN checks (#146705)

2025-02-28 05:53:19 +00:00

test_dynamo_distributed.py

Native matmul (#157743)

2025-10-14 04:22:30 +00:00

test_fake_pg.py

Fix periodic debug tests failing due to FakeProcessGroup things (#165479)

2025-10-15 18:16:08 +00:00

test_functional_api.py

Revert "Fix decorators skipping NCCL tests (#158846)"

2025-09-10 20:51:31 +00:00

test_inductor_collectives.py

[inductor][bucketing] Fx collectives bucketing of multiple dtypes (#162470)

2025-10-16 18:31:43 +00:00

test_launcher.py

Fix unused Python variables in test/[a-d]* (#134665)

2024-12-13 22:13:12 +00:00

test_local_tensor.py

LocalTensor (#164537)

2025-10-12 20:06:41 +00:00

test_multi_threaded_pg.py

add device generalization support for distributed tests (#156796)

2025-07-16 09:37:03 +00:00

test_nccl.py

Remove old ROCm skip conditions in tests (#164058)

2025-09-29 03:00:58 +00:00

test_nvshmem_triton.py

[NVSHMEM][Triton] Fix NVSHMEM triton test for wacky world sizes (#165704)

2025-10-17 19:33:26 +00:00

test_nvshmem.py

[SymmMem] Multi-root tile reduction (#164757)

2025-10-08 17:28:00 +00:00

test_p2p_ipc.py

[ROCm] Enabling several UTs (#161715)

2025-09-09 15:49:21 +00:00

test_pg_wrapper.py

[BE][6/6] fix typos in test/ (test/distributed/) (#157640)

2025-07-11 14:09:37 +00:00

test_run.py

154849 Add support to handle IGUSR1 and SIGUSR2 in multiprocessing (#160690)

2025-09-09 22:23:06 +00:00

test_serialization.py

distributed/serialization: support zero sized tensors (#164198)

2025-09-30 08:11:29 +00:00

test_store.py

[Fix] Adding missing f prefixes to formatted strings [4/N] (#164068)

2025-09-29 04:07:07 +00:00

test_symmetric_memory.py

[CI][CUDA] Add periodic b200 distributed job (#159323)

2025-10-16 21:54:04 +00:00