9095a9dfae
[CD] Apply the fix from #162455 to aarch64+cu129 build ( #165794 )
...
When trying to bring cu129 back in https://github.com/pytorch/pytorch/pull/163029 , I mainly looked at https://github.com/pytorch/pytorch/pull/163029 and missed another tweak coming from https://github.com/pytorch/pytorch/pull/162455
I discover this issue when testing aarch64+cu129 builds in https://github.com/pytorch/test-infra/actions/runs/18603342105/job/53046883322?pr=7373 . Surprisingly, there is no test running for aarch64 CUDA build from what I see in 79a37055e7https://github.com/pytorch/pytorch/pull/165794
Approved by: https://github.com/malfet
2025-10-18 04:16:24 +00:00
6dedd34c31
[CD] Skip 12.9 build on Windows ( #165665 )
...
Per title
Pull Request resolved: https://github.com/pytorch/pytorch/pull/165665
Approved by: https://github.com/Camyll , https://github.com/malfet
2025-10-16 19:11:27 +00:00
4400c5d31e
Continue to build nightly CUDA 12.9 for internal ( #163029 )
...
Revert part of https://github.com/pytorch/pytorch/pull/161916 to continue building CUDA 12.9 nightly
Pull Request resolved: https://github.com/pytorch/pytorch/pull/163029
Approved by: https://github.com/malfet
2025-10-11 08:26:47 +00:00
773c6762b8
[CD][CUDA13][NCCL] Fix nccl version typo for cu13 ( #164383 )
...
https://pypi.org/project/nvidia-nccl-cu13/#history does not have 2.27.5 but 2.27.7+.
Companion PR: https://github.com/pytorch/pytorch/pull/164352
Fixes a potential binary breakage due to non-existence of referenced NCCL cu13 version.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/164383
Approved by: https://github.com/tinglvv , https://github.com/Skylion007 , https://github.com/atalman
2025-10-01 21:32:25 +00:00
2610746375
Revert nccl upgrade back to 2.27.5 ( #164352 )
...
Revert https://github.com/pytorch/pytorch/pull/162351 as it breaks H100
Pull Request resolved: https://github.com/pytorch/pytorch/pull/164352
Approved by: https://github.com/atalman , https://github.com/malfet
2025-10-01 15:27:40 +00:00
5504a06e01
[BE]: Update NCCL to 2.28.3 ( #162351 )
...
@eqy New NCCL has some a bunch of bugfixes for features including reducing the number SMs needed by NVLINK collectives as well as some very useful new APIs for SymmetricMemory. Also allows FP8 support for non-reductive operations on pre-sm90 devices.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/162351
Approved by: https://github.com/ezyang , https://github.com/malfet , https://github.com/atalman
2025-09-28 01:38:59 +00:00
f1260c9b9a
[ROCm][CI/CD] upgrade nightly wheels to ROCm 7.0 ( #163937 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/163937
Approved by: https://github.com/jeffdaily
Co-authored-by: Jeff Daily <jeff.daily@amd.com >
2025-09-26 21:42:09 +00:00
bb1d53bc47
[CD] CUDA 13 specific followup changes ( #162455 )
...
Follow up for CUDA 13 bring up https://github.com/pytorch/pytorch/issues/159779
sm50-70 should not be added to sbsa build arch list, as previous archs had no support for arm.
remove platform_machine from PYTORCH_EXTRA_INSTALL_REQUIREMENTS
Pull Request resolved: https://github.com/pytorch/pytorch/pull/162455
Approved by: https://github.com/atalman
2025-09-11 00:03:47 +00:00
8922bbcaab
Use same NVSHMEM version across CUDA builds ( #162206 )
...
#161321 bumped NVSHMEM version to 3.3.24 for CUDA 13, leaving CUDA 12 with 3.3.20.
This PR bumps the NVSHMEM version to 3.3.24 for CUDA 12 as well.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/162206
Approved by: https://github.com/tinglvv , https://github.com/Skylion007
2025-09-09 20:59:50 +00:00
5ccf3ca3ec
Revert "Use same NVSHMEM version across CUDA builds ( #162206 )"
...
This reverts commit 0d9c95cd7ee299e2e8c09df26d395be8775b506b.
Reverted https://github.com/pytorch/pytorch/pull/162206 on behalf of https://github.com/malfet due to Broke lint, see 4dd73e659a/1https://github.com/pytorch/pytorch/pull/162206#issuecomment-3271040521 ))
2025-09-09 14:40:45 +00:00
0d9c95cd7e
Use same NVSHMEM version across CUDA builds ( #162206 )
...
#161321 bumped NVSHMEM version to 3.3.24 for CUDA 13, leaving CUDA 12 with 3.3.20.
This PR bumps the NVSHMEM version to 3.3.24 for CUDA 12 as well.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/162206
Approved by: https://github.com/tinglvv , https://github.com/Skylion007
2025-09-09 08:52:27 +00:00
9c991b63ff
[CD] [aarch64] Add CUDA 12.6 and 12.8 to build matrix, remove 12.9 build ( #162364 )
...
https://github.com/pytorch/pytorch/issues/159779
Add the full CUDA support matrix to sbsa build (12.6, 12.8)
Same arch support as x86 build
Remove 12.9 sbsa build
Pull Request resolved: https://github.com/pytorch/pytorch/pull/162364
Approved by: https://github.com/atalman
2025-09-08 20:00:25 +00:00
145a3a7bda
[CUDA 13][cuDNN] Bump CUDA 13 to cuDNN 9.13.0 ( #162268 )
...
Fixes some `d_qk` != `d_v` cases on Hopper that are broken by cuDNN 9.11-9.12
Pull Request resolved: https://github.com/pytorch/pytorch/pull/162268
Approved by: https://github.com/drisspg , https://github.com/Skylion007
2025-09-06 01:59:03 +00:00
bffc7dd1f3
[CD] Add cuda 13.0 libtorch builds, remove CUDA 12.9 builds ( #161916 )
...
Related to https://github.com/pytorch/pytorch/issues/159779
Adding CUDA 13.0 libtorch builds, followup after https://github.com/pytorch/pytorch/pull/160956
Removing CUDA 12.9 builds, See https://github.com/pytorch/pytorch/issues/159980
Pull Request resolved: https://github.com/pytorch/pytorch/pull/161916
Approved by: https://github.com/jeanschmidt , https://github.com/Skylion007
Co-authored-by: Ting Lu <tingl@nvidia.com >
2025-09-05 07:47:54 +00:00
71992dd805
S390x: build nightly binaries for new pythons ( #161920 )
...
Enable python 3.13t, 3.14 and 3.14t on s390x for nightly binaries
Fixes #161515
Pull Request resolved: https://github.com/pytorch/pytorch/pull/161920
Approved by: https://github.com/malfet
2025-09-03 17:38:38 +00:00
fefee08164
[CD] Add CUDA 13.0 Windows build ( #161663 )
...
Test CUDA 13.0 windows build
Pull Request resolved: https://github.com/pytorch/pytorch/pull/161663
Approved by: https://github.com/malfet , https://github.com/atalman
2025-09-01 15:27:17 +00:00
06c7516994
[BE] Upgrade XPU support package to 2025.2 ( #158733 )
...
Including below changes,
- Add XPU support package 2025.2 build and test in CI for both Linux and Windows
- Keep XPU support package 2025.1 build in CI to ensure no break issue until PyTorch 2.9 release
- Upgrade XPU support package from 2025.1 to 2025.2 in CD for both Linux and Windows
- Rename Linux CI job name & image name to n & n-1
- Update XPU runtime pypi packages dependencies of CD wheels
- Remove deprecated support package version docker image build
Pull Request resolved: https://github.com/pytorch/pytorch/pull/158733
Approved by: https://github.com/EikanWang , https://github.com/atalman
2025-08-27 19:33:38 +00:00
9632f4ea9f
[CD] [aarch64] Add CUDA 13.0 sbsa nightly build ( #161257 )
...
https://github.com/pytorch/pytorch/issues/159779
CUDA SBSA build for CUDA 13.0
1. Supported archs: sm_80 to sm_120. Including support for Thor (sm_110), SPARK (sm_121), GB300 (sm_103).
"This release adds support of SM110 GPUs for arm64-sbsa on Linux." from 13.0 release notes https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.html
2. Use -compress-mode=size for binary size reduction, 13.0 wheel is 2.18 GB, when compared with 12.9 3.28 GB, that is 1.1 GB of savings and ~33.5% smaller.
3. Refactored the libs_to_copy list with common libs, and version_specific_libs.
TODO: add the other CUDA archs in the existing support matrix of x86 to SBSA build as well
Pull Request resolved: https://github.com/pytorch/pytorch/pull/161257
Approved by: https://github.com/nWEIdia , https://github.com/atalman
2025-08-27 14:38:07 +00:00
ae8d319fd4
Update NVSHMEM to 3.3.24 and fix download link ( #161321 )
...
https://github.com/pytorch/pytorch/issues/159779
Update NVSHMEM 3.3.24 for [PyTorch CUDA13 Binary Cannot Be Built with SM_75 with NVSHMEM](https://github.com/pytorch/pytorch/issues/160980 )
Enabled back sm_75 for NVSHMEM
Fixed the NVSHMEM download link for the issue with 3.3.20 download in issue - [[CD] nvshem-3.3.9 wheels for aarch64 is not manylinux2_28 compliant](https://github.com/pytorch/pytorch/issues/160425 )
Todo: Should also enable back build ARM with NVSHMEM since it is compatible with manylinux2_28
Pull Request resolved: https://github.com/pytorch/pytorch/pull/161321
Approved by: https://github.com/Skylion007 , https://github.com/atalman
2025-08-26 13:26:18 +00:00
1a566c4909
Remove Python 3.9 nightly builds ( #161427 )
...
Please see https://github.com/pytorch/pytorch/issues/161167
Pull Request resolved: https://github.com/pytorch/pytorch/pull/161427
Approved by: https://github.com/huydhn
2025-08-25 22:05:40 +00:00
49ff884b1e
Add CUDA 13.0 x86 builds ( #160956 )
...
https://github.com/pytorch/pytorch/issues/159779
CUDA 13.0.0
NVSHMEM 3.3.20
CUDNN 9.12.0.46
Adding x86 linux builds for CUDA 13.
Adding libtorch docker.
Package naming changed for CUDA 13 (removed postfix -cu13 for some packages).
Preparation checklist:
1. Update index https://download.pytorch.org/whl/nightly/cu130 with pypi packages
2. Update packaging name based on https://pypi.org/project/cuda-toolkit/ metadata
Pull Request resolved: https://github.com/pytorch/pytorch/pull/160956
Approved by: https://github.com/atalman
Co-authored-by: atalman <atalman@fb.com >
2025-08-22 11:31:09 +00:00
e1a64b75ff
[CD] Delete full builds ( #161075 )
...
As they are no longer needed for Colab, see https://github.com/googlecolab/colabtools/issues/5508#issuecomment-3200871941 and
[<img width="896" height="128" alt="image" src="https://github.com/user-attachments/assets/a287393c-bde7-4e10-99bf-2e0d66346efe " />
](https://colab.research.google.com/drive/1YJ5Y0xsApXSewM1cQwWQ_AS3A77vytgq )
Fixes https://github.com/pytorch/pytorch/issues/160972
Pull Request resolved: https://github.com/pytorch/pytorch/pull/161075
Approved by: https://github.com/atalman
2025-08-20 19:40:15 +00:00
62db8ec391
windows python 3.14 nightly builds ( #159869 )
...
Related to https://github.com/pytorch/pytorch/issues/156856
Pull Request resolved: https://github.com/pytorch/pytorch/pull/159869
Approved by: https://github.com/malfet , https://github.com/williamwen42
2025-08-19 18:36:16 +00:00
7bd4cfaef4
[BE] Update nvshem dependency to 3.3.20 ( #160458 )
...
Which is manylinux2_28 compatible, even on aarch64 platform
archive contents and URL pattern changed quite drastically between 3.3.9 and 3.3.20, but hopefully it still works.
Package `libnvshmem_host.so.3` into gigantic aarch64+CUDA wheel
Should fix https://github.com/pytorch/pytorch/issues/160425
Pull Request resolved: https://github.com/pytorch/pytorch/pull/160458
Approved by: https://github.com/Skylion007 , https://github.com/kwen2501 , https://github.com/nWEIdia , https://github.com/atalman , https://github.com/tinglvv
2025-08-16 02:00:57 +00:00
c015e53d37
Revert "[BE] Update nvshem dependency to 3.3.20 ( #160458 )"
...
This reverts commit e0488d9f00865fb56c931580c80e099771c6285e.
Reverted https://github.com/pytorch/pytorch/pull/160458 on behalf of https://github.com/wdvr due to need to rerun workflow generation (failing workflow-checks) ([comment](https://github.com/pytorch/pytorch/pull/160458#issuecomment-3193133706 ))
2025-08-16 01:47:42 +00:00
e0488d9f00
[BE] Update nvshem dependency to 3.3.20 ( #160458 )
...
Which is manylinux2_28 compatible, even on aarch64 platform
archive contents and URL pattern changed quite drastically between 3.3.9 and 3.3.20, but hopefully it still works.
Package `libnvshmem_host.so.3` into gigantic aarch64+CUDA wheel
Should fix https://github.com/pytorch/pytorch/issues/160425
Pull Request resolved: https://github.com/pytorch/pytorch/pull/160458
Approved by: https://github.com/Skylion007 , https://github.com/kwen2501 , https://github.com/nWEIdia , https://github.com/atalman , https://github.com/tinglvv
2025-08-16 00:50:13 +00:00
16ce2c15fa
Add python 3.14 support to linux aarch64 builds ( #160788 )
...
Related to https://github.com/pytorch/pytorch/issues/156856
Pull Request resolved: https://github.com/pytorch/pytorch/pull/160788
Approved by: https://github.com/malfet
2025-08-16 00:03:21 +00:00
17de899709
Add py3.14 to macos arm64 ( #160593 )
...
Related to https://github.com/pytorch/pytorch/issues/156856
Pull Request resolved: https://github.com/pytorch/pytorch/pull/160593
Approved by: https://github.com/malfet , https://github.com/Skylion007
2025-08-15 18:52:10 +00:00
d0226719a9
[BE][EZ] Delete remains of split-build logic ( #159990 )
...
Hopefully last piece of https://github.com/pytorch/pytorch/issues/138750
Pull Request resolved: https://github.com/pytorch/pytorch/pull/159990
Approved by: https://github.com/atalman
ghstack dependencies: #159986
2025-08-07 01:59:30 +00:00
26d045bb60
Linux py 3.14 wheel builds ( #157559 )
...
Related to https://github.com/pytorch/pytorch/issues/156856
Pull Request resolved: https://github.com/pytorch/pytorch/pull/157559
Approved by: https://github.com/malfet , https://github.com/albanD
2025-08-04 20:55:19 +00:00
476874b37f
[BE]: Update NCCL to 2.27.5 ( #157108 )
...
Update NCCL to 2.27.5. Minor version, improves Blackwell, Symmem FP8 support, and fixes a bug with MNVVL.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/157108
Approved by: https://github.com/atalman
2025-07-08 15:40:54 +00:00
7275f28045
Fix cuda 12.9 aarch64 GPU builds. Update CUDA_STABLE variable. ( #157630 )
...
This contains 2 fixes that required in main and will need to be cherry-picked to Release 2.8 branch:
1. The PR https://github.com/pytorch/pytorch/pull/155819 missed to include triton change.
2. CUDA STABLE variable needs to be set to 12.8. Updating CUDA stable updates full static build
Pull Request resolved: https://github.com/pytorch/pytorch/pull/157630
Approved by: https://github.com/Skylion007 , https://github.com/jeanschmidt
2025-07-04 18:08:31 +00:00
a6fab82b16
[BE]: Fix NVSHMEM builds, add missing 12.9 dependency and update to latest for 2.8RC ( #157453 )
...
Fixed our bad builds of nvshmem, (we were not building or testing before) and also updates to the latest version. Newest versions has critical support for things that would actually make it useful, like bfloat16 and float16 support.
This is a proper fix for: https://github.com/pytorch/pytorch/pull/157411
Pull Request resolved: https://github.com/pytorch/pytorch/pull/157453
Approved by: https://github.com/kwen2501 , https://github.com/atalman
2025-07-03 22:55:18 +00:00
6a3d00aa3b
Add Windows cuda 12.9.1 build ( #156630 )
...
Without Support for SegmentReduce.cu
Test PR confirmed by Removing SegmentReduce.cu windows build for CUDA 12.9 can succeed
Related to: https://github.com/pytorch/pytorch/issues/156181
Pull Request resolved: https://github.com/pytorch/pytorch/pull/156630
Approved by: https://github.com/malfet
Co-authored-by: Ting Lu <tingl@nvidia.com >
Co-authored-by: Nikita Shulga <2453524+malfet@users.noreply.github.com >
2025-06-24 02:15:49 +00:00
0504480f37
Add CUDA 12.9 libtorch nightly ( #155895 )
...
https://github.com/pytorch/pytorch/issues/155196
with libtorch docker added, we can add the build script
Pull Request resolved: https://github.com/pytorch/pytorch/pull/155895
Approved by: https://github.com/atalman
2025-06-19 13:15:42 +00:00
a317c63d1b
[BE]: Update NCCL to 2.27.3 ( #155233 )
...
Fixes: https://github.com/pytorch/pytorch/issues/155052 and https://github.com/pytorch/pytorch/issues/153517
This upgrade is needed to effectively use those symmetric memory kernels anyway. Also fixes some nasty NCCL bugs.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/155233
Approved by: https://github.com/nWEIdia , https://github.com/kwen2501 , https://github.com/atalman , https://github.com/eqy
2025-06-14 19:20:31 +00:00
4574b39aa4
Revert "[BE]: Sync cusparselt 12.9 with static build and other cuda 12 ( #155709 )"
...
This reverts commit bbbced94a43cf764ddfe719e7d4c161a3992830c.
Reverted https://github.com/pytorch/pytorch/pull/155709 on behalf of https://github.com/clee2000 due to broke lint [GH job link](https://github.com/pytorch/pytorch/actions/runs/15645591737/job/44082402642 ) [HUD commit link](bbbced94a4https://github.com/pytorch/pytorch/pull/155709#issuecomment-2972094849 ))
2025-06-14 01:43:16 +00:00
bbbced94a4
[BE]: Sync cusparselt 12.9 with static build and other cuda 12 ( #155709 )
...
followup for https://github.com/pytorch/pytorch/pull/154980
Pull Request resolved: https://github.com/pytorch/pytorch/pull/155709
Approved by: https://github.com/tinglvv , https://github.com/atalman , https://github.com/nWEIdia , https://github.com/cyyever
2025-06-13 23:10:01 +00:00
344731fb25
Add CUDA 12.9.1 sbsa nightly binaries ( #155819 )
...
https://github.com/pytorch/pytorch/issues/155196
Pull Request resolved: https://github.com/pytorch/pytorch/pull/155819
Approved by: https://github.com/atalman
2025-06-13 18:52:41 +00:00
9cced33c7c
[BE]: Update cudnn to 9.10.2.21 ( #155576 )
...
Update to CUDNN 9.10.2.21
Pull Request resolved: https://github.com/pytorch/pytorch/pull/155576
Approved by: https://github.com/eqy , https://github.com/atalman
2025-06-12 12:50:36 +00:00
f59c76b549
Revert "[BE]: Update cudnn to 9.10.2.21 ( #155576 )"
...
This reverts commit 2d3615f577894c7a117a55e85bb8371bb598ec50.
Reverted https://github.com/pytorch/pytorch/pull/155576 on behalf of https://github.com/malfet due to breaks the same test again (I remember there were a version that adjusted tolerances), see bc3972b80a/1https://github.com/pytorch/pytorch/pull/155576#issuecomment-2964404710 ))
2025-06-11 22:03:45 +00:00
2d3615f577
[BE]: Update cudnn to 9.10.2.21 ( #155576 )
...
Update to CUDNN 9.10.2.21
Pull Request resolved: https://github.com/pytorch/pytorch/pull/155576
Approved by: https://github.com/eqy , https://github.com/atalman
2025-06-11 20:32:07 +00:00
4c3da611c2
Add CUDA 12.9.1 x86 nightly binaries ( #154980 )
...
Adding CUDA 12.9.1 to nightly binaries matrix for linux (x86) builds.
Add sbsa and libtorch build docker images, builds addition will be follow-up PRs.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/154980
Approved by: https://github.com/eqy , https://github.com/atalman
2025-06-11 13:43:17 +00:00
eaceb243df
[BE] Update the XPU support package to 2025.1.3 ( #154346 )
...
Fixes #153632
Pull Request resolved: https://github.com/pytorch/pytorch/pull/154346
Approved by: https://github.com/EikanWang , https://github.com/atalman
2025-06-11 09:46:18 +00:00
7a03b0d2ca
[BE] Remove CUDA 11 artifacts. Fix Check Binary workflow ( #155555 )
...
Please see: https://github.com/pytorch/pytorch/issues/147383
1. Remove CUDA 11 build and test artifacts. One place CUDA 12.4
2. Fix Check Binary Workflow to use Stable Cuda version variable rather then hardcoded one
Pull Request resolved: https://github.com/pytorch/pytorch/pull/155555
Approved by: https://github.com/malfet , https://github.com/Skylion007
2025-06-10 21:32:08 +00:00
0319044e92
[Easy] update pip sources for ROCm in nightly pull tool ( #145685 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/145685
Approved by: https://github.com/ezyang
2025-06-10 08:07:30 +00:00
8153340d10
[CI/CD] Remove CUDA 11.8 builds ( #155509 )
...
This removes CUDA 11.8 from CI/CD
Please see: https://github.com/pytorch/pytorch/issues/147383
TODO: Will followup of cleaning CUDA 11.8 config from scripts
Pull Request resolved: https://github.com/pytorch/pytorch/pull/155509
Approved by: https://github.com/cyyever , https://github.com/huydhn , https://github.com/malfet
2025-06-10 05:16:41 +00:00
3863bbb55b
[BE]: Update cusparselt to 0.7.1 ( #155232 )
...
Needed to support sparse operations on Blackwell, and implements new features for the library. Also optimizes library sizes vs 0.7
Pull Request resolved: https://github.com/pytorch/pytorch/pull/155232
Approved by: https://github.com/nWEIdia , https://github.com/malfet
2025-06-09 18:01:23 +00:00
9656251bb1
Revert "[BE] Update cudnn to 9.10.1.4 ( #155122 )"
...
This reverts commit a14f427db68e54500ef4cd9ed34cb9537263bb74.
Reverted https://github.com/pytorch/pytorch/pull/155122 on behalf of https://github.com/malfet due to Looks like it breaks a bunch of tests, see 36a722e20d/1https://github.com/pytorch/pytorch/pull/155122#issuecomment-2949209801 ))
2025-06-06 13:03:49 +00:00
a14f427db6
[BE] Update cudnn to 9.10.1.4 ( #155122 )
...
Follow up to #152782
Pull Request resolved: https://github.com/pytorch/pytorch/pull/155122
Approved by: https://github.com/malfet , https://github.com/atalman
2025-06-05 16:07:25 +00:00
