7a74294786
[sparse] enable meta tests ( #133379 )
...
The skip for dynamo is no longer needed.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/133379
Approved by: https://github.com/ezyang
2024-08-14 21:58:23 +00:00
a8490a0762
[traced-graph][sparse] propagate sparsity in fx graph ( #131920 )
...
This PR proceeds with implementing the feature request #117188 by generalizing more cases that already work with COO to work with the compressed sparse formats as well.
Feature request:
https://github.com/pytorch/pytorch/issues/117188
Rebranch of older PRs (for history):
https://github.com/pytorch/pytorch/pull/131474
https://github.com/pytorch/pytorch/pull/128549
Pull Request resolved: https://github.com/pytorch/pytorch/pull/131920
Approved by: https://github.com/ezyang
2024-08-05 15:49:53 +00:00
a4ea776881
Add pinned memory support to sparse COO/CSR/CSC/BSR/BSC tensors ( #129645 )
...
As in the title:
To register indices/values of a sparse XYZ tensor with CUDA, the following methods are supported
- `sparse_xyz_tensor(indices, values, pin_memory=True)`
- `sparse_xyz_tensor(indices, values).pin_memory()`
- `sparse_xyz_tensor(indices.pin_memory(), values.pin_memory())`
Fixes https://github.com/pytorch/pytorch/issues/115330
Pull Request resolved: https://github.com/pytorch/pytorch/pull/129645
Approved by: https://github.com/amjames , https://github.com/cpuhrsch , https://github.com/eqy
2024-08-02 08:55:55 +00:00
221350e3a4
Add None return type to init -- tests ( #132352 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/132352
Approved by: https://github.com/ezyang
ghstack dependencies: #132335 , #132351
2024-08-01 15:44:51 +00:00
82242a258a
rm duplicate index_dtype arg ( #130803 )
...
- Remove duplicate `index_dtype` argument for `_test_meta_sparse_compressed` operation.
- Also remove unused `y_v_numel` variable.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/130803
Approved by: https://github.com/soulitzer
2024-07-18 18:30:13 +00:00
ff82e2e7cf
[traced-graph][sparse] propagate sparsity metadata into traced graph ( #117907 )
...
Propagate sparsity metadata from sparse tensors of torch.sparse into the traced graph representation (with would be useful for a JIT backend that supports a "sparse compiler"). This is a first careful attempt, since the actual "meta" feature seem still incomplete for coo and completely lacking for csr/csc/bsr/bsc.
For background see forum postings (with examples):
https://discuss.pytorch.org/t/connecting-pytorch-sparse-tensors-with-mlir/195145
https://dev-discuss.pytorch.org/t/connecting-pytorch-sparse-tensors-with-mlir/1803
And feature request:
https://github.com/pytorch/pytorch/issues/117188
Pull Request resolved: https://github.com/pytorch/pytorch/pull/117907
Approved by: https://github.com/pearu , https://github.com/ezyang
2024-05-23 22:46:46 +00:00
0241ed9331
Fix sparse fake tensors detach ( #125679 )
...
As in the title.
Fixes a bug reported in https://github.com/pytorch/pytorch/pull/117907#discussion_r1589581536
Pull Request resolved: https://github.com/pytorch/pytorch/pull/125679
Approved by: https://github.com/amjames , https://github.com/lezcano
2024-05-09 15:40:57 +00:00
9043ccafdf
Require nnz==0 in sparse meta tensors ( #125221 )
...
As in the title and per discussion starting at https://github.com/pytorch/pytorch/pull/117907#issuecomment-2082426468
Pull Request resolved: https://github.com/pytorch/pytorch/pull/125221
Approved by: https://github.com/amjames , https://github.com/ezyang
2024-05-01 23:41:49 +00:00
16e8431963
Fix hybrid sparse COO tensor conversion to meta tensor ( #125120 )
...
As in the title.
Addresses a bug reported in https://github.com/pytorch/pytorch/pull/117907#issuecomment-2080035379
Pull Request resolved: https://github.com/pytorch/pytorch/pull/125120
Approved by: https://github.com/ezyang , https://github.com/amjames
2024-04-30 03:43:42 +00:00
57a12d9d0f
Add Half support to torch.sparse.addmm for CPU ( #124694 )
...
This PR is to add Half support to torch.sparse.addmm for CPU. It is a requested feature in model DCRNN for Half data type.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/124694
Approved by: https://github.com/pearu
2024-04-24 01:24:01 +00:00
5a1216bb2e
[BE]: Update ruff to 0.4.1 ( #124549 )
...
Update ruff to 0.4.1 .
This version fixes a lot false negatives/false positives, is 20-40% faster, and has various other bug fixes.
Below is a before and after table showing the execution time of ruff lint and ruff format in milliseconds courtesy of https://astral.sh/blog/ruff-v0.4.0
| Repository | Linter (v0.3) | Linter (v0.4) | Formatter (v0.3) | Formatter (v0.4) |
|----------------------------------------------------|---------------|---------------|------------------|------------------|
| [pytorch/pytorch](https://github.com/pytorch/pytorch ) | 328.7 | 251.8 | 351.1 | 274.9 |
Pull Request resolved: https://github.com/pytorch/pytorch/pull/124549
Approved by: https://github.com/ezyang
2024-04-21 14:06:23 +00:00
757daece95
[sparse] add meta support for add operation (and copy) ( #123594 )
...
This is a small step towards #117188
@pearu to review (this was split of #117907 )
Pull Request resolved: https://github.com/pytorch/pytorch/pull/123594
Approved by: https://github.com/pearu , https://github.com/peterbell10
2024-04-12 15:50:30 +00:00
eff1e4899c
Add sparse COO/CSR/CSC/BSR/BSC meta tensor input support to torch.sum ( #121673 )
...
As in the title.
Fixes an issue reported in https://github.com/pytorch/pytorch/pull/117907#issuecomment-1987212514
Pull Request resolved: https://github.com/pytorch/pytorch/pull/121673
Approved by: https://github.com/cpuhrsch
2024-04-06 21:11:22 +00:00
d895192e87
Fix zeros_like on sparse compressed fake tensors ( #123084 )
...
Fixes https://github.com/pytorch/pytorch/pull/117907#issuecomment-2025769663
Adds block compressed sparse tensors support to zeros_like
Pull Request resolved: https://github.com/pytorch/pytorch/pull/123084
Approved by: https://github.com/amjames , https://github.com/peterbell10
2024-04-03 16:11:11 +00:00
656134c38f
[ROCm] enable complex128 in test_addmm_sizes_all_sparse_csr for rocm for trivial (k,n,m) cases ( #120504 )
...
This PR enables `test_addmm_sizes_all_sparse_csr_k_*_n_*_m_*_cuda_complex128` for ROCm for trivial cases (m or n or k = 0)
CUSPARSE_SPMM_COMPLEX128_SUPPORTED also used for `test_addmm_all_sparse_csr` and ` test_sparse_matmul` and both of them are skipped for ROCm by `@skipIfRocm` or `@skipCUDAIf(not _check_cusparse_spgemm_available())`
Pull Request resolved: https://github.com/pytorch/pytorch/pull/120504
Approved by: https://github.com/jithunnair-amd , https://github.com/ezyang
2024-03-12 07:29:57 +00:00
ce2903080c
Add sparse compressed fake tensor support ( #120920 )
...
As in the title.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/120920
Approved by: https://github.com/ezyang
2024-03-04 14:38:45 +00:00
b8e6ca6f76
Add sparse compressed meta tensor support ( #120707 )
...
As in the title.
Replaces https://github.com/pytorch/pytorch/pull/120498 and https://github.com/pytorch/pytorch/pull/120562
Pull Request resolved: https://github.com/pytorch/pytorch/pull/120707
Approved by: https://github.com/ezyang
ghstack dependencies: #120703
2024-03-01 13:28:47 +00:00
8a32a07856
Revert "Add meta device support to sparse compressed tensors ( #120498 )"
...
This reverts commit 5d71ba688563ef491bb28d47c493ec6fc7791da2.
Reverted https://github.com/pytorch/pytorch/pull/120498 on behalf of https://github.com/zou3519 due to broke CI ([comment](https://github.com/pytorch/pytorch/pull/120498#issuecomment-1964491999 ))
2024-02-26 15:59:36 +00:00
5d71ba6885
Add meta device support to sparse compressed tensors ( #120498 )
...
As in the title.
Unblocks https://github.com/pytorch/pytorch/pull/117907#discussion_r1499251745
Pull Request resolved: https://github.com/pytorch/pytorch/pull/120498
Approved by: https://github.com/ezyang
2024-02-25 16:50:17 +00:00
4a37f57c69
Add batched sparse CSR/CSC/BSR/BSC to sparse COO conversion support ( #116206 )
...
As in the title.
Fixes https://github.com/pytorch/pytorch/issues/104868
Pull Request resolved: https://github.com/pytorch/pytorch/pull/116206
Approved by: https://github.com/amjames , https://github.com/lezcano , https://github.com/cpuhrsch
2024-01-07 19:42:02 +00:00
2a87ab4508
Refactor some tests by using TEST_CUDA & TEST_MULTIGPU instead ( #116083 )
...
as https://github.com/pytorch/pytorch/pull/116014#discussion_r1430510759 stated, refactor some tests related.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/116083
Approved by: https://github.com/fduwjj
2024-01-03 08:53:59 +00:00
3fe437b24b
[BE]: Update flake8 to v6.1.0 and fix lints ( #116591 )
...
Updates flake8 to v6.1.0 and fixes a few lints using sed and some ruff tooling.
- Replace `assert(0)` with `raise AssertionError()`
- Remove extraneous parenthesis i.e.
- `assert(a == b)` -> `assert a == b`
- `if(x > y or y < z):`->`if x > y or y < z:`
- And `return('...')` -> `return '...'`
Co-authored-by: Nikita Shulga <2453524+malfet@users.noreply.github.com >
Pull Request resolved: https://github.com/pytorch/pytorch/pull/116591
Approved by: https://github.com/albanD , https://github.com/malfet
2024-01-03 06:04:44 +00:00
3a4fe835cc
Fixed segfault when trying to permute empty tensor ( #116335 )
...
Fixes #116325 .
Fixed unchecked access to first element of `dims` when permuting an empty tensor. Added test to prevent regressions.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/116335
Approved by: https://github.com/Skylion007
2023-12-23 23:14:28 +00:00
194d57dae7
Add values backward support for sparse CSR, CSC, BSR, and BSC tensors ( #115586 )
...
Fixes https://github.com/pytorch/pytorch/issues/107286
Pull Request resolved: https://github.com/pytorch/pytorch/pull/115586
Approved by: https://github.com/cpuhrsch , https://github.com/albanD
2023-12-14 23:09:13 +00:00
193f87857e
[BC breaking] Remove check_sparse_nnz argument of gradcheck ( #115658 )
...
As in title per deprecation plan.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/115658
Approved by: https://github.com/cpuhrsch , https://github.com/soulitzer
2023-12-13 17:34:30 +00:00
670eb83573
Enable test_sparse_addmm for crossref tests ( #115536 )
...
Fixes https://github.com/pytorch/pytorch/issues/97284
Pull Request resolved: https://github.com/pytorch/pytorch/pull/115536
Approved by: https://github.com/cpuhrsch
2023-12-12 17:26:40 +00:00
f98b0f3ebc
Add bfloat16 support to torch.sparse.addmm for CPU ( #115535 )
...
Fixes https://github.com/pytorch/pytorch/issues/73145 .
Pull Request resolved: https://github.com/pytorch/pytorch/pull/115535
Approved by: https://github.com/cpuhrsch
2023-12-12 13:26:33 +00:00
d90d67a146
Added a check to prevent accessing blocksize during Tensor.to_sparse … ( #114905 )
...
…conversion if empty. The main problem was that blocksize is an `optional<ArrayRef>`, so checking for `.has_value()` will be true even if the containing `ArrayRef` is empty.
Fixes #114865 .
Pull Request resolved: https://github.com/pytorch/pytorch/pull/114905
Approved by: https://github.com/malfet
2023-12-01 12:36:15 +00:00
0bd4d1f4ab
Add sparse tensors support to dataloader. ( #112842 )
...
Fixes https://github.com/pytorch/pytorch/issues/106837
Pull Request resolved: https://github.com/pytorch/pytorch/pull/112842
Approved by: https://github.com/cpuhrsch , https://github.com/gokulavasan
2023-11-19 16:05:27 +00:00
26b5e27ace
Add Half support for cummax, cummin, cumprod, logcumsumexp, and prod on CPU ( #112132 )
...
Add Half support for cummax, cummin, cumprod, logcumsumexp, and prod on CPU.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/112132
Approved by: https://github.com/cpuhrsch
2023-11-05 12:31:38 +00:00
0a26e5fd8f
Use 'device' argument in test_sparse.py::TestSparseAnyCUDA::test_as_sparse_gradcheck_* ( #111584 )
...
Argument "device" was missed.
So, "test_sparse.py::TestSparseAnyCUDA::test_as_sparse_gradcheck_*_cuda" was always run on the default device ("cpu") if another default torch device was not configured before.
This fix will probably detect a number of issues on various devices which were previously missed.
Should fix failed rocm CI jobs with "##[error]The action has timed out." and speedup test execution
Pull Request resolved: https://github.com/pytorch/pytorch/pull/111584
Approved by: https://github.com/soulitzer
2023-10-24 00:03:50 +00:00
1f20531939
fall back to eager on NotImplementedError ( #107863 )
...
Follow-up to https://github.com/pytorch/pytorch/pull/107710 :
Help dynamo fall back to eager when compiling unimplemented numpy constructs:
- arrays of strings
- (arg){min, max} for complex types
- various arguments typed as NotImplemented (`np.ones(4, order="F")` etc)
- numpy functions which torch._numpy does not implement
To test, run (we do not implement arrays of strings)
```
import torch
import numpy as np
@torch.compile(fullgraph=False)
def fn():
return np.asarray(["L", "U"])
```
and observe it compiles with fullgraph=False and fails with fullgraph=True
Fixes https://github.com/pytorch/pytorch/issues/107970
Pull Request resolved: https://github.com/pytorch/pytorch/pull/107863
Approved by: https://github.com/ezyang , https://github.com/lezcano
2023-09-07 21:22:20 +00:00
c5ad44be1d
Add torch.sparse.as_sparse_gradcheck decorator of gradcheck that allows gradcheck input function to receive and return sparse tensors ( #107150 )
...
Compared to #104848 , this PR makes a step further: when the enable_sparse_support decorator is applied to `torch.autograd.gradcheck`, the resulting callable is equivalent to `torch.autograd.gradcheck` with an extra feature of supporting functions that can have input sparse tensors or/and can return sparse tensors.
At the same time, the underlying call to `torch.autograd.gradcheck` will operate on strided tensors only. This basically means that torch/autograd/gradcheck.py can be cleaned up by removing the code that deals with sparse tensors.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/107150
Approved by: https://github.com/albanD , https://github.com/amjames , https://github.com/cpuhrsch
ghstack dependencies: #107638 , #107777
2023-08-26 07:24:31 +00:00
e4b38b9ce9
Support torch.sparse_mask on strided input with sparse CSR, CSC, BSR, and BSC mask. ( #107777 )
...
While `input.sparse_mask(mask)` can be defined as `input.mul(ones_like(mask))`, implementing this definition leads to a chicken-and-egg problem because the multiplication of dense and sparse_compressed tensors relies on the `sparse_mask` support.
This PR implements `sparse_mask` support for sparse compressed masks using utility functions from sparse compressed tensor conversions support.
Fixes https://github.com/pytorch/pytorch/issues/107373
Fixes https://github.com/pytorch/pytorch/issues/107370
Pull Request resolved: https://github.com/pytorch/pytorch/pull/107777
Approved by: https://github.com/amjames , https://github.com/cpuhrsch
ghstack dependencies: #107638
2023-08-26 07:24:31 +00:00
fe3309b4b8
Add optional is_coalesced argument to sparse coo tensor factory function. ( #107638 )
...
Resolves https://github.com/pytorch/pytorch/issues/107097
After this PR, instead of
```python
torch.sparse_coo_tensor(indices, values, size)._coalesced_(is_coalesced)
```
(that does not work in the autograd context, see #107097 ), use
```python
torch.sparse_coo_tensor(indices, values, size, is_coalesced=is_coalesced)
```
All sparse coo factory functions that take indices as input support the `is_coalesced` argument.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/107638
Approved by: https://github.com/cpuhrsch
2023-08-26 07:24:29 +00:00
a816aa785b
Implement autograd support for sparse compressed tensor constructors ( #107384 )
...
Fixes https://github.com/pytorch/pytorch/issues/107126
Pull Request resolved: https://github.com/pytorch/pytorch/pull/107384
Approved by: https://github.com/cpuhrsch
ghstack dependencies: #107447
2023-08-21 20:26:39 +00:00
017499b078
Update reduction_ops groupings to include primtorch types ( #107338 )
...
Fixes https://github.com/pytorch/pytorch/issues/107335 . The skips were updated for the _ref ops to match those for eager mode where necessary. Part of breakdown of https://github.com/pytorch/pytorch/pull/104489 .
Pull Request resolved: https://github.com/pytorch/pytorch/pull/107338
Approved by: https://github.com/ezyang
2023-08-19 02:09:11 +00:00
5b7b9e7896
Update binary_ufuncs groupings to include primtorch types ( #107419 )
...
Fixes #107335 . The skips were updated for the _ref ops to match those for eager mode where necessary. Part of breakdown of https://github.com/pytorch/pytorch/pull/104489 .
Pull Request resolved: https://github.com/pytorch/pytorch/pull/107419
Approved by: https://github.com/ezyang
2023-08-18 20:45:36 +00:00
e108f33299
Update distutils.Version to packaging.version due to the deprecation … ( #107207 )
...
Update distutils.Version to packaging.version due to the deprecation warning.
```python
/root/Git.d/pytorch/pytorch/torch/testing/_internal/common_methods_invocations.py:17136: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
active_if=TEST_SCIPY and LooseVersion(scipy.__version__) < "1.4.0"),
/root/Git.d/pytorch/pytorch/torch/testing/_internal/common_methods_invocations.py:17138: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
active_if=TEST_SCIPY and LooseVersion(scipy.__version__) < "1.4.0"),
/root/Git.d/pytorch/pytorch/torch/testing/_internal/common_methods_invocations.py:17140: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
active_if=TEST_SCIPY and LooseVersion(scipy.__version__) < "1.4.0"),
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/107207
Approved by: https://github.com/soulitzer
2023-08-17 11:19:44 +00:00
01069ad4be
sparse.mm.backward: fix for non-contiguous grad values on CPU ( #106127 )
...
Fixes https://github.com/pytorch/pytorch/issues/102493 .
The problem was that the backward implementation assumed inputs to be contiguous.
This might supersede https://github.com/pytorch/pytorch/pull/104520 .
Pull Request resolved: https://github.com/pytorch/pytorch/pull/106127
Approved by: https://github.com/cpuhrsch
2023-07-28 01:25:00 +00:00
73e1455327
[BE] Enable ruff's UP rules and autoformat test/ ( #105434 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/105434
Approved by: https://github.com/albanD
2023-07-19 20:36:06 +00:00
fc2f87b281
Add semi-structured sparse conversions ( #103830 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/103830
Approved by: https://github.com/amjames , https://github.com/jcaip , https://github.com/cpuhrsch
2023-07-13 21:09:09 +00:00
437bc5b1b7
sparse_mask: backward support for sparse lhs (take 2) ( #104341 )
...
This is a copy of https://github.com/pytorch/pytorch/pull/95165 with some bug fixes.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/104341
Approved by: https://github.com/albanD , https://github.com/pearu , https://github.com/amjames
2023-07-03 14:12:44 +00:00
7274582390
Revert "sparse_mask: backward support for sparse lhs ( #95165 )"
...
This reverts commit f090fdf3b49164679fb6316e9ae15e0c4fb3c9eb.
Reverted https://github.com/pytorch/pytorch/pull/95165 on behalf of https://github.com/huydhn due to Sorry for reverting this. I think one of the tests test_sparse.py::TestSparseCUDA::test_sparse_mask_backward_cuda_complex128 is failing on slow gradcheck f090fdf3b4https://github.com/pytorch/pytorch/pull/95165#issuecomment-1604696109 ))
2023-06-23 18:40:15 +00:00
f090fdf3b4
sparse_mask: backward support for sparse lhs ( #95165 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/95165
Approved by: https://github.com/pearu , https://github.com/cpuhrsch
2023-06-23 12:27:27 +00:00
ab8fc41e2f
Support bfloat16 dtype for CUTLASS-based semi-structured sparsity ( #103978 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/103978
Approved by: https://github.com/cpuhrsch
2023-06-22 15:53:27 +00:00
09fdea8564
Fix autograd issue with identity conversions ( #92022 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/92022
Approved by: https://github.com/pearu , https://github.com/mtaaooby , https://github.com/amjames , https://github.com/cpuhrsch
2023-06-21 21:23:03 +00:00
8fc687f7ee
Add activation functions (ReLU and SiLU for now) for structured sparse linear operator ( #101339 )
...
Differential Revision: [D46453476](https://our.internmc.facebook.com/intern/diff/D46453476 )
Pull Request resolved: https://github.com/pytorch/pytorch/pull/101339
Approved by: https://github.com/cpuhrsch
2023-06-16 17:24:59 +00:00
2f893d04c8
Implement adding bias vector into structured sparse linear operator ( #100881 )
...
Differential Revision: [D46453477](https://our.internmc.facebook.com/intern/diff/D46453477 )
Pull Request resolved: https://github.com/pytorch/pytorch/pull/100881
Approved by: https://github.com/cpuhrsch , https://github.com/malfet
2023-06-15 16:16:09 +00:00
45401ef745
Enable float16 and complex32 support for sparse CSR elementwise multiplication operation. ( #100394 )
...
As in the title. In addition, the PR adds float16 addcmul support for CPU device.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/100394
Approved by: https://github.com/amjames , https://github.com/cpuhrsch
2023-06-14 14:42:39 +00:00
