bd72e28314
[1/N] Change #include <c10/util/Optional.h> to #include <optional> ( #128301 )
...
Fixes #ISSUE_NUMBER
Pull Request resolved: https://github.com/pytorch/pytorch/pull/128301
Approved by: https://github.com/ezyang
2024-06-14 23:21:01 +00:00
00999fd8b1
Prefer flip over index_select ( #126783 )
...
It's faster and has a lower memory footprint in eager.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/126783
Approved by: https://github.com/Skylion007 , https://github.com/albanD
ghstack dependencies: #114471
2024-05-29 09:10:25 +00:00
ed327876f5
[codemod] c10:optional -> std::optional ( #126135 )
...
Generated by running the following from PyTorch root:
```
find . -regex ".*\.\(cpp\|h\|cu\|hpp\|cc\|cxx\)$" | grep -v "build/" | xargs -n 50 -P 4 perl -pi -e 's/c10::optional/std::optional/'
```
`c10::optional` is just an alias for `std::optional`. This removes usages of that alias in preparation for eliminating it entirely.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/126135
Approved by: https://github.com/Skylion007 , https://github.com/malfet , https://github.com/albanD , https://github.com/aaronenyeshi
2024-05-14 19:35:51 +00:00
761a7b84ba
[Dynamo] Fix alias issue with respect to wrapped numbers ( #124731 ) ( #124774 )
...
This PR fixes an issue presented when calling `aten.alias(int)` raises a TypeError.
```python
import torch
import torch.autograd.forward_ad as fwAD
def f(x):
return 4312491 * x
device = "cpu"
with torch._subclasses.fake_tensor.FakeTensorMode():
with fwAD.dual_level():
x = torch.randn(3, device=device)
y = torch.ones_like(x)
dual = fwAD.make_dual(x, y)
f(dual)
```
The test case above illustrates this bug.
1) `4312491` turns into a tensor that is a wrapped number
2) Forward mode AD calls `aten::alias` internally
3) The wrapped number (`4312491`) becomes a python integer
4) `aten.alias(int)` raises a `TypeError`
Pull Request resolved: https://github.com/pytorch/pytorch/pull/124774
Approved by: https://github.com/albanD , https://github.com/zou3519
2024-04-30 14:11:46 +00:00
2fe672b146
compile: ban mutations on non-compositional uses of as_strided ( #122502 )
...
Fixes https://github.com/pytorch/pytorch/issues/104505
I was originally going to ban all usages of as_strided + mutation in functionalization. But I'm pretty sure that as_strided + mutation is fine when we are calling as_strided on a base tensor.
So in this PR I added a slightly more conservative check: if we see an as_strided + mutation, where the input to an as_strided was **another** view op, then I error loudly in functionalization and link to the github issue above (in case anyone runs into this in the real world)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/122502
Approved by: https://github.com/ezyang , https://github.com/albanD
2024-04-12 01:12:23 +00:00
7ffad9ab04
Use out-of-place version of put inside take_backward ( #123268 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/123268
Approved by: https://github.com/zou3519
ghstack dependencies: #122211 , #122212 , #122213
2024-04-05 03:29:11 +00:00
7fb2d69282
[PT2] - Fix cat backwards wrapping on symints ( #121527 )
...
Summary:
Wrapping was comparing Symint and ints forcing a guard. Rewrite it with TORCH_GUARD_SIZE_OBLIVIOUS
```
[trainer0|0]: File "<invalid>", line 0, in THPEngine_run_backward(_object*, _object*, _object*)
[trainer0|0]: File "<invalid>", line 0, in torch::autograd::python::PythonEngine::execute(std::vector<torch::autograd::Edge, std::allocator<torch::autograd::Edge>> const&, std::vector<at::Tensor, std::allocator<at::Tensor>> const&, bool, bool, bool, std::vector<torch::autograd::Edge, std::allocator<torch::autograd::Edge>> const&)
[trainer0|0]: File "<invalid>", line 0, in torch::autograd::Engine::execute(std::vector<torch::autograd::Edge, std::allocator<torch::autograd::Edge>> const&, std::vector<at::Tensor, std::allocator<at::Tensor>> const&, bool, bool, bool, std::vector<torch::autograd::Edge, std::allocator<torch::autograd::Edge>> const&)
[trainer0|0]: File "<invalid>", line 0, in torch::autograd::python::PythonEngine::execute_with_graph_task(std::shared_ptr<torch::autograd::GraphTask> const&, std::shared_ptr<torch::autograd::Node>, torch::autograd::InputBuffer&&)
[trainer0|0]: File "<invalid>", line 0, in torch::autograd::Engine::execute_with_graph_task(std::shared_ptr<torch::autograd::GraphTask> const&, std::shared_ptr<torch::autograd::Node>, torch::autograd::InputBuffer&&)
[trainer0|0]: File "<invalid>", line 0, in torch::autograd::Engine::thread_main(std::shared_ptr<torch::autograd::GraphTask> const&)
[trainer0|0]: File "<invalid>", line 0, in torch::autograd::Engine::evaluate_function(std::shared_ptr<torch::autograd::GraphTask>&, torch::autograd::Node*, torch::autograd::InputBuffer&, std::shared_ptr<torch::autograd::ReadyQueue> const&)
[trainer0|0]: File "<invalid>", line 0, in torch::autograd::Node::operator()(std::vector<at::Tensor, std::allocator<at::Tensor>>&&)
[trainer0|0]: File "<invalid>", line 0, in torch::autograd::generated::CatBackward0::apply(std::vector<at::Tensor, std::allocator<at::Tensor>>&&)
[trainer0|0]: File "<invalid>", line 0, in torch::autograd::generated::details::cat_tensors_backward(at::Tensor const&, std::vector<std::vector<c10::SymInt, std::allocator<c10::SymInt>>, std::allocator<std::vector<c10::SymInt, std::allocator<c10::SymInt>>>> const&, std::vector<c10::ScalarType, std::allocator<c10::ScalarType>> const&, long)
[trainer0|0]: File "<invalid>", line 0, in c10::operator==(c10::SymInt const&, int)
[trainer0|0]: File "<invalid>", line 0, in c10::SymBool::guard_bool(char const*, long) const
[trainer0|0]: File "<invalid>", line 0, in torch::impl::PythonSymNodeImpl::guard_bool(char const*, long)
```
Test Plan: Regular CI
Differential Revision: D54667300
Pull Request resolved: https://github.com/pytorch/pytorch/pull/121527
Approved by: https://github.com/ezyang
2024-03-19 18:03:02 +00:00
0a7666801d
SymIntify prod_backward ( #120776 )
...
Fixes https://github.com/pytorch/pytorch/issues/120608
Signed-off-by: Edward Z. Yang <ezyang@meta.com >
Pull Request resolved: https://github.com/pytorch/pytorch/pull/120776
Approved by: https://github.com/albanD
2024-02-29 20:05:22 +00:00
87c6cd2f00
[1/N] Replace std::tie with structural binding ( #119774 )
...
This PR replaces some std::tie calls with structural binding from C++17. This not only makes the code more compact, but also has some performance gain.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/119774
Approved by: https://github.com/albanD , https://github.com/malfet
2024-02-14 09:25:04 +00:00
08657b82f5
Reduce scope of dispatching in logcumsumexp_backward ( #119397 )
...
Everything inside the `AT_DISPATCH` block is being compiled 5 times,
so it makes sense to limit it to the only line that uses `scalar_t` which is
the `numeric_limits` query.
Also a small optimization, instead of computing `grad.log()` and `(-grad).log()`
we can compute `grad.abs().log()` which is 2 pointwise ops instead of 3.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/119397
Approved by: https://github.com/lezcano , https://github.com/albanD
2024-02-08 15:09:22 +00:00
34fe850d00
SymInt'ify sparse_compressed_tensor ( #107903 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/107903
Approved by: https://github.com/cpuhrsch
ghstack dependencies: #115586
2023-12-17 17:36:20 +00:00
194d57dae7
Add values backward support for sparse CSR, CSC, BSR, and BSC tensors ( #115586 )
...
Fixes https://github.com/pytorch/pytorch/issues/107286
Pull Request resolved: https://github.com/pytorch/pytorch/pull/115586
Approved by: https://github.com/cpuhrsch , https://github.com/albanD
2023-12-14 23:09:13 +00:00
165f4f6ccf
[PyTorch] Redirect c10::optional to std::optional ( #101995 )
...
We have C++17 now!
I am intentionally dropping the `c10::optional<c10::ArrayRef>` size optimization. It was intended to improve dispatch, but thanks to D34602980 / #70864 we don't use `optional<ArrayRef>` in function arguments anymore anyway.
Differential Revision: [D46079028](https://our.internmc.facebook.com/intern/diff/D46079028/ )
Pull Request resolved: https://github.com/pytorch/pytorch/pull/101995
Approved by: https://github.com/malfet , https://github.com/Skylion007 , https://github.com/ezyang
2023-11-30 02:46:41 +00:00
77f16eb00c
Fix prod double backward when there are 2+ zeros ( #113969 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/113969
Approved by: https://github.com/albanD
2023-11-21 01:32:10 +00:00
8c4812be80
Replace expect_int with guard_int ( #113921 )
...
The idea is that instead of erroring, we will just specialize at these sites.
Fixes https://github.com/pytorch/pytorch/issues/113142
Signed-off-by: Edward Z. Yang <ezyang@meta.com >
Pull Request resolved: https://github.com/pytorch/pytorch/pull/113921
Approved by: https://github.com/zou3519
2023-11-20 21:27:48 +00:00
070b2d3cff
cholesky_solve_backward: speed up using output_mask ( #112981 )
...
Introduces a faster path for `cholesky_solve_backward` when the gradient with respect to the cholesky factor isn't required.
Adds test coverage in `test_linalg.py`.
# Example
## Setup
```py
import torch
torch.set_num_threads(1)
mat = torch.randn(500, 1000)
mat = mat @ mat.T
L = torch.linalg.cholesky(mat, upper=False)
rhs = torch.randn(500, 1)
rhs.requires_grad = True
sol = torch.cholesky_solve(rhs, L, upper=False).sum(dim=0)
```
## Before
```
%timeit torch.autograd.grad(sol, rhs, retain_graph=True)
2.61 ms ± 18.4 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
```
## After
```
%timeit torch.autograd.grad(sol, rhs, retain_graph=True)
109 µs ± 3.42 µs per loop (mean ± std. dev. of 7 runs, 10,000 loops each)
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/112981
Approved by: https://github.com/lezcano
2023-11-16 18:30:57 +00:00
325e0fdfdd
Enable masked_scatter_backward for inductor ( #109642 )
...
masked_scatter_backward was previously implemented as a
CompositeExplicitAutograd, which involved a decomp that calls
masked_select, and masked_select in general produces data-dependent
shapes that inductor doesn't support. But masked_scatter_backward
reshapes the return value of masked_select such that the end result has
a static shape again.
I have converted masked_scatter_backward into an aten op to avoid this
issue.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/109642
Approved by: https://github.com/ezyang
ghstack dependencies: #108170
2023-11-09 01:27:57 +00:00
c84c86f018
SymIntify convolution ( #111599 )
...
Signed-off-by: Edward Z. Yang <ezyang@meta.com >
Pull Request resolved: https://github.com/pytorch/pytorch/pull/111599
Approved by: https://github.com/wanchaol , https://github.com/bdhirsh
2023-10-21 03:03:20 +00:00
17348b0f51
Implement split_with_sizes backward for NT ( #110647 )
...
Needed internally. Note that `split_with_sizes()` for NT is currently supported only on `dim=-1`.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/110647
Approved by: https://github.com/cpuhrsch , https://github.com/soulitzer
ghstack dependencies: #110646
2023-10-06 18:44:22 +00:00
48240ec62e
Make unbind() overrideable for NT subclass ( #110646 )
...
Reland of #109122 . Fixed the memory leak by not saving the outputs of `unbind()` for backward. Rather, the NT sizes are saved so undefined grads can replaced with zeros of the correct size.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/110646
Approved by: https://github.com/soulitzer , https://github.com/cpuhrsch
2023-10-06 18:44:22 +00:00
a3c1e3c95c
Generalize toAccumulateType() ( #108248 )
...
Trying to address this comment: https://github.com/pytorch/pytorch/pull/106666#discussion_r1297397554
Pull Request resolved: https://github.com/pytorch/pytorch/pull/108248
Approved by: https://github.com/kulinseth , https://github.com/albanD
2023-10-02 16:34:36 +00:00
b083058e45
Revert "Make unbind() overrideable for NT subclass ( #109122 )"
...
This reverts commit f5a23ca78d13c5e536f5062325c815c50be5f4c2.
Reverted https://github.com/pytorch/pytorch/pull/109122 on behalf of https://github.com/PaliC due to breaking slow tests ([comment](https://github.com/pytorch/pytorch/pull/109122#issuecomment-1741555305 ))
2023-09-29 22:41:56 +00:00
f5a23ca78d
Make unbind() overrideable for NT subclass ( #109122 )
...
Goal: avoid making unbind composite implicit so we can override it within `__torch_dispatch__()` for the NT subclass.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/109122
Approved by: https://github.com/cpuhrsch , https://github.com/soulitzer
2023-09-28 01:26:22 +00:00
51d2d825ab
[3/N] apply clang-tidy in torch/csrc/autograd ( #109368 )
...
This PR applies clang-tidy fixes in torch/csrc/autograd/FunctionsManual.cpp. There are also other fixes.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/109368
Approved by: https://github.com/Skylion007
2023-09-17 07:26:59 +00:00
a14d30d8d1
[1/N] apply clang-tidy in torch/csrc/autograd ( #109032 )
...
This PR begins a new series of patches for enabling clang-tidy checks in torch/csrc/augograd
Pull Request resolved: https://github.com/pytorch/pytorch/pull/109032
Approved by: https://github.com/albanD , https://github.com/Skylion007
2023-09-15 23:28:43 +00:00
d465d6a838
[inductor] scatter_reduce - skip .item() in backward if GradMode is not enabled ( #107353 )
...
Repeats #106429 for scatter_reduce so that the backward will pass for PT2. The .item() call is only needed to make double-backward work, which isn't supported anyway for PT2; so an easy fix is to just skip the .item() call if we know we won't need double-backward.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/107353
Approved by: https://github.com/eellison
2023-08-18 07:17:29 +00:00
4bfc55ba8b
[MPS] Enable forward test for renorm ( #106666 )
...
Enabled forward test for renorm
Pull Request resolved: https://github.com/pytorch/pytorch/pull/106666
Approved by: https://github.com/kulinseth , https://github.com/albanD
2023-08-17 16:46:06 +00:00
393e9eed90
[inductor] modify index_reduce to pass opinfo tests ( #106429 )
...
1. add a python meta registration, to fix an issue with the forward pass. The problem was that previously, the C++ meta registration calls [numel()](7b14a14e27/aten/src/ATen/native/TensorAdvancedIndexing.cpp (L329)https://github.com/pytorch/pytorch/pull/106429
Approved by: https://github.com/zou3519
2023-08-10 18:14:00 +00:00
437bc5b1b7
sparse_mask: backward support for sparse lhs (take 2) ( #104341 )
...
This is a copy of https://github.com/pytorch/pytorch/pull/95165 with some bug fixes.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/104341
Approved by: https://github.com/albanD , https://github.com/pearu , https://github.com/amjames
2023-07-03 14:12:44 +00:00
c4a6f86062
[pt2] add metas for max_unpool2d and max_unpool3d ( #103821 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/103821
Approved by: https://github.com/Skylion007 , https://github.com/Chillee
2023-07-01 01:33:35 +00:00
5cf3a99013
sampled_addmm: backward performance improvements ( #103544 )
...
No need to do double `sparse_mask`, let's squash everything into one call!
This PR exercises https://github.com/pytorch/pytorch/pull/103750 , so here is an autogened code for the backward pass.
```
at::Tensor sparse_sampled_addmm(c10::DispatchKeySet ks, const at::Tensor & self, const at::Tensor & mat1, const at::Tensor & mat2, const at::Scalar & beta, const at::Scalar & alpha) {
auto& self_ = unpack(self, "self", 0);
auto& mat1_ = unpack(mat1, "mat1", 1);
auto& mat2_ = unpack(mat2, "mat2", 2);
[[maybe_unused]] auto _any_requires_grad = compute_requires_grad( self, mat1, mat2 );
std::shared_ptr<SparseSampledAddmmBackward0> grad_fn;
if (_any_requires_grad) {
grad_fn = std::shared_ptr<SparseSampledAddmmBackward0>(new SparseSampledAddmmBackward0(), deleteNode);
grad_fn->set_next_edges(collect_next_edges( self, mat1, mat2 ));
grad_fn->alpha = alpha;
grad_fn->beta = beta;
if (grad_fn->should_compute_output(2)) {
grad_fn->mat1_ = SavedVariable(mat1, false);
}
if (grad_fn->should_compute_output(1)) {
grad_fn->mat2_ = SavedVariable(mat2, false);
}
grad_fn->self_ = SavedVariable(self, false);
}
```
As you can see, we do not save tensors unless needed.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/103544
Approved by: https://github.com/soulitzer
2023-06-28 08:49:54 +00:00
567b5e5b28
Multioutput backward formula: allow conditional guards against saving ( #103750 )
...
Multi-output backward formulas break the ability of autogen to decide which variables have to be stored in a graph.
This PR introduces a macro `wrap_opt_if` which could be used to hint autogen about variable interdependence.
For example, the following code is being generated for `_trilinear` with this modification:
```
at::Tensor _trilinear(c10::DispatchKeySet ks, const at::Tensor & i1, const at::Tensor & i2, const at::Tensor & i3, at::IntArrayRef expand1, at::IntArrayRef expand2, at::IntArrayRef expand3, at::IntArrayRef sumdim, int64_t unroll_dim) {
auto& i1_ = unpack(i1, "i1", 0);
auto& i2_ = unpack(i2, "i2", 1);
auto& i3_ = unpack(i3, "i3", 2);
[[maybe_unused]] auto _any_requires_grad = compute_requires_grad( i1, i2, i3 );
[[maybe_unused]] auto _any_has_forward_grad_result = (isFwGradDefined(i1) || isFwGradDefined(i2) || isFwGradDefined(i3));
std::shared_ptr<TrilinearBackward0> grad_fn;
if (_any_requires_grad) {
grad_fn = std::shared_ptr<TrilinearBackward0>(new TrilinearBackward0(), deleteNode);
grad_fn->set_next_edges(collect_next_edges( i1, i2, i3 ));
grad_fn->expand1 = expand1.vec();
grad_fn->expand2 = expand2.vec();
grad_fn->expand3 = expand3.vec();
if (grad_fn->should_compute_output(1) || grad_fn->should_compute_output(2)) {
grad_fn->i1_ = SavedVariable(i1, false);
}
if (grad_fn->should_compute_output(0) || grad_fn->should_compute_output(2)) {
grad_fn->i2_ = SavedVariable(i2, false);
}
if (grad_fn->should_compute_output(0) || grad_fn->should_compute_output(1)) {
grad_fn->i3_ = SavedVariable(i3, false);
}
grad_fn->sumdim = sumdim.vec();
}
```
with the following backward modifications:
```
- name: _trilinear(Tensor i1, Tensor i2, Tensor i3, int[] expand1, int[] expand2, int[] expand3, int[] sumdim, int unroll_dim=1) -> Tensor
- i1, i2, i3: _trilinear_backward(grad, i1, i2, i3, expand1, expand2, expand3, sumdim, grad_input_mask)
+ i1, i2, i3: "_trilinear_backward(grad,
+ wrap_opt_if(i1, grad_input_mask[1] || grad_input_mask[2]),
+ wrap_opt_if(i2, grad_input_mask[0] || grad_input_mask[2]),
+ wrap_opt_if(i3, grad_input_mask[0] || grad_input_mask[1]),
+ expand1, expand2, expand3, sumdim, grad_input_mask)"
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/103750
Approved by: https://github.com/soulitzer
2023-06-27 15:12:09 +00:00
d4a98280a8
[Reland] Use missing-prototypes in torch_cpu ( #104138 )
...
This PR enables Wmissing-prototypes in torch_cpu except some generated cpp files and the mps and metal,vulkan backends and caffe2 sources.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/104138
Approved by: https://github.com/albanD , https://github.com/malfet
2023-06-26 22:53:43 +00:00
7274582390
Revert "sparse_mask: backward support for sparse lhs ( #95165 )"
...
This reverts commit f090fdf3b49164679fb6316e9ae15e0c4fb3c9eb.
Reverted https://github.com/pytorch/pytorch/pull/95165 on behalf of https://github.com/huydhn due to Sorry for reverting this. I think one of the tests test_sparse.py::TestSparseCUDA::test_sparse_mask_backward_cuda_complex128 is failing on slow gradcheck f090fdf3b4https://github.com/pytorch/pytorch/pull/95165#issuecomment-1604696109 ))
2023-06-23 18:40:15 +00:00
f090fdf3b4
sparse_mask: backward support for sparse lhs ( #95165 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/95165
Approved by: https://github.com/pearu , https://github.com/cpuhrsch
2023-06-23 12:27:27 +00:00
b5594f7df0
Revert "Use missing-prototypes in torch_cpu ( #103725 )"
...
This reverts commit 716b3b893d2826f1e47ab5321f082b48c66c8c92.
Reverted https://github.com/pytorch/pytorch/pull/103725 on behalf of https://github.com/osalpekar due to Broke caffe2 builds due. More info at [D46920675](https://www.internalfb.com/diff/D46920675 ) ([comment](https://github.com/pytorch/pytorch/pull/103725#issuecomment-1603129273 ))
2023-06-22 18:30:31 +00:00
716b3b893d
Use missing-prototypes in torch_cpu ( #103725 )
...
This PR enables Wmissing-prototypes in torch_cpu except some generated cpp files and the mps and metal backends.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/103725
Approved by: https://github.com/albanD
2023-06-21 13:19:55 +00:00
056d92e2a0
sparse.mm backward: performance improvements ( #94991 )
...
`torch.sparse.mm` - faster and without syncs in "most" cases.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/94991
Approved by: https://github.com/Skylion007 , https://github.com/pearu , https://github.com/cpuhrsch
2023-06-12 20:57:29 +00:00
2b3d955ffd
[pt2] add meta and SymInt support for linalg_matrix_exp ( #102945 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/102945
Approved by: https://github.com/lezcano
2023-06-09 22:45:16 +00:00
4bda4a7e4d
[pt2] add meta for lu_unpack ( #102937 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/102937
Approved by: https://github.com/lezcano
2023-06-06 08:06:53 +00:00
1c2dfdf30c
Add renorm forward-ad ( #100798 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/100798
Approved by: https://github.com/soulitzer
2023-06-05 20:25:35 +00:00
995ac703cd
[pt2] add SymInt support for linalg.pinv ( #102367 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/102367
Approved by: https://github.com/lezcano
2023-05-27 11:10:47 +00:00
da3aba1e46
Revert "[pt2] add SymInt support for linalg.pinv ( #102367 )"
...
This reverts commit 0d5b74da0cab798fbfdb9caa53fad816999c8386.
Reverted https://github.com/pytorch/pytorch/pull/102367 on behalf of https://github.com/kit1980 due to Broke slow tests https://github.com/pytorch/pytorch/actions/runs/5095190248/jobs/9160028124 ([comment](https://github.com/pytorch/pytorch/pull/102367#issuecomment-1565104562 ))
2023-05-27 00:33:42 +00:00
0d5b74da0c
[pt2] add SymInt support for linalg.pinv ( #102367 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/102367
Approved by: https://github.com/lezcano
2023-05-26 15:20:34 +00:00
9eb1748b2b
[pt2] add meta and SymInt support for linalg_lu ( #101372 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/101372
Approved by: https://github.com/lezcano , https://github.com/albanD
2023-05-15 20:25:00 +00:00
a8964d6377
[pt2] add meta and SymInt support for linalg_householder_product ( #101315 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/101315
Approved by: https://github.com/lezcano
2023-05-15 02:56:49 +00:00
36d91b5513
Add differentiable mkldnn_rnn_layer_backward to support double backward of LSTM ( #100627 )
...
### Description
This PR is to fix #99413 , which shows the limitation of double backward using oneDNN in LSTM.
This PR does not implement double backward function itself, because that is pretty hard to spell out. Instead, it implements mkldnn_rnn_layer_backward using differentiable operations, so that double backward can be done automatically.
During backward process, it needs to use gates and hidden states between cells during one layer. However, these middle variables are stored in the `workspace`, and it is hard to figure them out. Therefore, in backward, we need re-calculate them first.
Corresponding UT has been added based on the failing case in # 99413. The UT with gradcheck and gradgradcheck which is added in https://github.com/pytorch/pytorch/pull/26660 cannot test LSTM using oneDNN, because UT only supports `double` datatype, while oneDNN does not support it.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/100627
Approved by: https://github.com/jgong5 , https://github.com/soulitzer
2023-05-09 12:58:57 +00:00
6eb0d7541d
[pt2] add SymInt support for linalg_qr_backward ( #100833 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/100833
Approved by: https://github.com/ezyang
2023-05-08 13:48:25 +00:00
6af509860e
Add logcumsumexp forward-ad ( #100629 )
...
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at 8bb6158</samp>
This pull request adds forward and backward AD support for the `logcumsumexp` operator in functorch, a library for composable function transformations. It implements a forward-mode formula and a decomposition in `derivatives.yaml`, a C++ function for computing directional derivatives in `FunctionsManual.cpp`, and updates the tests and metadata in `test_ops.py` and `common_methods_invocations.py`.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/100629
Approved by: https://github.com/soulitzer
2023-05-06 04:08:55 +00:00
d66add688f
Revert "Add logcumsumexp forward-ad ( #100629 )"
...
This reverts commit d658c62677b7c096b0fda3ce7a4f0accc727430e.
Reverted https://github.com/pytorch/pytorch/pull/100629 on behalf of https://github.com/clee2000 due to broke slow test, see above comment for details ([comment](https://github.com/pytorch/pytorch/pull/100629#issuecomment-1536575442 ))
2023-05-05 17:42:35 +00:00
