c91f59b1a0
Fix performance regression when indexing by Numpy arrays ( #163280 )
...
Benchmark script:
```
import time
import numpy as np
import torch
def main() -> None:
for i in range(10):
block_indices = np.arange(16384, dtype=np.int32)
block_indices = block_indices.reshape(-1).clip(max=255)
batch_indices = np.zeros(16384, dtype=np.int64)
virtual_batches = 32
block_table = torch.randn(32, 256)
start = time.perf_counter()
block_table[batch_indices, block_indices].view(virtual_batches, -1)
end = time.perf_counter()
time_elapsed_ms = (end - start) * 1000
print(f"Function execution time: {time_elapsed_ms:.1f}ms")
if __name__ == "__main__":
main()
```
Before:
```
(a) [ezyang@devvm006.dkl0 ~/local/b/pytorch] python ben.py
Function execution time: 28.5ms
Function execution time: 12.9ms
Function execution time: 12.6ms
Function execution time: 13.5ms
Function execution time: 12.0ms
Function execution time: 13.4ms
Function execution time: 12.9ms
Function execution time: 12.9ms
Function execution time: 13.1ms
Function execution time: 13.0ms
```
After:
```
Function execution time: 17.8ms
Function execution time: 2.5ms
Function execution time: 1.3ms
Function execution time: 2.5ms
Function execution time: 2.3ms
Function execution time: 1.3ms
Function execution time: 2.4ms
Function execution time: 2.5ms
Function execution time: 2.5ms
Function execution time: 2.4ms
```
Signed-off-by: Edward Z. Yang <ezyang@meta.com >
Pull Request resolved: https://github.com/pytorch/pytorch/pull/163280
Approved by: https://github.com/SherlockNoMad , https://github.com/cyyever
2025-09-19 05:02:58 +00:00
9a1c5c0a07
Detect torch function in lists as well ( #160256 )
...
We basically follow the same pattern we do for tensor arguments. The major downside is we now have to traverse the entirety of the int list / etc where previously we didn't have. Benchmark suggests 2% regression for relevant things.
Signed-off-by: Edward Yang <ezyang@meta.com >
Pull Request resolved: https://github.com/pytorch/pytorch/pull/160256
Approved by: https://github.com/albanD
2025-09-02 16:22:42 +00:00
a6401cb5aa
Revert "flip the list-as-tuple behavior for short lists ( #160794 )"
...
This reverts commit febfc3ec03004116dfd6d504e6853ff02a1dd6e0.
Reverted https://github.com/pytorch/pytorch/pull/160794 on behalf of https://github.com/seemethere due to This if failing internal tests, see D80671241 ([comment](https://github.com/pytorch/pytorch/pull/160794#issuecomment-3211314867 ))
2025-08-21 16:33:30 +00:00
febfc3ec03
flip the list-as-tuple behavior for short lists ( #160794 )
...
Per title, previously we started throwing noisy warnings, but given how popular this pattern was in our test suite decided to leave it as warning, not as silent behavior change for one release.
Now `treatSequenceAsTuple` would return `true` in the only case where the sequence was indeed a tuple, so no need for a special function anymore.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/160794
Approved by: https://github.com/albanD
2025-08-20 22:40:42 +00:00
b0556110e5
Remove unsafe PyTorchError constructor ( #154961 )
...
Use libfmt in call sites of PyTorchError.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/154961
Approved by: https://github.com/albanD
2025-07-11 18:22:53 +00:00
d9b8369f39
fix warning spam for list indexing ( #155815 )
...
Per title, #154806 incorrectly placed a warning
Pull Request resolved: https://github.com/pytorch/pytorch/pull/155815
Approved by: https://github.com/Skylion007 , https://github.com/albanD
2025-06-12 23:07:24 +00:00
34e3930401
fix numpy compatibility for 2d small list indices ( #154806 )
...
Will fix #119548 and linked issues once we switch from warning to the new behavior,
but for now, given how much this syntax was used in our test suite, we suspect a silent change will be disruptive.
We will change the behavior after 2.8 branch is cut.
Numpy behavior was changed at least in numpy 1.24 (more than 2 years ago)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/154806
Approved by: https://github.com/cyyever , https://github.com/Skylion007 , https://github.com/albanD
2025-06-04 01:58:52 +00:00
bf1b8adee6
Turn static inline into static function ( #139843 )
...
Fixes #ISSUE_NUMBER
Pull Request resolved: https://github.com/pytorch/pytorch/pull/139843
Approved by: https://github.com/ezyang
2024-11-07 23:58:18 +00:00
68f1b52d8a
Revert "Turn static inline into static function ( #139843 )"
...
This reverts commit 72d3f5b26d90396f7a357fa3e5d82656ca74c102.
Reverted https://github.com/pytorch/pytorch/pull/139843 on behalf of https://github.com/ZainRizvi due to Sorry but this is causing tests to fail on trunk. See [GH job link](https://github.com/pytorch/pytorch/actions/runs/11729669425/job/32675829894 ) [HUD commit link](72d3f5b26dhttps://github.com/pytorch/pytorch/pull/139843#issuecomment-2463354131 ))
2024-11-07 22:29:45 +00:00
72d3f5b26d
Turn static inline into static function ( #139843 )
...
Fixes #ISSUE_NUMBER
Pull Request resolved: https://github.com/pytorch/pytorch/pull/139843
Approved by: https://github.com/ezyang
2024-11-07 19:08:41 +00:00
fb36daac9f
[7/N] Fix extra warnings brought by clang-tidy-17 ( #138972 )
...
Fix extra warnings brought by clang-tidy-17
Pull Request resolved: https://github.com/pytorch/pytorch/pull/138972
Approved by: https://github.com/Skylion007
2024-10-26 19:09:47 +00:00
929d2f8253
[3/N] Fix clang-tidy warnings in torch/csrc/autograd ( #133389 )
...
Follows #133295
Pull Request resolved: https://github.com/pytorch/pytorch/pull/133389
Approved by: https://github.com/Skylion007
2024-08-16 00:57:54 +00:00
f4dcf2ae93
[1/N] Change #include <c10/util/Optional.h> to #include <optional> ( #128301 )
...
Fixes #ISSUE_NUMBER
Pull Request resolved: https://github.com/pytorch/pytorch/pull/128301
Approved by: https://github.com/ezyang , https://github.com/r-barnes
2024-07-08 07:03:53 +00:00
846bb30e13
Revert "[1/N] Change #include <c10/util/Optional.h> to #include <optional> ( #128301 )"
...
This reverts commit bd72e28314d8d63bb347becb8309f5ac7761c6b5.
Reverted https://github.com/pytorch/pytorch/pull/128301 on behalf of https://github.com/huydhn due to Sorry for reverting your change but it fails XLA build bd72e28314https://github.com/pytorch/pytorch/pull/128301#issuecomment-2169035822 ))
2024-06-15 01:58:20 +00:00
bd72e28314
[1/N] Change #include <c10/util/Optional.h> to #include <optional> ( #128301 )
...
Fixes #ISSUE_NUMBER
Pull Request resolved: https://github.com/pytorch/pytorch/pull/128301
Approved by: https://github.com/ezyang
2024-06-14 23:21:01 +00:00
ed327876f5
[codemod] c10:optional -> std::optional ( #126135 )
...
Generated by running the following from PyTorch root:
```
find . -regex ".*\.\(cpp\|h\|cu\|hpp\|cc\|cxx\)$" | grep -v "build/" | xargs -n 50 -P 4 perl -pi -e 's/c10::optional/std::optional/'
```
`c10::optional` is just an alias for `std::optional`. This removes usages of that alias in preparation for eliminating it entirely.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/126135
Approved by: https://github.com/Skylion007 , https://github.com/malfet , https://github.com/albanD , https://github.com/aaronenyeshi
2024-05-14 19:35:51 +00:00
392dc45597
Made FlexAttention rewrite getitem calls to use aten.index in score_mod ( #124799 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/124799
Approved by: https://github.com/drisspg
ghstack dependencies: #124444
2024-04-26 17:22:13 +00:00
e913f77c60
Revert "Made FlexAttention rewrite getitem calls to use aten.index in score_mod ( #124799 )"
...
This reverts commit 9bccafc31c9d489b727155e95633efd19adbceaa.
Reverted https://github.com/pytorch/pytorch/pull/124799 on behalf of https://github.com/clee2000 due to broke tests but only on crossref https://github.com/pytorch/pytorch/actions/runs/8841521519/job/24279075171 , added no td label so itll actually run this time ([comment](https://github.com/pytorch/pytorch/pull/124799#issuecomment-2078530797 ))
2024-04-26 02:35:14 +00:00
9bccafc31c
Made FlexAttention rewrite getitem calls to use aten.index in score_mod ( #124799 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/124799
Approved by: https://github.com/drisspg
ghstack dependencies: #124444
2024-04-26 01:02:28 +00:00
678662a557
Revert "Made FlexAttention rewrite getitem calls to use aten.index in score_mod ( #124799 )"
...
This reverts commit acc4cbea395c25410c26d6fd3c88c072ce24c918.
Reverted https://github.com/pytorch/pytorch/pull/124799 on behalf of https://github.com/jeanschmidt due to checking if this diff introduced regressions on linux-focal-py3.11-clang10 and linux-focal-py3.8-clang10 ([comment](https://github.com/pytorch/pytorch/pull/124799#issuecomment-2076756876 ))
2024-04-25 09:29:57 +00:00
acc4cbea39
Made FlexAttention rewrite getitem calls to use aten.index in score_mod ( #124799 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/124799
Approved by: https://github.com/drisspg
2024-04-25 06:19:55 +00:00
8a3c241094
Remove unused header inclusion ( #119667 )
...
Fixes #ISSUE_NUMBER
Pull Request resolved: https://github.com/pytorch/pytorch/pull/119667
Approved by: https://github.com/Skylion007
2024-02-12 05:36:25 +00:00
dabb90f2a4
Revert "[Exception] [6/N] Remove use of torch::TypeError ( #117964 )"
...
This reverts commit 87335fabaeca41f9721ba5d5eb7eafcf70b7afad.
Reverted https://github.com/pytorch/pytorch/pull/117964 on behalf of https://github.com/facebook-github-bot due to Diff reverted internally ([comment](https://github.com/pytorch/pytorch/pull/117964#issuecomment-1913079096 ))
2024-01-27 08:44:34 +00:00
87335fabae
[Exception] [6/N] Remove use of torch::TypeError ( #117964 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/117964
Approved by: https://github.com/albanD
2024-01-25 03:35:58 +00:00
396a5c3091
[Exception] [4/N] Replace torch::IndexError and torch::ValueError with C10 counterparts ( #117317 )
...
Fixes #ISSUE_NUMBER
Pull Request resolved: https://github.com/pytorch/pytorch/pull/117317
Approved by: https://github.com/ezyang
2024-01-18 00:35:29 +00:00
0cbca85707
Add check to prevent NumPy ndarray from being treated as tuple when indexing ( #108954 )
...
Fixes #108689
Pull Request resolved: https://github.com/pytorch/pytorch/pull/108954
Approved by: https://github.com/lezcano
2023-09-15 08:51:58 +00:00
2b9ad3d5c4
Fix setitem with SymInt ( #108873 )
...
Fixes https://github.com/pytorch/pytorch/issues/101939
Several fixes bundled together:
1. When we valueToTensor, we only handled non-symbolic inputs and not symbolic inputs. We support symbolic Scalar, so also handle symbolic values.
2. In the symbolic case, we MUST NOT lift_fresh, as you're not going to inline a constant into the graph, it's going to be from a `scalar_tensor` call (so no need to clone it to avoid mutations)
3. In indexing scalarToTensor, must not do the static, directly read out the scalar contents logic with the scalar is symbolic
Signed-off-by: Edward Z. Yang <ezyang@meta.com >
Pull Request resolved: https://github.com/pytorch/pytorch/pull/108873
Approved by: https://github.com/jansel
2023-09-10 06:44:22 +00:00
596491f1f5
Propagate dynamic int on __setitem__. ( #105923 )
...
Fix : #105533
This PR propagates dynamic ints used as indices for `__setitem__`. In summary, we:
- Replace the integer type for `TensorIndex` (both the enum and the corresponding
functions)
- Accordingly modify _python_variable_indexing.cpp_
Pull Request resolved: https://github.com/pytorch/pytorch/pull/105923
Approved by: https://github.com/ezyang
2023-08-01 17:34:03 +00:00
f89ae0a7f4
Revert "Only truncate leading 1s if the value is too big. ( #94521 )"
...
This reverts commit 03f4a63fd86fe2d22202c7aee6a4e62c13b4f561.
Reverted https://github.com/pytorch/pytorch/pull/94521 on behalf of https://github.com/ezyang due to fails internal tests
2023-02-19 15:05:56 +00:00
03f4a63fd8
Only truncate leading 1s if the value is too big. ( #94521 )
...
If it's just right, broadcasting will do the right thing
automatically.
This helps with unbacked SymInts as I can avoid testing one
equality on the inside.
Signed-off-by: Edward Z. Yang <ezyang@meta.com >
Pull Request resolved: https://github.com/pytorch/pytorch/pull/94521
Approved by: https://github.com/voznesenskym
2023-02-16 21:33:12 +00:00
f59845db40
Symintify pytorch slicing logic ( #91340 )
...
Differential Revision: [D42398023](https://our.internmc.facebook.com/intern/diff/D42398023 )
Pull Request resolved: https://github.com/pytorch/pytorch/pull/91340
Approved by: https://github.com/Skylion007 , https://github.com/albanD
2023-01-08 22:51:42 +00:00
b3603f8129
Revert "Deduplicate c10 error and PyTorchError hierarchy ( #87855 )"
...
This reverts commit 34f2d3e6ae56744c20c2f859f97101dff291bbbc.
Reverted https://github.com/pytorch/pytorch/pull/87855 on behalf of https://github.com/osalpekar due to perf regression in quantization tests
2023-01-06 19:56:35 +00:00
3bb63aa387
Revert "Symintify pytorch slicing logic ( #91340 )"
...
This reverts commit 8c172fa98a52e95675e9425ac4b23f190f53f9ed.
Reverted https://github.com/pytorch/pytorch/pull/91340 on behalf of https://github.com/clee2000 due to breaking mac builds 8c172fa98ahttps://github.com/pytorch/pytorch/actions/runs/3845932024/jobs/6550654339 , marking this as weird because it was merged via codev?
2023-01-05 17:14:49 +00:00
8c172fa98a
Symintify pytorch slicing logic ( #91340 )
...
Differential Revision: [D42223260](https://our.internmc.facebook.com/intern/diff/D42223260 )
Pull Request resolved: https://github.com/pytorch/pytorch/pull/91340
Approved by: https://github.com/Skylion007 , https://github.com/albanD
2023-01-05 10:33:37 +00:00
34f2d3e6ae
Deduplicate c10 error and PyTorchError hierarchy ( #87855 )
...
Fixes #53370
Pull Request resolved: https://github.com/pytorch/pytorch/pull/87855
Approved by: https://github.com/albanD
2023-01-02 15:53:36 +00:00
28ceccec21
cleanup old python_compat code ( #91162 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/91162
Approved by: https://github.com/ezyang
2022-12-20 18:13:19 +00:00
d5a7e6db38
ATen/native (1/6): Use per-operator headers ( #75571 )
...
Differential Revision: [D40126698](https://our.internmc.facebook.com/intern/diff/D40126698 )
Pull Request resolved: https://github.com/pytorch/pytorch/pull/75571
Approved by: https://github.com/malfet
2022-10-15 01:43:26 +00:00
be8627827e
More symintification of get/set item ( #86605 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/86605
Approved by: https://github.com/anjali411
2022-10-11 12:00:40 +00:00
55663b7f81
Reland 3 of Symintify getitem and add the required helper functions ( #86207 ) ( #86487 )
...
Note that this might not cover every use of the function (we know it doesn't)
But this is enough to get few models passing.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/86487
Approved by: https://github.com/ezyang
2022-10-10 15:54:28 +00:00
5b69b87d5a
Revert "Symintify getitem and add the required helper functions ( #86207 )"
...
This reverts commit fd5085c445c3f1a4c90e55154cf26fe30f52a0ab.
Reverted https://github.com/pytorch/pytorch/pull/86207 on behalf of https://github.com/seemethere due to Fails internal tests, see: https://www.internalfb.com/intern/sandcastle/job/22517998926071860/insights
2022-10-07 16:10:30 +00:00
fd5085c445
Symintify getitem and add the required helper functions ( #86207 )
...
Note that this might not cover every use of the function (we know it doesn't)
But this is enough to get few models passing.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/86207
Approved by: https://github.com/ezyang , https://github.com/Chillee , https://github.com/bdhirsh
2022-10-06 04:46:19 +00:00
168ba066e3
Revert "Symintify getitem and add the required helper functions ( #86207 )"
...
This reverts commit 17addb307ee9a4d12ad6918e90358a9a47a4f12b.
Reverted https://github.com/pytorch/pytorch/pull/86207 on behalf of https://github.com/malfet due to Broke lint, by double-registering `meta_index_put`, but no CI was run during the outage
2022-10-05 22:42:56 +00:00
17addb307e
Symintify getitem and add the required helper functions ( #86207 )
...
Note that this might not cover every use of the function (we know it doesn't)
But this is enough to get few models passing.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/86207
Approved by: https://github.com/ezyang
2022-10-05 21:19:00 +00:00
0e0dfaa057
Add support for select of batch dims for all sparse compressed formats. ( #82119 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/82119
Approved by: https://github.com/nikitaved , https://github.com/bhosmer
2022-08-06 02:24:20 +00:00
452af0bc44
Call lift fresh in valueToTensor ( #81927 )
...
`valueToTensor` is invoked in many python apis such as `x[0] = 0.5`, which lifts the 0.5 to a tensor. the problem is described similarly in https://github.com/pytorch/pytorch/pull/81609 (/s/scalar_to_tensor/valueToTensor)
> scalar_to_tensor is not dispatched and thus there is no interposition point for modes to ensure that the resulting tensor is appropriately wrapped. lift_fresh introduces this interposition point. This prevents FakeTensorMode from erroring
Pull Request resolved: https://github.com/pytorch/pytorch/pull/81927
Approved by: https://github.com/ezyang
2022-07-26 22:07:59 +00:00
30fb2c4aba
[lint] autoformat test/cpp and torch/csrc
...
Let's have some fun.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/78828
Approved by: https://github.com/ezyang
2022-06-11 21:11:16 +00:00
6ad51c9422
Support indexing of the underlying tensors for nested tensors ( #78934 )
...
Fixes #76843
Pull Request resolved: https://github.com/pytorch/pytorch/pull/78934
Approved by: https://github.com/cpuhrsch , https://github.com/jbschlosser
2022-06-08 21:05:04 +00:00
ab6c7b4b3f
fix __torch_function__ bug in getindex that causes an error not set exception
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/78781
Approved by: https://github.com/ezyang
2022-06-06 17:02:57 +00:00
407e8eba8c
Enable simple indexing into CSR tensor, add torch.select for CSR
...
This PR implements `torch.select` for CSR tensors. Currently, it's not possible to select rows or columns for batched CSR. The non-batched case works fine by converting to COO and calling select. Initially, I implemented raw manipulations of indices but converting to COO is only slightly slower and more readable.
This PR also enables indexing into batched CSR tensor with `[x, y, z]`. Assigning is disabled.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/76228
Approved by: https://github.com/cpuhrsch
2022-04-23 02:36:03 +00:00
37e0d2e361
Fix segfault while real and imaginary attributes are set to a number ( #73867 )
...
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/73867
Fixes https://github.com/pytorch/pytorch/issues/72947
Test Plan: Imported from OSS
Reviewed By: davidberard98
Differential Revision: D34695956
Pulled By: anjali411
fbshipit-source-id: 2f3eda272a5214335eae506bd387ce8da4d81b8c
(cherry picked from commit fdb07354cac22c30aa047e65fbac9840608db811)
2022-03-08 18:58:26 +00:00
