This enables a check that which a class which only inherits from immutable classes like str, tuple, and NamedTuple, also defined `__slots__` so they don't allocate memory unnecessarily. This also ensure contributors think about how they define their classes with subclass NamedTuples and str, of which we have many in our codebase
Pull Request resolved: https://github.com/pytorch/pytorch/pull/146276
Approved by: https://github.com/aorenste
This is a lot of files changed! Don't panic! Here's how it works:
* Previously, we set `follow_imports = silent` for our mypy.ini configuration. Per https://mypy.readthedocs.io/en/stable/running_mypy.html#follow-imports, what this does is whenever we have an import to a module which is not listed as a file to be typechecked in mypy, we typecheck it as normal but suppress all errors that occurred in that file.
* When mypy is run inside lintrunner, the list of files is precisely the files covered by the glob in lintrunner.toml, but with files in excludes excluded.
* The top-level directive `# mypy: ignore-errors` instructs mypy to typecheck the file as normal, but ignore all errors.
* Therefore, it should be equivalent to set `follow_imports = normal`, if we put `# mypy: ignore-errors` on all files that were previously excluded from the file list.
* Having done this, we can remove the exclude list from .lintrunner.toml, since excluding a file from typechecking is baked into the files themselves.
* torch/_dynamo and torch/_inductor were previously in the exclude list, because they were covered by MYPYINDUCTOR. It is not OK to mark these as `# mypy: ignore-errors` as this will impede typechecking on the alternate configuration. So they are temporarily being checked twice, but I am suppressing the errors in these files as the configurations are not quite the same. I plan to unify the configurations so this is only a temporary state.
* There were some straggler type errors after these changes somehow, so I fixed them as needed. There weren't that many.
In the future, to start type checking a file, just remove the ignore-errors directive from the top of the file.
The codemod was done with this script authored by GPT-4:
```
import glob
exclude_patterns = [
...
]
for pattern in exclude_patterns:
for filepath in glob.glob(pattern, recursive=True):
if filepath.endswith('.py'):
with open(filepath, 'r+') as f:
content = f.read()
f.seek(0, 0)
f.write('# mypy: ignore-errors\n\n' + content)
```
Signed-off-by: Edward Z. Yang <ezyang@meta.com>
Pull Request resolved: https://github.com/pytorch/pytorch/pull/118414
Approved by: https://github.com/thiagocrepaldi, https://github.com/albanD
Fixes#113940. This vendors the relevant parts of [`packaging==23.2.0`]() to have access to `Version` and `InvalidVersion` without taking a runtime dependency on `setuptools` or `packaging`.
I didn't find any vendoring policy so I put it under `torch._vendor.packaging`. While I have only vendored the files we need, I have not touched or trimmed the files otherwise.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/114108
Approved by: https://github.com/malfet, https://github.com/albanD
Summary:
Resolves https://github.com/pytorch/pytorch/issues/71280
We used to use `from pkg_resources import packaging`. To recap, this has
three potential problems:
1) `pkg_resources` is a really slow import
2) We have an undeclared runtime dependency on `setuptools`
3) We're relying on `pkg_resources`'s secret vendored copy of
`packaging`. This is obviously not part of the public API of
`pkg_resources`.
In https://github.com/pytorch/pytorch/issues/71345 this was made a lazy import, which is great! It means we don't
run into these problems as long as users don't use `torch.__version__`.
This change additionally helps further address problems 1 and 3, by
directly importing `packaging`, if present, and only falling back to the
vendored copy in `pkg_resources`.
Benchmark for speed difference in a virtual environment with a couple
hundred packages installed:
```
λ hyperfine -w 2 'python -c "from pkg_resources import packaging"' 'python -c "import packaging.version"'
Benchmark 1: python -c "from pkg_resources import packaging"
Time (mean ± σ): 706.7 ms ± 77.1 ms [User: 266.5 ms, System: 156.8 ms]
Range (min … max): 627.9 ms … 853.2 ms 10 runs
Benchmark 2: python -c "import packaging.version"
Time (mean ± σ): 53.8 ms ± 8.5 ms [User: 34.8 ms, System: 14.4 ms]
Range (min … max): 46.3 ms … 72.3 ms 53 runs
'python -c "import packaging.version"' ran
13.14 ± 2.52 times faster than 'python -c "from pkg_resources import packaging"'
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/71902
Reviewed By: mikaylagawarecki
Differential Revision: D34343145
Pulled By: malfet
fbshipit-source-id: a6bd7ecf0cbb6b5c20ab18a22576aa2df9eb3324
(cherry picked from commit 0a249044c8f83ecbc81b6d1e7a16541b82b74243)
Summary:
As it is a pretty big package and to be used during normal
course of PyTorch initialization
Fixes https://github.com/pytorch/pytorch/issues/71280
Pull Request resolved: https://github.com/pytorch/pytorch/pull/71345
Reviewed By: seemethere
Differential Revision: D33594547
Pulled By: malfet
fbshipit-source-id: e0abea82dbdc29914512b610692701140d3e68a2
(cherry picked from commit 1ff7f65cc1ad499a71457368894ca14bed069749)
Summary:
Currently the [TorchVersion](1022443168/torch/torch_version.py (L13)) only only supports 'greater than', and 'equal to' operations for comparing torch versions and something like `TorchVersion('1.5.0') < (1,5,1)` or `TorchVersion('1.5.0') >= (1,5)` will throw an error.
I have added 'less than' (`__lt__()`), 'greater than or equal to' (`__ge__()`) and 'less than or equal to' (`__le__()`) operations, so that the TorchVersion object can be useful for wider range of version comparisons.
cc seemethere zsol
Pull Request resolved: https://github.com/pytorch/pytorch/pull/63848
Reviewed By: fmassa, heitorschueroff
Differential Revision: D30526996
Pulled By: seemethere
fbshipit-source-id: 1db6bee555043e0719fd541cec27810852590940
Summary:
This PR adds greater equal comparison so that not the base class's (str) comparison method is used.
This is necessary for a correct comparison with a version string.
Previously the following was the case:
```py
>>> torch.__version__
'1.10.0.dev20210830+cpu'
>>> torch.__version__>"1.9"
True
>>> torch.__version__>="1.9"
False # Wrong output since the base class (str) was used for __ge__ comparison
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/64565
Reviewed By: raghuramank100
Differential Revision: D30790463
Pulled By: mrshenli
fbshipit-source-id: 79c680f8b448001b34d3e5d5332124a78bea4e34
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/61556
Prior to 1.10.0 `torch.__version__` was stored as a str and so many did
comparisons against `torch.__version__` as if it were a str. In order to not
break them we have TorchVersion which masquerades as a str while also
having the ability to compare against both packaging.version.Version as
well as tuples of values, eg. (1, 2, 1)
Examples:
Comparing a TorchVersion object to a Version object
```
TorchVersion('1.10.0a') > Version('1.10.0a')
```
Comparing a TorchVersion object to a Tuple object
```
TorchVersion('1.10.0a') > (1, 2) # 1.2
TorchVersion('1.10.0a') > (1, 2, 1) # 1.2.1
```
Comparing a TorchVersion object against a string
```
TorchVersion('1.10.0a') > '1.2'
TorchVersion('1.10.0a') > '1.2.1'
```
Resolves https://github.com/pytorch/pytorch/issues/61540
Signed-off-by: Eli Uriegas <eliuriegas@fb.com>
Test Plan: Imported from OSS
Reviewed By: zou3519
Differential Revision: D29671234
Pulled By: seemethere
fbshipit-source-id: 6044805918723b4aca60bbec4b5aafc1189eaad7
