This is a lot of files changed! Don't panic! Here's how it works:
* Previously, we set `follow_imports = silent` for our mypy.ini configuration. Per https://mypy.readthedocs.io/en/stable/running_mypy.html#follow-imports, what this does is whenever we have an import to a module which is not listed as a file to be typechecked in mypy, we typecheck it as normal but suppress all errors that occurred in that file.
* When mypy is run inside lintrunner, the list of files is precisely the files covered by the glob in lintrunner.toml, but with files in excludes excluded.
* The top-level directive `# mypy: ignore-errors` instructs mypy to typecheck the file as normal, but ignore all errors.
* Therefore, it should be equivalent to set `follow_imports = normal`, if we put `# mypy: ignore-errors` on all files that were previously excluded from the file list.
* Having done this, we can remove the exclude list from .lintrunner.toml, since excluding a file from typechecking is baked into the files themselves.
* torch/_dynamo and torch/_inductor were previously in the exclude list, because they were covered by MYPYINDUCTOR. It is not OK to mark these as `# mypy: ignore-errors` as this will impede typechecking on the alternate configuration. So they are temporarily being checked twice, but I am suppressing the errors in these files as the configurations are not quite the same. I plan to unify the configurations so this is only a temporary state.
* There were some straggler type errors after these changes somehow, so I fixed them as needed. There weren't that many.
In the future, to start type checking a file, just remove the ignore-errors directive from the top of the file.
The codemod was done with this script authored by GPT-4:
```
import glob
exclude_patterns = [
...
]
for pattern in exclude_patterns:
for filepath in glob.glob(pattern, recursive=True):
if filepath.endswith('.py'):
with open(filepath, 'r+') as f:
content = f.read()
f.seek(0, 0)
f.write('# mypy: ignore-errors\n\n' + content)
```
Signed-off-by: Edward Z. Yang <ezyang@meta.com>
Pull Request resolved: https://github.com/pytorch/pytorch/pull/118414
Approved by: https://github.com/thiagocrepaldi, https://github.com/albanD
Fixes#113940. This vendors the relevant parts of [`packaging==23.2.0`]() to have access to `Version` and `InvalidVersion` without taking a runtime dependency on `setuptools` or `packaging`.
I didn't find any vendoring policy so I put it under `torch._vendor.packaging`. While I have only vendored the files we need, I have not touched or trimmed the files otherwise.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/114108
Approved by: https://github.com/malfet, https://github.com/albanD
Summary:
Resolves https://github.com/pytorch/pytorch/issues/71280
We used to use `from pkg_resources import packaging`. To recap, this has
three potential problems:
1) `pkg_resources` is a really slow import
2) We have an undeclared runtime dependency on `setuptools`
3) We're relying on `pkg_resources`'s secret vendored copy of
`packaging`. This is obviously not part of the public API of
`pkg_resources`.
In https://github.com/pytorch/pytorch/issues/71345 this was made a lazy import, which is great! It means we don't
run into these problems as long as users don't use `torch.__version__`.
This change additionally helps further address problems 1 and 3, by
directly importing `packaging`, if present, and only falling back to the
vendored copy in `pkg_resources`.
Benchmark for speed difference in a virtual environment with a couple
hundred packages installed:
```
λ hyperfine -w 2 'python -c "from pkg_resources import packaging"' 'python -c "import packaging.version"'
Benchmark 1: python -c "from pkg_resources import packaging"
Time (mean ± σ): 706.7 ms ± 77.1 ms [User: 266.5 ms, System: 156.8 ms]
Range (min … max): 627.9 ms … 853.2 ms 10 runs
Benchmark 2: python -c "import packaging.version"
Time (mean ± σ): 53.8 ms ± 8.5 ms [User: 34.8 ms, System: 14.4 ms]
Range (min … max): 46.3 ms … 72.3 ms 53 runs
'python -c "import packaging.version"' ran
13.14 ± 2.52 times faster than 'python -c "from pkg_resources import packaging"'
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/71902
Reviewed By: mikaylagawarecki
Differential Revision: D34343145
Pulled By: malfet
fbshipit-source-id: a6bd7ecf0cbb6b5c20ab18a22576aa2df9eb3324
(cherry picked from commit 0a249044c8f83ecbc81b6d1e7a16541b82b74243)
Summary:
As it is a pretty big package and to be used during normal
course of PyTorch initialization
Fixes https://github.com/pytorch/pytorch/issues/71280
Pull Request resolved: https://github.com/pytorch/pytorch/pull/71345
Reviewed By: seemethere
Differential Revision: D33594547
Pulled By: malfet
fbshipit-source-id: e0abea82dbdc29914512b610692701140d3e68a2
(cherry picked from commit 1ff7f65cc1ad499a71457368894ca14bed069749)
Summary:
Currently the [TorchVersion](1022443168/torch/torch_version.py (L13)) only only supports 'greater than', and 'equal to' operations for comparing torch versions and something like `TorchVersion('1.5.0') < (1,5,1)` or `TorchVersion('1.5.0') >= (1,5)` will throw an error.
I have added 'less than' (`__lt__()`), 'greater than or equal to' (`__ge__()`) and 'less than or equal to' (`__le__()`) operations, so that the TorchVersion object can be useful for wider range of version comparisons.
cc seemethere zsol
Pull Request resolved: https://github.com/pytorch/pytorch/pull/63848
Reviewed By: fmassa, heitorschueroff
Differential Revision: D30526996
Pulled By: seemethere
fbshipit-source-id: 1db6bee555043e0719fd541cec27810852590940
Summary:
This PR adds greater equal comparison so that not the base class's (str) comparison method is used.
This is necessary for a correct comparison with a version string.
Previously the following was the case:
```py
>>> torch.__version__
'1.10.0.dev20210830+cpu'
>>> torch.__version__>"1.9"
True
>>> torch.__version__>="1.9"
False # Wrong output since the base class (str) was used for __ge__ comparison
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/64565
Reviewed By: raghuramank100
Differential Revision: D30790463
Pulled By: mrshenli
fbshipit-source-id: 79c680f8b448001b34d3e5d5332124a78bea4e34
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/61556
Prior to 1.10.0 `torch.__version__` was stored as a str and so many did
comparisons against `torch.__version__` as if it were a str. In order to not
break them we have TorchVersion which masquerades as a str while also
having the ability to compare against both packaging.version.Version as
well as tuples of values, eg. (1, 2, 1)
Examples:
Comparing a TorchVersion object to a Version object
```
TorchVersion('1.10.0a') > Version('1.10.0a')
```
Comparing a TorchVersion object to a Tuple object
```
TorchVersion('1.10.0a') > (1, 2) # 1.2
TorchVersion('1.10.0a') > (1, 2, 1) # 1.2.1
```
Comparing a TorchVersion object against a string
```
TorchVersion('1.10.0a') > '1.2'
TorchVersion('1.10.0a') > '1.2.1'
```
Resolves https://github.com/pytorch/pytorch/issues/61540
Signed-off-by: Eli Uriegas <eliuriegas@fb.com>
Test Plan: Imported from OSS
Reviewed By: zou3519
Differential Revision: D29671234
Pulled By: seemethere
fbshipit-source-id: 6044805918723b4aca60bbec4b5aafc1189eaad7
