Beginning of process for 3.14 bringup.
State of things from this PR:
- Nothing too scary looking from the Dynamo CPython side, nothing we heavily rely on seems to be missing @williamwen42
- The existing check that makes torch.compile() nicely fail is working as expected. So all these empty functions shouldn't cause any weirdness.
- The `__module__` update changes look suspicious, we should investigate what is the reason and impact of that, in particular for our public API checking @jbschlosser
- Leaving the weakref.py thread safety change as a follow up to keep this a bit simpler. I vendored the whole struct in the meantime FYI @ezyang
EDIT: The `__module__` change is even more cursed than I though due to changes to Union and Optional type where the `__module__` field cannot be changed anymore. See https://github.com/python/cpython/issues/132139 for details.
For now, I'm just skipping the `__module__` setting for 3.14 which will trip the public API checks. Will revisit once I have a final answer on the cpython issue.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/158184
Approved by: https://github.com/msaroufim
Fixes#112633
Fixed errors relating to pydocstyle in the following files. The remaining errors are not covered in this issue. `torch/utils/dlpack.py` was not modified as the errors are relating to the function signature in the first line in the docstring which must be maintained as is for proper Sphinx interpretation.
```python
def from_dlpack(ext_tensor: Any) -> 'torch.Tensor':
"""from_dlpack(ext_tensor) -> Tensor
.....
"""
```
pydocstyle torch/utils/_contextlib.py --count
before: 4
after: 0
pydocstyle torch/backends/mps/__init__.py --count
before: 8
after: 1
**remaining errors**
```
torch/backends/mps/__init__.py:1 at module level:
D104: Missing docstring in public package
```
pydocstyle torch/backends/xeon/run_cpu.py --count
before: 13
after: 1
**remaining errors**
```
torch/backends/xeon/run_cpu.py:864 in public function `main`:
D103: Missing docstring in public function
```
pydocstyle torch/backends/cpu/__init__.py --count
before: 2
after: 1
**remaining errors**
```
torch/backends/cpu/__init__.py:1 at module level:
D104: Missing docstring in public package
```
pydocstyle torch/utils/cpp_backtrace.py --count
before: 4
after: 1
**remaining errors**
```
torch/utils/cpp_backtrace.py:1 at module level:
D100: Missing docstring in public module
```
pydocstyle torch/utils/bundled_inputs.py --count
before: 8
after: 1
**remaining errors**
```
torch/utils/bundled_inputs.py:1 at module level:
D100: Missing docstring in public module
```
pydocstyle torch/utils/file_baton.py --count
before: 8
after: 1
**remaining errors**
```
torch/utils/file_baton.py:1 at module level:
D100: Missing docstring in public module
```
pydocstyle torch/utils/mobile_optimizer.py --count
before: 6
after: 1
**remaining errors**
```
torch/utils/mobile_optimizer.py:8 in public class `LintCode`:
D101: Missing docstring in public class
```
pydocstyle torch/backends/opt_einsum/__init__.py --count
before: 7
after: 5
**remaining errors**
```
torch/backends/opt_einsum/__init__.py:1 at module level:
D104: Missing docstring in public package
torch/backends/opt_einsum/__init__.py:67 in public function `set_flags`:
D103: Missing docstring in public function
torch/backends/opt_einsum/__init__.py:77 in public function `flags`:
D103: Missing docstring in public function
torch/backends/opt_einsum/__init__.py:93 in public class `OptEinsumModule`:
D101: Missing docstring in public class
torch/backends/opt_einsum/__init__.py:94 in public method `__init__`:
D107: Missing docstring in __init__
```
pydocstyle torch/utils/_device.py --count
before: 9
after: 6
**remaining errors**
```
torch/utils/_device.py:58 in public class `DeviceContext`:
D101: Missing docstring in public class
torch/utils/_device.py:59 in public method `__init__`:
D107: Missing docstring in __init__
torch/utils/_device.py:62 in public method `__enter__`:
D105: Missing docstring in magic method
torch/utils/_device.py:68 in public method `__exit__`:
D105: Missing docstring in magic method
torch/utils/_device.py:73 in public method `__torch_function__`:
D105: Missing docstring in magic method
torch/utils/_device.py:80 in public function `device_decorator`:
D103: Missing docstring in public function
```
pydocstyle torch/utils/_freeze.py --count
before: 15
after: 7
**remaining errors**
```
torch/utils/_freeze.py:77 in public function `indent_msg`:
D103: Missing docstring in public function
torch/utils/_freeze.py:89 in public class `FrozenModule`:
D101: Missing docstring in public class
torch/utils/_freeze.py:100 in public class `Freezer`:
D101: Missing docstring in public class
torch/utils/_freeze.py:101 in public method `__init__`:
D107: Missing docstring in __init__
torch/utils/_freeze.py:106 in public method `msg`:
D102: Missing docstring in public method
torch/utils/_freeze.py:185 in public method `get_module_qualname`:
D102: Missing docstring in public method
torch/utils/_freeze.py:206 in public method `compile_string`:
D102: Missing docstring in public method
```
pydocstyle torch/utils/throughput_benchmark.py --count
before: 25
after: 8
**remaining errors**
```
torch/utils/throughput_benchmark.py:1 at module level:
D100: Missing docstring in public module
torch/utils/throughput_benchmark.py:27 in public class `ExecutionStats`:
D101: Missing docstring in public class
torch/utils/throughput_benchmark.py:28 in public method `__init__`:
D107: Missing docstring in __init__
torch/utils/throughput_benchmark.py:33 in public method `latency_avg_ms`:
D102: Missing docstring in public method
torch/utils/throughput_benchmark.py:37 in public method `num_iters`:
D102: Missing docstring in public method
torch/utils/throughput_benchmark.py:46 in public method `total_time_seconds`:
D102: Missing docstring in public method
torch/utils/throughput_benchmark.py:50 in public method `__str__`:
D105: Missing docstring in magic method
torch/utils/throughput_benchmark.py:94 in public method `__init__`:
D107: Missing docstring in __init__
```
pydocstyle torch/utils/hooks.py --count
before: 14
after: 11
**remaining errors**
```
torch/utils/hooks.py:1 at module level:
D100: Missing docstring in public module
torch/utils/hooks.py:23 in public method `__init__`:
D107: Missing docstring in __init__
torch/utils/hooks.py:34 in public method `remove`:
D102: Missing docstring in public method
torch/utils/hooks.py:44 in public method `__getstate__`:
D105: Missing docstring in magic method
torch/utils/hooks.py:50 in public method `__setstate__`:
D105: Missing docstring in magic method
torch/utils/hooks.py:64 in public method `__enter__`:
D105: Missing docstring in magic method
torch/utils/hooks.py:67 in public method `__exit__`:
D105: Missing docstring in magic method
torch/utils/hooks.py:82 in public function `warn_if_has_hooks`:
D103: Missing docstring in public function
torch/utils/hooks.py:103 in public method `__init__`:
D107: Missing docstring in __init__
torch/utils/hooks.py:188 in public method `setup_input_hook`:
D102: Missing docstring in public method
torch/utils/hooks.py:197 in public method `setup_output_hook`:
D102: Missing docstring in public method
```
pydocstyle torch/utils/_traceback.py --count
before: 19
after: 14
**remaining errors**
```
torch/utils/_traceback.py:47 in public function `report_compile_source_on_error`:
D103: Missing docstring in public function
torch/utils/_traceback.py:160 in public class `CapturedTraceback`:
D101: Missing docstring in public class
torch/utils/_traceback.py:163 in public method `__init__`:
D107: Missing docstring in __init__
torch/utils/_traceback.py:167 in public method `cleanup`:
D102: Missing docstring in public method
torch/utils/_traceback.py:170 in public method `summary`:
D102: Missing docstring in public method
torch/utils/_traceback.py:182 in public method `__getstate__`:
D105: Missing docstring in magic method
torch/utils/_traceback.py:190 in public method `extract`:
D205: 1 blank line required between summary line and description (found 0)
torch/utils/_traceback.py:190 in public method `extract`:
D400: First line should end with a period (not 't')
torch/utils/_traceback.py:213 in public method `format`:
D205: 1 blank line required between summary line and description (found 0)
torch/utils/_traceback.py:213 in public method `format`:
D400: First line should end with a period (not 'f')
torch/utils/_traceback.py:213 in public method `format`:
D401: First line should be in imperative mood (perhaps 'Format', not 'Formats')
torch/utils/_traceback.py:224 in public method `format_all`:
D200: One-line docstring should fit on one line with quotes (found 3)
torch/utils/_traceback.py:247 in private function `_extract_symbolized_tb`:
D205: 1 blank line required between summary line and description (found 0)
torch/utils/_traceback.py:247 in private function `_extract_symbolized_tb`:
D400: First line should end with a period (not 'f')
```
pydocstyle torch/utils/mkldnn.py --count
before: 28
after: 26
**remaining errors**
```
torch/utils/mkldnn.py:1 at module level:
D100: Missing docstring in public module
torch/utils/mkldnn.py:4 in public class `MkldnnLinear`:
D101: Missing docstring in public class
torch/utils/mkldnn.py:5 in public method `__init__`:
D107: Missing docstring in __init__
torch/utils/mkldnn.py:19 in public method `__getstate__`:
D105: Missing docstring in magic method
torch/utils/mkldnn.py:23 in public method `__setstate__`:
D105: Missing docstring in magic method
torch/utils/mkldnn.py:29 in public method `forward`:
D102: Missing docstring in public method
torch/utils/mkldnn.py:75 in public class `MkldnnConv1d`:
D101: Missing docstring in public class
torch/utils/mkldnn.py:76 in public method `__init__`:
D107: Missing docstring in __init__
torch/utils/mkldnn.py:82 in public method `__setstate__`:
D105: Missing docstring in magic method
torch/utils/mkldnn.py:88 in public class `MkldnnConv2d`:
D101: Missing docstring in public class
torch/utils/mkldnn.py:89 in public method `__init__`:
D107: Missing docstring in __init__
torch/utils/mkldnn.py:100 in public method `__setstate__`:
D105: Missing docstring in magic method
torch/utils/mkldnn.py:110 in public class `MkldnnConv3d`:
D101: Missing docstring in public class
torch/utils/mkldnn.py:111 in public method `__init__`:
D107: Missing docstring in __init__
torch/utils/mkldnn.py:122 in public method `__setstate__`:
D105: Missing docstring in magic method
torch/utils/mkldnn.py:133 in public class `MkldnnBatchNorm`:
D101: Missing docstring in public class
torch/utils/mkldnn.py:136 in public method `__init__`:
D107: Missing docstring in __init__
torch/utils/mkldnn.py:155 in public method `__getstate__`:
D105: Missing docstring in magic method
torch/utils/mkldnn.py:163 in public method `__setstate__`:
D105: Missing docstring in magic method
torch/utils/mkldnn.py:171 in public method `forward`:
D102: Missing docstring in public method
torch/utils/mkldnn.py:184 in public class `MkldnnPrelu`:
D101: Missing docstring in public class
torch/utils/mkldnn.py:185 in public method `__init__`:
D107: Missing docstring in __init__
torch/utils/mkldnn.py:190 in public method `__getstate__`:
D105: Missing docstring in magic method
torch/utils/mkldnn.py:194 in public method `__setstate__`:
D105: Missing docstring in magic method
torch/utils/mkldnn.py:199 in public method `forward`:
D102: Missing docstring in public method
torch/utils/mkldnn.py:205 in public function `to_mkldnn`:
D103: Missing docstring in public function
```
pydocstyle torch/utils/weak.py --count
before: 32
after: 30
**remaining errors**
```
torch/utils/weak.py:1 at module level:
D100: Missing docstring in public module
torch/utils/weak.py:42 in public class `WeakIdRef`:
D101: Missing docstring in public class
torch/utils/weak.py:45 in public method `__init__`:
D107: Missing docstring in __init__
torch/utils/weak.py:54 in public method `__call__`:
D102: Missing docstring in public method
torch/utils/weak.py:61 in public method `__hash__`:
D105: Missing docstring in magic method
torch/utils/weak.py:64 in public method `__eq__`:
D105: Missing docstring in magic method
torch/utils/weak.py:84 in public class `WeakIdKeyDictionary`:
D101: Missing docstring in public class
torch/utils/weak.py:87 in public method `__init__`:
D107: Missing docstring in __init__
torch/utils/weak.py:131 in public method `__delitem__`:
D105: Missing docstring in magic method
torch/utils/weak.py:135 in public method `__getitem__`:
D105: Missing docstring in magic method
torch/utils/weak.py:138 in public method `__len__`:
D105: Missing docstring in magic method
torch/utils/weak.py:145 in public method `__repr__`:
D105: Missing docstring in magic method
torch/utils/weak.py:148 in public method `__setitem__`:
D105: Missing docstring in magic method
torch/utils/weak.py:151 in public method `copy`:
D102: Missing docstring in public method
torch/utils/weak.py:162 in public method `__deepcopy__`:
D105: Missing docstring in magic method
torch/utils/weak.py:172 in public method `get`:
D102: Missing docstring in public method
torch/utils/weak.py:175 in public method `__contains__`:
D105: Missing docstring in magic method
torch/utils/weak.py:182 in public method `items`:
D102: Missing docstring in public method
torch/utils/weak.py:189 in public method `keys`:
D102: Missing docstring in public method
torch/utils/weak.py:198 in public method `values`:
D102: Missing docstring in public method
torch/utils/weak.py:216 in public method `popitem`:
D102: Missing docstring in public method
torch/utils/weak.py:224 in public method `pop`:
D102: Missing docstring in public method
torch/utils/weak.py:228 in public method `setdefault`:
D102: Missing docstring in public method
torch/utils/weak.py:231 in public method `update`:
D102: Missing docstring in public method
torch/utils/weak.py:241 in public method `__ior__`:
D105: Missing docstring in magic method
torch/utils/weak.py:245 in public method `__or__`:
D105: Missing docstring in magic method
torch/utils/weak.py:252 in public method `__ror__`:
D105: Missing docstring in magic method
torch/utils/weak.py:262 in public method `__eq__`:
D105: Missing docstring in magic method
torch/utils/weak.py:276 in public method `__init__`:
D107: Missing docstring in __init__
torch/utils/weak.py:280 in public method `__call__`:
D102: Missing docstring in public method
```
@mikaylagawarecki @jbschlosser @svekars
Pull Request resolved: https://github.com/pytorch/pytorch/pull/113311
Approved by: https://github.com/ezyang
This PR re-lands
- [Typing] Fix PEP 484 Violation (#105022)
- Update mypy to 1.4.1 (#91983)
That were reverted due to the conflict with internal source repo.
Mostly fixes for PEP-484 violation (i.e. when default arg is set to None, but type is not annotated as optional)
Plus few real fixes:
- Add missing `_get_upgraders_entry_map` to `torch/_C/__init__.pyi`
- Add missing return statement to `torch._export. deserialize_graph`
- Fix error message in `torch.ao.ns.fx.weight_utils.get_lstm_mod_weights`
- Add assert it `torch/optim/optimizer.py` that Optional list is not None
TODO (in followup PR):
- Fix erroneous `isinstance` check in `torch/ao/quantization/_pt2e/qat_utils.py`
Unrelated, to bypass CI failures due to the gcc9 dependency update in Ubuntu-18.04:
- Add hack to squash older libstdc++ from conda environment in favor one from OS to `.ci/docker/install_conda.sh`
- Update bazel cuda builds to focal, as with libstdc++-6.0.32 bazel builds loose the ability to catch exceptions (probably because they link with cupti statically, but I could not found where it is done)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/105227
Approved by: https://github.com/atalman, https://github.com/albanD, https://github.com/Skylion007
This PR re-lands
- [Typing] Fix PEP 484 Violation (#105022)
- Update mypy to 1.4.1 (#91983)
That were reverted due to the conflict with internal source repo.
Mostly fixes for PEP-484 violation (i.e. when default arg is set to None, but type is not annotated as optional)
Plus few real fixes:
- Add missing `_get_upgraders_entry_map` to `torch/_C/__init__.pyi`
- Add missing return statement to `torch._export. deserialize_graph`
- Fix error message in `torch.ao.ns.fx.weight_utils.get_lstm_mod_weights`
- Add assert it `torch/optim/optimizer.py` that Optional list is not None
TODO (in followup PR):
- Fix erroneous `isinstance` check in `torch/ao/quantization/_pt2e/qat_utils.py`
Pull Request resolved: https://github.com/pytorch/pytorch/pull/105227
Approved by: https://github.com/atalman, https://github.com/albanD, https://github.com/Skylion007
CUDA Graph Trees
Design doc: https://docs.google.com/document/d/1ZrxLGWz7T45MSX6gPsL6Ln4t0eZCSfWewtJ_qLd_D0E/edit
Not currently implemented :
- Right now, we are using weak tensor refs from outputs to check if a tensor has dies. This doesn't work because a) aliasing, and b) aot_autograd detaches tensors (see note [Detaching saved tensors in AOTAutograd]). Would need either https://github.com/pytorch/pytorch/issues/91395 to land to use storage weak refs or manually add a deleter fn that does what I want. This is doable but theres some interactions with the caching allocator checkpointing so saving for a stacked pr.
- Reclaiming memory from the inputs during model recording. This isn't terribly difficult but deferring to another PR. You would need to write over the input memory during warmup, and therefore copy the inputs to cpu. Saving for a stacked pr.
- Warning on overwriting previous generation outputs. and handling nested torch.compile() calls in generation tracking
Differential Revision: [D43999887](https://our.internmc.facebook.com/intern/diff/D43999887)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/89146
Approved by: https://github.com/ezyang
In the prior patch, I just YOLOed a mutable mapping implementation.
Many edge cases were not handled correctly. In this PR, I just
copy paste the WeakKeyDictionary from CPython and the hacked it up
to use WeakIdRef instead of weakref.ref. You can see each line
I changed with the comment CHANGED; there aren't many.
Being exactly API compatible with WeakKeyDictionary means I can also
rob all of the tests from CPython, which I also did for
test/test_weak.py
How to review? You could either try taking the delta from CPython
(recommended), or review everything from scratch (not recommended).
Can post diff representing delta on request.
Signed-off-by: Edward Z. Yang <ezyang@fb.com>
Pull Request resolved: https://github.com/pytorch/pytorch/pull/90825
Approved by: https://github.com/albanD
