33daaad7d0
dynamo: Handle objects in graph that do not support weakref ( #163168 )
...
We are seeing crashes of the form
```
Traceback (most recent call last):
File "/packages/aps_ads_vm/launcher_multiapp-inplace#link-tree/torch/_dynamo/symbolic_convert.py", line 1487, in run
while self.step():
File "/packages/aps_ads_vm/launcher_multiapp-inplace#link-tree/torch/_dynamo/symbolic_convert.py", line 1348, in step
self.dispatch_table[inst.opcode](self, inst)
File "/packages/aps_ads_vm/launcher_multiapp-inplace#link-tree/torch/_dynamo/symbolic_convert.py", line 2437, in LOAD_ATTR
self._load_attr(inst)
File "/packages/aps_ads_vm/launcher_multiapp-inplace#link-tree/torch/_dynamo/symbolic_convert.py", line 2425, in _load_attr
result = BuiltinVariable(getattr).call_function(
File "/packages/aps_ads_vm/launcher_multiapp-inplace#link-tree/torch/_dynamo/variables/builtin.py", line 1347, in call_function
return handler(tx, args, kwargs)
File "/packages/aps_ads_vm/launcher_multiapp-inplace#link-tree/torch/_dynamo/variables/builtin.py", line 967, in <lambda>
tx, [v.realize() for v in args], kwargs
File "/packages/aps_ads_vm/launcher_multiapp-inplace#link-tree/torch/_dynamo/variables/builtin.py", line 967, in <listcomp>
tx, [v.realize() for v in args], kwargs
File "/packages/aps_ads_vm/launcher_multiapp-inplace#link-tree/torch/_dynamo/variables/lazy.py", line 72, in realize
self._cache.realize()
File "/packages/aps_ads_vm/launcher_multiapp-inplace#link-tree/torch/_dynamo/variables/lazy.py", line 33, in realize
self.vt = builder.VariableBuilder(tx, self.source)(self.value)
File "/packages/aps_ads_vm/launcher_multiapp-inplace#link-tree/torch/_dynamo/variables/builder.py", line 445, in __call__
vt = self._wrap(value)
File "/packages/aps_ads_vm/launcher_multiapp-inplace#link-tree/torch/_dynamo/variables/builder.py", line 1043, in _wrap
torch._dynamo.utils.store_user_object_weakref(value)
File "/packages/aps_ads_vm/launcher_multiapp-inplace#link-tree/torch/_dynamo/utils.py", line 4694, in store_user_object_weakref
user_obj_id_to_weakref[obj_id] = weakref.ref(obj)
torch._dynamo.exc.InternalTorchDynamoError: TypeError: cannot create weak reference to 'torch.Event' object
```
This pull request makes us gracefully graph break, vs explicitly crashing.
I've added a test which reproduces the issue. There is a side discussion re:
how did torch.Event support ever work here, since it appears you cannot take a
weakref to a torch.Event
Pull Request resolved: https://github.com/pytorch/pytorch/pull/163168
Approved by: https://github.com/Lucaskabela , https://github.com/jansel
2025-09-22 22:11:09 +00:00
8e217a9f6d
[precompile] Fix issues with guard serialization on distributed types. ( #162418 )
...
Summary: Add more support for torch internal distributed data structures.
Test Plan:
test_guard_serialization.py
Rollback Plan:
Differential Revision: D81927732
Pull Request resolved: https://github.com/pytorch/pytorch/pull/162418
Approved by: https://github.com/dolpm
2025-09-11 23:09:55 +00:00
eb18d32bda
Add range_iterator ( #161800 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/161800
Approved by: https://github.com/anijain2305
ghstack dependencies: #161799
2025-09-03 16:55:04 +00:00
049c08eda8
Revert "[dynamo] [guard] Add caching for inside torch.compile.disable function to avoid unnecessary recompilation. ( #160934 )"
...
This reverts commit 8f31aa97a3e1e17bed29b6cedf9884f0c6b145e9.
Reverted https://github.com/pytorch/pytorch/pull/160934 on behalf of https://github.com/anijain2305 due to causes memory leak leading to OOMs ([comment](https://github.com/pytorch/pytorch/pull/160934#issuecomment-3234426359 ))
2025-08-28 17:56:36 +00:00
0fd63fd88b
Guard config copy for pickle errors ( #161659 )
...
Differential Revision: [D81168335](https://our.internmc.facebook.com/intern/diff/D81168335 )
Pull Request resolved: https://github.com/pytorch/pytorch/pull/161659
Approved by: https://github.com/zou3519
2025-08-28 06:27:48 +00:00
014b98dd09
Revert "Add inductor backend to device interface; make minifier_tests more device agnostic ( #151314 )"
...
This reverts commit 77bc959fe122bfd131e339ca36cab445a1860806.
Reverted https://github.com/pytorch/pytorch/pull/151314 on behalf of https://github.com/atalman due to sorry change is faling internally ([comment](https://github.com/pytorch/pytorch/pull/151314#issuecomment-3229774015 ))
2025-08-27 21:21:19 +00:00
8b78ba07b1
[dynamo, nested graph breaks] add nested graph break tests ( #144516 )
...
Note: nested graph break tests (and wrapped tests) are xfailed/skipped for now - we will iteratively enable the tests as more of the nested graph break implementation is complete.
Differential Revision: [D81084809](https://our.internmc.facebook.com/intern/diff/D81084809 )
Pull Request resolved: https://github.com/pytorch/pytorch/pull/144516
Approved by: https://github.com/anijain2305
2025-08-27 03:00:56 +00:00
6686974ddd
Revert "[dynamo, nested graph breaks] add nested graph break tests ( #144516 )"
...
This reverts commit 9a756c2d710a0680bac93ab0b42db519ec2dc6cf.
Reverted https://github.com/pytorch/pytorch/pull/144516 on behalf of https://github.com/atalman due to failing internal tests ([comment](https://github.com/pytorch/pytorch/pull/144516#issuecomment-3225659358 ))
2025-08-26 20:40:17 +00:00
77bc959fe1
Add inductor backend to device interface; make minifier_tests more device agnostic ( #151314 )
...
Tried to decouple the always cpu <=> c++, cuda <=> triton assumption. Tried to keep it relatively simple by just guarding things more specifically, at the moment.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/151314
Approved by: https://github.com/eellison
2025-08-26 19:40:37 +00:00
9a756c2d71
[dynamo, nested graph breaks] add nested graph break tests ( #144516 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/144516
Approved by: https://github.com/anijain2305
ghstack dependencies: #157971 , #159281
2025-08-26 00:57:58 +00:00
2fdd4f918c
Log exception_stack_trace to dynamo_compile ( #161096 )
...
Note: Adding unit test for this is tricky as having errors in the specific unit test would cause test_utils.py to crash all together.
Tested as follows:
1. Added x = 1/0 after guarded_code = compile_inner(code, one_graph, hooks, transform) in convert_frame.py
2. Printed exception_stack_trace and got: ['Traceback (most recent call last):\n File "/data/users/jovian/pytorch/torch/_dynamo/convert_frame.py", line 1207, in _compile\n x = 1/0\n ~^~\nZeroDivisionError: division by zero\n']
Pull Request resolved: https://github.com/pytorch/pytorch/pull/161096
Approved by: https://github.com/c00w
2025-08-22 03:29:15 +00:00
54cc63b467
[BE][Dynamo] Type coverage for symbolic_convert ( #160922 )
...
As part of better engineering, we add type coverage to `dynamo/symbolic_convert.py`, which is the main work engine of dynamo for emulating python bytecode.
Running
```
mypy torch/_dynamo/symbolic_convert.py --linecount-report /tmp/coverage_log
```
| -------- | Lines Annotated | Lines Total | % lines covered | Funcs Annotated | Funcs Total | % funcs covered |
| -------- | ------- | -------- | ------- | ------- | ------- | ------- |
| Main | 764 | 4286 | 17.83% | 43 | 241 | 17.84% |
| This PR | 4322 | 4322 | 100.00% | 241 | 241 | 100.00% |
| Delta | +3558 | +36 | +82.17% | +198 | 0 | +82.16% |
Pull Request resolved: https://github.com/pytorch/pytorch/pull/160922
Approved by: https://github.com/StrongerXi
2025-08-20 01:24:31 +00:00
8f31aa97a3
[dynamo] [guard] Add caching for inside torch.compile.disable function to avoid unnecessary recompilation. ( #160934 )
...
Fixes #157399
cherry pick of d6a5c03
@mlazos
Pull Request resolved: https://github.com/pytorch/pytorch/pull/160934
Approved by: https://github.com/mlazos
2025-08-19 06:01:26 +00:00
052c441cf4
Add logging for when inbuilt_inline_nn_modules will help with ID_MATCH guard triggered recompiles ( #160592 )
...
We add a logging around when an ID_MATCH guard is added at a place where inbuilt_inline_nn_modules would inline it. This is done with the aim of tagging recompiles that could be avoided by setting inbuilt_inline_nn_modules flag.
It will help us log and track the flag's adoption and potentially quantify saving in the the number of recompiles.
Differential Revision: D80075975
Pull Request resolved: https://github.com/pytorch/pytorch/pull/160592
Approved by: https://github.com/anijain2305
2025-08-15 17:09:39 +00:00
cd8d8c18f5
[pytorch][dynamo_compile] Log graph_node_shape to dynamo_compile ( #160556 )
...
This PR adds the dynamo graph node shape logging to dynamo compile. Also added unit tests to check if correct graph node shape is being logged.
Test Plan:
$ python -m test_utils
Ran 12 tests in 36.447s
OK
Note: Will merge after D80185628 lands.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/160556
Approved by: https://github.com/masnesral , https://github.com/jingsh
2025-08-14 16:42:35 +00:00
63654ba4c5
[BE][Dynamo] Type improvements in _dynamo/utils to generics ( #159824 )
...
Follow up to #159580
Pull Request resolved: https://github.com/pytorch/pytorch/pull/159824
Approved by: https://github.com/williamwen42
2025-08-14 16:06:50 +00:00
33d9401866
Revert "[BE][Dynamo] Type improvements in _dynamo/utils to generics ( #159824 )"
...
This reverts commit 3ef2e1ef769582a82c6ddf150e9d11bf4bf1c44f.
Reverted https://github.com/pytorch/pytorch/pull/159824 on behalf of https://github.com/clee2000 due to I think this broke dynamo/test_trace_rules.py::TraceRuleTests::test_almost_impossible_missing_name [GH job link](https://github.com/pytorch/pytorch/actions/runs/16948305999/job/48035192324 ) [HUD commit link](3ef2e1ef76https://github.com/pytorch/pytorch/pull/159824#issuecomment-3186003531 ))
2025-08-13 22:17:29 +00:00
3ef2e1ef76
[BE][Dynamo] Type improvements in _dynamo/utils to generics ( #159824 )
...
Follow up to #159580
Pull Request resolved: https://github.com/pytorch/pytorch/pull/159824
Approved by: https://github.com/williamwen42
2025-08-13 20:17:01 +00:00
9a0f7a3bb0
[retry-land][pytorch][dynamo_compile] Log stack_trace to dynamo_compile ( #160348 )
...
refer: https://github.com/pytorch/pytorch/pull/159655
Earlier pr failed on dynamo/test_utils.py::TestDynamoTimed::test_dynamo_timed.
Updated test_dynamo_timed + re-ran locally to test.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/160348
Approved by: https://github.com/masnesral
2025-08-12 06:24:54 +00:00
206c1eef65
Revert "[pytorch][dynamo_compile] Log stack_trace to dynamo_compile ( #159655 )"
...
This reverts commit 2ee22e435131369a7e4f8cc4732579acc29a941b.
Reverted https://github.com/pytorch/pytorch/pull/159655 on behalf of https://github.com/clee2000 due to broke dynamo/test_utils.py::TestDynamoTimed::test_dynamo_timed [GH job link](https://github.com/pytorch/pytorch/actions/runs/16839294394/job/47711078667 ) [HUD commit link](2ee22e4351https://github.com/pytorch/pytorch/pull/159655#issuecomment-3169400889 ))
2025-08-08 22:04:22 +00:00
2ee22e4351
[pytorch][dynamo_compile] Log stack_trace to dynamo_compile ( #159655 )
...
This change logs the stack trace of the code being compiled by Dynamo, improving visibility into what is compiled. It adds a stack_trace field to compilation metrics. This helps with debugging and analysis of Dynamo compilation behavior.
Ref [D79287964](https://www.internalfb.com/diff/D79287964 )
Test Plan:
$ python -m test_utils
Internal: ref [D79372519](https://www.internalfb.com/diff/D79372519 )
Pull Request resolved: https://github.com/pytorch/pytorch/pull/159655
Approved by: https://github.com/c00w
2025-08-08 19:53:47 +00:00
a7f3bdf550
[Dynamo][Better Engineering] Type coverage for torch/_dynamo/utils.py ( #159580 )
...
As part of better engineering effort, we would like to improve out type support to improve dev experience in dynamo
This PR adds strict typing support to `torch/_dynamo/utils.py`
Running
```
mypy torch/_dynamo/utils.py --linecount-report /tmp/coverage_log
```
| -------- | Lines Annotated | Lines Total | % lines covered | Funcs Annotated | Funcs Total | % funcs covered |
| -------- | ------- | -------- | ------- | ------- | ------- | ------- |
| Main | 2163 | 4792 | 45.14% | 121 | 268 | 45.15% |
| This PR | 4818 | 4818 | 100.00% | 268 | 268 | 100.00% |
| Delta | +2655 | +26 | +54.84% | +147 | 0 | +54.85% |
Pull Request resolved: https://github.com/pytorch/pytorch/pull/159580
Approved by: https://github.com/williamwen42
2025-08-04 21:51:53 +00:00
e4b123b5e4
Revert direct updates ( #159654 )
...
reverts:
```
commit 5711a8f06948eeee56ed5f53f171fa519f78491c (tag: trunk/5711a8f06948eeee56ed5f53f171fa519f78491c, origin/main, main)
Author: Jovian Anthony Jaison <38627145+jovianjaison@users.noreply.github.com >
Date: Fri Aug 1 09:32:52 2025 -0700
Update test_utils.py
commit b4b71d011ed07a41c2086ff0dec2988a63662877 (tag: trunk/b4b71d011ed07a41c2086ff0dec2988a63662877)
Author: Jovian Anthony Jaison <38627145+jovianjaison@users.noreply.github.com >
Date: Fri Aug 1 09:27:54 2025 -0700
Update utils.py
commit 52376b9b6fbf9fe24f5d82038dc520f0c64b6f8d (tag: trunk/52376b9b6fbf9fe24f5d82038dc520f0c64b6f8d)
Author: Jovian Anthony Jaison <38627145+jovianjaison@users.noreply.github.com >
Date: Fri Aug 1 09:26:05 2025 -0700
```
(commits pushed directly to main by mistake)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/159654
Approved by: https://github.com/atalman
2025-08-01 16:54:51 +00:00
b4b71d011e
Update utils.py
2025-08-01 09:27:54 -07:00
cb4f41e125
Revert "[dynamo] [guard] Add caching for inside torch.compile.disable function to avoid unnecessary recompilation. ( #157566 )"
...
This reverts commit 8e07c9870d07c5a318ab21bb16b3fa27576851e6.
Reverted https://github.com/pytorch/pytorch/pull/157566 on behalf of https://github.com/yangw-dev due to failed an odd internal test, please reach out to metamate to fix it, D79112610 ([comment](https://github.com/pytorch/pytorch/pull/157566#issuecomment-3141840110 ))
2025-08-01 01:27:45 +00:00
2b1ae29960
[Dynamo][Better Engineering] Add typing annotations to guard and source ( #158397 ) ( #159491 )
...
Summary:
X-link: https://github.com/pytorch/executorch/pull/12986
As part of better engineering week, we would like to improve out type support to improve dev experience in dynamo
This PR adds strict typing support to a critical set of files for dynamo, `source.py` and the base `_guards.py`
Running
```
mypy torch/_dynamo/source.py torch/_guards.py --linecount-report /tmp/coverage_log
```
| -------- | Lines Unannotated | Lines Total | % lines covered | Funcs Unannotated | Funcs Total | % funcs covered |
| -------- | ------- | -------- | ------- | ------- | ------- | ------- |
| Main | 1227 | 2208 | 55.57% | 207 | 362 | 57.18% |
| This PR | 2217 | 2217 | 100.00% | 362 | 362 | 100.00% |
| Delta | +990 | +9 | +44.43% | +155 | 0 | +42.82% |
cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 jerryzh168 voznesenskym penguinwu EikanWang Guobing-Chen zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov coconutruben
Test Plan:
Imported from GitHub, without a `Test Plan:` line.
Rollback Plan:
Reviewed By: JacobSzwejbka, yangw-dev
Differential Revision: D79199389
Pulled By: Lucaskabela
Pull Request resolved: https://github.com/pytorch/pytorch/pull/159491
Approved by: https://github.com/anijain2305 , https://github.com/yangw-dev
2025-07-30 22:57:50 +00:00
d987a6f7f0
Revert "[Dynamo][Better Engineering] Add typing annotations to guard and source ( #158397 )"
...
This reverts commit abcb24f4de11f8fedf2c2c9ff53b6092ef42306d.
Reverted https://github.com/pytorch/pytorch/pull/158397 on behalf of https://github.com/yangw-dev due to Suggested to fix failing internal signals on D78911890 ([comment](https://github.com/pytorch/pytorch/pull/158397#issuecomment-3133823766 ))
2025-07-29 19:49:40 +00:00
c55e72bea1
[Re-land][Inductor] Support native Inductor as backend for MTIA ( #159211 )
...
The previous [diff/PR] (https://github.com/pytorch/pytorch/pull/158526 ) was reverted due to this docstring lint error:
<img width="1736" height="722" alt="image" src="https://github.com/user-attachments/assets/216b1720-4002-48da-b5f3-32b5d48aaa54 " />
I didn't add the docstring cause I thought I'm not supposed to add docstring for an EXISTING function.
So this diff/PR is an exactly copy of the previous one, except for adding the docstring.
-------------
This diff/PR includes the changes to support native Inductor integration for MTIA. The goal is to support `torch.compile(backend="inductor")` for MTIA. Inductor should generate code(triton kernel + python wrapper code) similar to CUDA. And the triton kernels can be launched eagerly.
The changes include:
- Add MTIA device interfaces used by Dynamo and Inductor, including APIs on device, stream, event, etc.
- Add required torch.mtia APIs, like is_bf16_supported, memory_allocated, set_stream_by_id, etc.
- MTIA specific codegen logic, for example, loading MTIA dynamic_library.
- Other necessary changes to integrate with Inductor codegn, following other devices like CUDA, XPU.
- Integrate with the [empty_strided_mtia](https://www.internalfb.com/code/fbsource/[0d017d3a4a1bdff7253f9c66a9f38e77bd62166b]/fbcode/caffe2/aten/src/ATen/native/mtia/EmptyTensor.cpp?lines=49%2C63%2C71%2C74%2C78 ) API that we’ve added for the new MTIA ATen backend.
- A change in Inductor runtime to avoid re-initialize MTIADriver.
- BUCK changes to include ATen-mtia in Inductor, and to use -USE_MTIA preprocessor flag.
- Update `test_mnist_e2e.py` to cover native Inductor as backend, using the `--use_native_inductor` flag.
- Add a personal script(`scripts/anwang/run_native_inductor_script.py`) for testing purpose.
Note:
- This approach(option 3) aims to provide a pytorch native approach of Inductor integration for MTIA, minimizing the onboarding overhead. The downside of this approach is that it doesn't leverage MTIA specific graph optimization, and is limited to eagerly launch overhead.
- MTIA will support another approach(option 2) to provide best performance, based on WrapperFxCodegen. We should be able to reuse the fundamental changes of this diff for option 2, like the device interfaces, steam/event APIs, etc, especially as WrapperFxCodegen inherits PythonWrapperCodegen.
Internal:
References:
- [post for context](https://fb.workplace.com/groups/mtiasw/permalink/1718377262384606/ )
- [Inductor integration discussion(option 1/2/3)](https://docs.google.com/document/d/1p6363OXtVIRv1hPoaKlRSK3j-iir3QIbDd5bjyqCNig/edit?tab=t.0#heading=h.7s4ns6wcnhmb )
- [Project design doc(option 3)](https://docs.google.com/document/d/1jXUmhgoV9WvkMf-bcY3Od_kK9K_RDOdgHdt1LoQ5Tc4/edit?tab=t.0#heading=h.y43gwdqlv46w )
- [early prototying diff](https://www.internalfb.com/diff/D75110196 )
- [MPS integration PR](https://github.com/pytorch/pytorch/pull/153959 )
- [empty_strided_xpu PR](https://github.com/pytorch/pytorch/pull/126678 )
Differential Revision: [D79040806](https://our.internmc.facebook.com/intern/diff/D79040806/ )
Pull Request resolved: https://github.com/pytorch/pytorch/pull/159211
Approved by: https://github.com/eellison , https://github.com/blaine-rister , https://github.com/jansel
2025-07-29 17:03:24 +00:00
14d67eec05
Revert "[dynamo][fsdp] Consistent behavior of int attributes ( #157262 )"
...
This reverts commit 9b4d938f04c95cebe0fbd96974f64c935567e039.
Reverted https://github.com/pytorch/pytorch/pull/157262 on behalf of https://github.com/ZainRizvi due to This was reverted internally. Somehow this PR didn't get reverted alongside it. See D78772867. To validate your fixes internally, you can follow the instructions here: https://fburl.com/fixing-ghfirst-reverts ([comment](https://github.com/pytorch/pytorch/pull/157262#issuecomment-3128148475 ))
2025-07-28 16:58:27 +00:00
8e07c9870d
[dynamo] [guard] Add caching for inside torch.compile.disable function to avoid unnecessary recompilation. ( #157566 )
...
inside torch.compile.disable function always triggers recompilation. because a user inside function decorated with torch._dynamo.disable would be used as an argument in the resume_in_xx function. In the current implementation, it will always be a new object, resulting in the ID_MATCH guard always failing and triggering recompilation.
Fixes https://github.com/pytorch/pytorch/issues/157399
@xmfan
Pull Request resolved: https://github.com/pytorch/pytorch/pull/157566
Approved by: https://github.com/mlazos , https://github.com/anijain2305
2025-07-28 12:44:22 +00:00
fe0ff12dab
Revert "[Inductor] Support native Inductor as backend for MTIA ( #158526 )"
...
This reverts commit cd68559d0451185f8521912c23e77b83d76b87cf.
Reverted https://github.com/pytorch/pytorch/pull/158526 on behalf of https://github.com/facebook-github-bot due to Diff reverted internally ([comment](https://github.com/pytorch/pytorch/pull/158526#issuecomment-3122186057 ))
2025-07-26 17:58:00 +00:00
cd68559d04
[Inductor] Support native Inductor as backend for MTIA ( #158526 )
...
This diff/PR includes the changes to support native Inductor integration for MTIA. The goal is to support `torch.compile(backend="inductor")` for MTIA. Inductor should generate code(triton kernel + python wrapper code) similar to CUDA. And the triton kernels can be launched eagerly.
The changes include:
- Add MTIA device interfaces used by Dynamo and Inductor, including APIs on device, stream, event, etc.
- Add required torch.mtia APIs, like is_bf16_supported, memory_allocated, set_stream_by_id, etc.
- MTIA specific codegen logic, for example, loading MTIA dynamic_library.
- Other necessary changes to integrate with Inductor codegn, following other devices like CUDA, XPU.
- Integrate with the [empty_strided_mtia](https://www.internalfb.com/code/fbsource/[0d017d3a4a1bdff7253f9c66a9f38e77bd62166b]/fbcode/caffe2/aten/src/ATen/native/mtia/EmptyTensor.cpp?lines=49%2C63%2C71%2C74%2C78 ) API that we’ve added for the new MTIA ATen backend.
- A change in Inductor runtime to avoid re-initialize MTIADriver.
- BUCK changes to include ATen-mtia in Inductor, and to use -USE_MTIA preprocessor flag.
- Update `test_mnist_e2e.py` to cover native Inductor as backend, using the `--use_native_inductor` flag.
- Add a personal script(`scripts/anwang/run_native_inductor_script.py`) for testing purpose.
Note:
- This approach(option 3) aims to provide a pytorch native approach of Inductor integration for MTIA, minimizing the onboarding overhead. The downside of this approach is that it doesn't leverage MTIA specific graph optimization, and is limited to eagerly launch overhead.
- MTIA will support another approach(option 2) to provide best performance, based on WrapperFxCodegen. We should be able to reuse the fundamental changes of this diff for option 2, like the device interfaces, steam/event APIs, etc, especially as WrapperFxCodegen inherits PythonWrapperCodegen.
Internal:
References:
- [post for context](https://fb.workplace.com/groups/mtiasw/permalink/1718377262384606/ )
- [Inductor integration discussion(option 1/2/3)](https://docs.google.com/document/d/1p6363OXtVIRv1hPoaKlRSK3j-iir3QIbDd5bjyqCNig/edit?tab=t.0#heading=h.7s4ns6wcnhmb )
- [Project design doc(option 3)](https://docs.google.com/document/d/1jXUmhgoV9WvkMf-bcY3Od_kK9K_RDOdgHdt1LoQ5Tc4/edit?tab=t.0#heading=h.y43gwdqlv46w )
- [early prototying diff](https://www.internalfb.com/diff/D75110196 )
- [MPS integration PR](https://github.com/pytorch/pytorch/pull/153959 )
- [empty_strided_xpu PR](https://github.com/pytorch/pytorch/pull/126678 )
Differential Revision: [D78458745](https://our.internmc.facebook.com/intern/diff/D78458745/ )
Pull Request resolved: https://github.com/pytorch/pytorch/pull/158526
Approved by: https://github.com/blaine-rister , https://github.com/jansel , https://github.com/eellison
2025-07-26 08:16:34 +00:00
abcb24f4de
[Dynamo][Better Engineering] Add typing annotations to guard and source ( #158397 )
...
As part of better engineering week, we would like to improve out type support to improve dev experience in dynamo
This PR adds strict typing support to a critical set of files for dynamo, `source.py` and the base `_guards.py`
Running
```
mypy torch/_dynamo/source.py torch/_guards.py --linecount-report /tmp/coverage_log
```
| -------- | Lines Unannotated | Lines Total | % lines covered | Funcs Unannotated | Funcs Total | % funcs covered |
| -------- | ------- | -------- | ------- | ------- | ------- | ------- |
| Main | 1227 | 2208 | 55.57% | 207 | 362 | 57.18% |
| This PR | 2217 | 2217 | 100.00% | 362 | 362 | 100.00% |
| Delta | +990 | +9 | +44.43% | +155 | 0 | +42.82% |
Pull Request resolved: https://github.com/pytorch/pytorch/pull/158397
Approved by: https://github.com/anijain2305
2025-07-24 15:55:18 +00:00
9b4d938f04
[dynamo][fsdp] Consistent behavior of int attributes ( #157262 )
...
Reimpl of https://github.com/pytorch/pytorch/pull/150954
Pull Request resolved: https://github.com/pytorch/pytorch/pull/157262
Approved by: https://github.com/bdhirsh
2025-07-22 11:26:54 +00:00
9498d95b9c
[Dynamo][BetterEngineering] Type trace_rules.py ( #158679 )
...
As part of better engineering week, we would like to improve out type support to improve dev experience in dynamo
This PR adds strict typing support to a core file, `trace_rules.py`
Running
```
mypy torch/_dynamo/trace_rules.py --linecount-report /tmp/coverage_log
```
| -------- | Lines Unannotated | Lines Total | % lines covered | Funcs Unannotated | Funcs Total | % funcs covered |
| -------- | ------- | -------- | ------- | ------- | ------- | ------- |
| Main | 2564 | 3997 | 64.15% | 34 | 53 | 64.15% |
| This PR | 4022 | 4022 | 100.00% | 53 | 53 | 100.00% |
| Delta | +1458 | +25 | +35.85% | +19 | 0 | +35.85% |
Pull Request resolved: https://github.com/pytorch/pytorch/pull/158679
Approved by: https://github.com/williamwen42
2025-07-21 22:12:59 +00:00
b1a0c34dd3
[pt2 event logging] add configurable prefix ( #157678 )
...
Summary:
# Why
make experiments easier to find
# What
- dynamo config to provide a prefix
- use the prefix when sending data to scuba through the self.id_ field
Test Plan:
```
# code edited to set the prefix as `coconutruben-02`
buck2 run mode/opt scripts/coconutruben/torchmm:experiment 2>&1 | tee /tmp/epx040
```
on scuba
```
| autotune_dtypes | autotune_offset | autotune_shape | autotune_strides | event | run_id |
| -----| -----| -----| -----| -----| ----- |
| "torch.float16, torch.float16" | "0, 0" | "4096x3008, 3008x2048" | "[3008, 1], [2048, 1]" | "mm_template_autotuning" | "coconutruben-02-e6bdccc5-6dcf-4d68-9a04-b34f2c6d94fd" |
| "torch.float16, torch.float16" | "0, 0" | "4096x3008, 3008x2048" | "[3008, 1], [2048, 1]" | "mm_template_autotuning" | "coconutruben-02-14165153-5842-4eaa-9e6c-3b0cbc016375" |
```
Rollback Plan:
Differential Revision: D77837550
Pull Request resolved: https://github.com/pytorch/pytorch/pull/157678
Approved by: https://github.com/stashuk-olek
2025-07-21 20:41:03 +00:00
22920c9138
Grab bag of (mostly) typing improvements ( #158075 )
...
Collects some scattershot improvements made while attempting to enable training for AOTInductor. Non-typing changes are:
1. Swapping a few custom searches for the output node in an FX graph for calling `graph.output_node()`.
2. Removing two unused parameters from `torch.export._unlift._unlift`.
3. Switching handles to constants in `cpp_wrapper_cpu` to use C++ references for memory efficiency.
4. Cleaning out unused, unexported imports from `torch/export/__init__.py`, and adding one missing export to `__all__`.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/158075
Approved by: https://github.com/Skylion007
2025-07-21 19:17:01 +00:00
89850bbc07
[Dynamo] Use proper sources for constructing dataclass defaults ( #157993 )
...
Partially fixes https://github.com/pytorch/pytorch/issues/154009
Pull Request resolved: https://github.com/pytorch/pytorch/pull/157993
Approved by: https://github.com/williamwen42 , https://github.com/anijain2305
2025-07-18 21:51:40 +00:00
94995eba07
[Log] add a hook for recompile user context ( #157961 )
...
Users may want compile-related but customized logging info to dynamo_compile. One example is to logging the current training iteration index when recompilation happens. In general, current training iteration index is not available to compiler, since the same compiled function may be called multiple times in the same training iteration. The user could provide the training iteration index in a user hook where torch.compile logs it when recompilation happens.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/157961
Approved by: https://github.com/masnesral
2025-07-11 03:41:33 +00:00
e517066f41
Revert "[dynamo][fsdp] Consistent behavior of int attributes ( #157262 )"
...
This reverts commit 178fe7aa98987111a73534375099f4ad255e8b59.
Reverted https://github.com/pytorch/pytorch/pull/157262 on behalf of https://github.com/huydhn due to This fails some internal tests and needs to be relanded ([comment](https://github.com/pytorch/pytorch/pull/157262#issuecomment-3059463896 ))
2025-07-10 23:11:18 +00:00
82765dad16
Fix logging of config_suppress_errors and config_inline_inbuilt_nn_modules ( #157947 )
...
Currently ~50% of the time we fail or crash before logging metrics, so moving where this is logged will let us have more comprehensive (less-null) data.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/157947
Approved by: https://github.com/masnesral , https://github.com/jovianjaison
2025-07-10 12:05:43 +00:00
178fe7aa98
[dynamo][fsdp] Consistent behavior of int attributes ( #157262 )
...
Reimpl of https://github.com/pytorch/pytorch/pull/150954
Pull Request resolved: https://github.com/pytorch/pytorch/pull/157262
Approved by: https://github.com/bdhirsh
2025-07-08 22:11:33 +00:00
e49acfc5c5
[list] Raise exception in invalid list method call ( #156148 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/156148
Approved by: https://github.com/zou3519
ghstack dependencies: #153969
2025-07-07 14:51:10 +00:00
0e7f02fe2e
[Dynamo] [FrozensetSubclass] Add support for user defined frozensets ( #154263 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/154263
Approved by: https://github.com/williamwen42
ghstack dependencies: #153150 , #152991 , #154539 , #153553 , #154063 , #154064 , #154065 , #154066
2025-07-04 00:46:05 +00:00
22abe6ded4
[Dynamo] [SetSubclass] Add support for user defined sets ( #153553 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/153553
Approved by: https://github.com/williamwen42 , https://github.com/zou3519
ghstack dependencies: #153150 , #152991 , #154539
2025-07-04 00:45:25 +00:00
e7167dbacf
[Set] Support sets in VariableBuilder ( #153150 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/153150
Approved by: https://github.com/zou3519
2025-07-04 00:45:03 +00:00
8c0df6fe17
Revert "[dynamo][fsdp] Consistent behavior of int attributes ( #157262 )"
...
This reverts commit 42b48ee67229286127390000f103a11dfc8901f5.
Reverted https://github.com/pytorch/pytorch/pull/157262 on behalf of https://github.com/jeanschmidt due to Newly introduced tests are red in internal runs, check D77593713 ([comment](https://github.com/pytorch/pytorch/pull/157262#issuecomment-3026944993 ))
2025-07-02 08:30:39 +00:00
42b48ee672
[dynamo][fsdp] Consistent behavior of int attributes ( #157262 )
...
Reimpl of https://github.com/pytorch/pytorch/pull/150954
Pull Request resolved: https://github.com/pytorch/pytorch/pull/157262
Approved by: https://github.com/bdhirsh
2025-06-30 22:32:52 +00:00
dcb8982969
[dynamo] move error_on_graph_break out of config ( #156762 )
...
error_on_graph_break doesn't need to be in config, so we move it out. It should make the functorch_maml_omniglot regression less severe.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/156762
Approved by: https://github.com/jansel
ghstack dependencies: #154283 , #154289 , #154782
2025-06-26 21:40:38 +00:00
1b2146fc6d
[BE][4/16] fix typos in torch/ (torch/_dynamo/) ( #156314 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/156314
Approved by: https://github.com/jingsh
ghstack dependencies: #156313
2025-06-23 02:57:19 +00:00
