92108f4abd
Helper to augment graph with additional deps ( #163959 )
...
In comm-compute overlap we will have a graph with:
```
def foo(...):
ag = all_gather(...)
hiding_compute = mm(...)
wait(ag)
```
There is no explicit dependency between the hiding compute and the collectives, but we want to add implicit dependencies from wait->hiding_compute, and from hiding_compute->all_gather to preserve overlap.
Additionally, while bucketing, we will merge collective starts and collective waits together. In this case, we will want to treat the two nodes as a single subgraph - each node in the merged set will have the union of all deps in the set.
This pr adds `AugmentedGraphHelper` that adds the apis, and allows querying for dependency with this augmented graph.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/163959
Approved by: https://github.com/v0i0 , https://github.com/IvanKobzarev
ghstack dependencies: #163215 , #163754
2025-09-30 04:53:58 +00:00
84dc54ae5e
Revert "Helper to augment graph with additional deps ( #163959 )"
...
This reverts commit b5d4d350f573db12b8181ee13f9386d6ef8a1e57.
Reverted https://github.com/pytorch/pytorch/pull/163959 on behalf of https://github.com/yangw-dev due to seems fails inductor/test_aten_comm_compute_reordering for macos test, see c9b5af9a38 (51526707590-box)https://github.com/pytorch/pytorch/pull/163215#issuecomment-3349177940 ))
2025-09-29 21:53:42 +00:00
b5d4d350f5
Helper to augment graph with additional deps ( #163959 )
...
In comm-compute overlap we will have a graph with:
```
def foo(...):
ag = all_gather(...)
hiding_compute = mm(...)
wait(ag)
```
There is no explicit dependency between the hiding compute and the collectives, but we want to add implicit dependencies from wait->hiding_compute, and from hiding_compute->all_gather to preserve overlap.
Additionally, while bucketing, we will merge collective starts and collective waits together. In this case, we will want to treat the two nodes as a single subgraph - each node in the merged set will have the union of all deps in the set.
This pr adds `AugmentedGraphHelper` that adds the apis, and allows querying for dependency with this augmented graph.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/163959
Approved by: https://github.com/v0i0 , https://github.com/IvanKobzarev
ghstack dependencies: #163215 , #163754
2025-09-29 20:43:12 +00:00
450517f346
[Dynamo][Hierarchical Compile] Flatten tuple inputs for regions ( #158812 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/158812
Approved by: https://github.com/anijain2305
ghstack dependencies: #158810 , #158811
2025-08-18 18:03:11 +00:00
fb7e60ba7a
[Dynamo][Hierarchical Compile] Flatten tuple outputs in graph dedupe pass ( #158811 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/158811
Approved by: https://github.com/anijain2305
ghstack dependencies: #158810
2025-08-16 04:45:31 +00:00
371eacb2ae
[Dynamo][Hierarchical Compile] Refactor for tuple flattening ( #158810 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/158810
Approved by: https://github.com/StrongerXi
2025-08-14 22:45:44 +00:00
d0faa9985d
[Dynamo] Fix typing in graph_deduplication.py ( #152572 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/152572
Approved by: https://github.com/Skylion007 , https://github.com/anijain2305
ghstack dependencies: #152389 , #152505 , #152410 , #152506 , #152570
2025-05-13 12:17:59 +00:00
a415c9831f
[Hierarchical Compile] Replace tracing alias and mutation check with dynamo impl ( #152570 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/152570
Approved by: https://github.com/anijain2305
ghstack dependencies: #152389 , #152505 , #152410 , #152506
2025-05-13 12:17:59 +00:00
57dafb90ef
[Hierarchical Compile] Take into account mutation deps in cycle detection ( #152506 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/152506
Approved by: https://github.com/anijain2305
ghstack dependencies: #152389 , #152505 , #152410
2025-05-13 12:17:59 +00:00
118192011e
[Hierarchical Compile] Add mutation dependencies to topological sorting ( #152410 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/152410
Approved by: https://github.com/anijain2305
ghstack dependencies: #152389 , #152505
2025-05-13 12:17:59 +00:00
3592cb52d9
[Hierarchical Compilation] Use universal flatten APIs ( #152505 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/152505
Approved by: https://github.com/anijain2305
ghstack dependencies: #152389
2025-05-13 12:17:59 +00:00
78d752e96a
Revert "[Hierarchical Compilation] Use universal flatten APIs ( #152505 )"
...
This reverts commit f9e3a9058e80fde310e5f0919d3a21e28cd024a8.
Reverted https://github.com/pytorch/pytorch/pull/152505 on behalf of https://github.com/jeanschmidt due to [TENTATIVE] reverting to check if reverting this stack partially caused the introduction of https://github.com/pytorch/pytorch/actions/runs/14966121510/job/42049638969#step:22:875 ([comment](https://github.com/pytorch/pytorch/pull/152505#issuecomment-2872869990 ))
2025-05-12 14:48:08 +00:00
47df195065
Revert "[Hierarchical Compile] Add mutation dependencies to topological sorting ( #152410 )"
...
This reverts commit bc8b305eb816106de31602f8b7fd80d4113e6ee8.
Reverted https://github.com/pytorch/pytorch/pull/152410 on behalf of https://github.com/jeanschmidt due to Breaking internal signal citadel-fbcode-test-mode-opt-for-pt2_stack_for_internal-linux-0 please see diff [D74531503](https://www.internalfb.com/diff/D74531503 ) for more details ([comment](https://github.com/pytorch/pytorch/pull/152410#issuecomment-2871168679 ))
2025-05-12 07:15:09 +00:00
0e36887209
Revert "[Hierarchical Compile] Take into account mutation deps in cycle detection ( #152506 )"
...
This reverts commit 779e647999645d19eebf01fa686fb792176f8940.
Reverted https://github.com/pytorch/pytorch/pull/152506 on behalf of https://github.com/jeanschmidt due to Breaking internal signal citadel-fbcode-test-mode-opt-for-pt2_stack_for_internal-linux-0 please see diff [D74531503](https://www.internalfb.com/diff/D74531503 ) for more details ([comment](https://github.com/pytorch/pytorch/pull/152410#issuecomment-2871168679 ))
2025-05-12 07:15:09 +00:00
53ebcabb52
Revert "[Hierarchical Compile] Replace tracing alias and mutation check with dynamo impl ( #152570 )"
...
This reverts commit 50df08eb5e4d9276b72929fd859ad892880bab0f.
Reverted https://github.com/pytorch/pytorch/pull/152570 on behalf of https://github.com/jeanschmidt due to Breaking internal signal citadel-fbcode-test-mode-opt-for-pt2_stack_for_internal-linux-0 please see diff [D74531503](https://www.internalfb.com/diff/D74531503 ) for more details ([comment](https://github.com/pytorch/pytorch/pull/152410#issuecomment-2871168679 ))
2025-05-12 07:15:09 +00:00
0071fdab9e
Revert "[Dynamo] Fix typing in graph_deduplication.py ( #152572 )"
...
This reverts commit 15166be691454f8a0e626b54b6be0bea51938f86.
Reverted https://github.com/pytorch/pytorch/pull/152572 on behalf of https://github.com/jeanschmidt due to Breaking internal signal citadel-fbcode-test-mode-opt-for-pt2_stack_for_internal-linux-0 please see diff [D74531503](https://www.internalfb.com/diff/D74531503 ) for more details ([comment](https://github.com/pytorch/pytorch/pull/152410#issuecomment-2871168679 ))
2025-05-12 07:15:09 +00:00
15166be691
[Dynamo] Fix typing in graph_deduplication.py ( #152572 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/152572
Approved by: https://github.com/Skylion007 , https://github.com/anijain2305
ghstack dependencies: #152389 , #152505 , #152410 , #152506 , #152570
2025-05-10 08:27:56 +00:00
50df08eb5e
[Hierarchical Compile] Replace tracing alias and mutation check with dynamo impl ( #152570 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/152570
Approved by: https://github.com/anijain2305
ghstack dependencies: #152389 , #152505 , #152410 , #152506
2025-05-10 08:27:45 +00:00
779e647999
[Hierarchical Compile] Take into account mutation deps in cycle detection ( #152506 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/152506
Approved by: https://github.com/anijain2305
ghstack dependencies: #152389 , #152505 , #152410
2025-05-10 08:27:31 +00:00
bc8b305eb8
[Hierarchical Compile] Add mutation dependencies to topological sorting ( #152410 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/152410
Approved by: https://github.com/anijain2305
ghstack dependencies: #152389 , #152505
2025-05-10 08:27:19 +00:00
f9e3a9058e
[Hierarchical Compilation] Use universal flatten APIs ( #152505 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/152505
Approved by: https://github.com/anijain2305
ghstack dependencies: #152389
2025-05-10 08:27:07 +00:00
4649fd17b0
[invoke_subgraph] Unpacked operands ( #152547 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/152547
Approved by: https://github.com/ydwu4 , https://github.com/zou3519
2025-05-02 05:44:46 +00:00
a09a3f4c30
[Hierarchical compile] Ensure output nodes are sorted last ( #151295 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/151295
Approved by: https://github.com/anijain2305
ghstack dependencies: #151293 , #151294
2025-04-22 05:13:07 +00:00
283884b224
[Hierarchical Compile] Handle autocast ctx manager ( #151294 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/151294
Approved by: https://github.com/anijain2305
ghstack dependencies: #151293
2025-04-22 05:13:07 +00:00
0d44a8aea1
[Hierarchical Compile] Apply deduplication after output node creation ( #150306 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/150306
Approved by: https://github.com/anijain2305
ghstack dependencies: #150303 , #150304 , #150305
2025-04-01 20:54:18 +00:00
8740ffa760
[Hierarchical Compile] Add cycle detection to graph region expansion ( #150305 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/150305
Approved by: https://github.com/anijain2305
ghstack dependencies: #150303 , #150304
2025-04-01 20:54:18 +00:00
d2c0c65ea1
[Dynamo] Add debug linting option for graph dedupe ( #150053 )
...
As title
Pull Request resolved: https://github.com/pytorch/pytorch/pull/150053
Approved by: https://github.com/StrongerXi , https://github.com/anijain2305
2025-03-28 14:27:09 +00:00
21c2565f35
Document dynamo ( #146736 )
...
Many files in dynamo are currently lacking file/module-level documentation, which makes it hard to know what they do at a glance and without digging into the code. This fixes that.
Note: documentation was AI-generated and could be incorrect, please review carefully.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/146736
Approved by: https://github.com/jansel , https://github.com/StrongerXi , https://github.com/anijain2305 , https://github.com/zou3519
2025-02-13 00:02:21 +00:00
a79100ab11
PEP585 update - torch/_dynamo ( #145105 )
...
See #145101 for details.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/145105
Approved by: https://github.com/bobrenjc93
2025-01-18 20:47:11 +00:00
1fe3af2c68
Migrate from Tuple -> tuple in torch/_dynamo ( #144261 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/144261
Approved by: https://github.com/aorenste , https://github.com/zou3519
2025-01-10 07:45:57 +00:00
fd23cf5848
[Dynamo] check node class first for graph dedup ( #143609 )
...
as title
Pull Request resolved: https://github.com/pytorch/pytorch/pull/143609
Approved by: https://github.com/williamwen42
2024-12-20 04:09:46 +00:00
4eafbe5288
[Dynamo] Flatten slices during graph deduplication ( #143522 )
...
I encountered this issue while debugging torchtune - overall we need to make sure to not miss nodes that are slice arguments.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/143522
Approved by: https://github.com/williamwen42
2024-12-18 23:12:34 +00:00
c31543c7ae
[Dynamo] Initial deduplication pass impl ( #141383 )
...
This PR implements the deduplication pass (blocked by config currently) for dynamo where identical regions from https://github.com/pytorch/pytorch/pull/141381 are replaced with a common subgraph.
The two phases of deduplication are explained below.
**Subgraph creation**:
Subgraph creation works by taking one representative region from each region group and creating a subgraph from it, which will then be used to replace all regions in the group. This is implemented by first copying all nodes of the region to the new subgraph and then finding all inputs which are not within the region and creating placeholders for them. For the outputs, all regions in a region group need to be scanned to ensure the largest set of outputs is found, and then an output node is created which returns a tuple of all outputs.
**Graph replacement**:
To replace each region with the extracted subgraph, the node index in the region and argument index within the node's flattened args and kwargs are recorded once during subgraph creation. This allows us to determine which (external to the region) nodes and in which order these nodes are passed as inputs. For the outputs, getitem nodes are created for each output, and all nodes in the region with external outputs are replaced by the proper getitem node. Finally, all original nodes are erased (there should be no uses of these left in the graph).
Pull Request resolved: https://github.com/pytorch/pytorch/pull/141383
Approved by: https://github.com/zou3519
ghstack dependencies: #141381 , #141382
2024-12-11 02:22:21 +00:00
