mirror of
https://github.com/pytorch/pytorch.git
synced 2025-10-21 05:34:18 +08:00
8af1c655e5
it seems like `_disable_dynamo` actually has a fair amount of overhead (especially when it was added to `DTensor.__new__`: this change speeds up @wanchaol 's repro from 0.380 -> 0.312s: P1378202570 (that repro runs a vanilla MLP using 2D parallelism, and calls the DTensor constructor 1280 times). It looks like most of the slowndown is in the fact that we are repeatedly running `import torch._dynamo` and constructing an instance of `torch._dynamo.disable(fn, recursive)` on every call to the constructor - this PR caches it on the first invocation. ~~Update: I realized I cannot use `torch.compiler.is_compiling` to know when to fast-path, because when we hit a graph break, cpython will be running so it will return False.~~ ~~As a test / potential fix, I added a new config, `torch._dynamo.config._is_compiling` that is set to True **always** inside a compiled region (even on frames that are run by cpython). This definitely seems to do what I want in terms of knowing when to fastpath and avoid overhead - although interested in feedback on how reasonable this is~~ Pull Request resolved: https://github.com/pytorch/pytorch/pull/127325 Approved by: https://github.com/wanchaol, https://github.com/anijain2305
37 lines
1.2 KiB
Python
37 lines
1.2 KiB
Python
"""
|
|
APIs related to torch.compile which lazily import torch._dynamo to avoid
|
|
circular dependencies.
|
|
"""
|
|
import functools
|
|
|
|
|
|
def _disable_dynamo(fn=None, recursive=True):
|
|
"""
|
|
This API should be only used inside torch, external users should still use
|
|
torch._dynamo.disable. The main goal of this API is to avoid circular
|
|
imports issues that is common while using _dynamo.disable inside torch
|
|
itself.
|
|
|
|
This API avoids it by lazily importing torch._dynamo from the import time to
|
|
the invocation of the decorated function.
|
|
"""
|
|
if fn is not None:
|
|
|
|
@functools.wraps(fn)
|
|
def inner(*args, **kwargs):
|
|
# cache this on the first invocation to avoid adding too much overhead.
|
|
disable_fn = getattr(fn, "__dynamo_disable", None)
|
|
if disable_fn is None:
|
|
import torch._dynamo
|
|
|
|
disable_fn = torch._dynamo.disable(fn, recursive)
|
|
fn.__dynamo_disable = disable_fn
|
|
|
|
return disable_fn(*args, **kwargs)
|
|
|
|
return inner
|
|
else:
|
|
# decorator usage like @_disable_dynamo(recursive=False). The resulting
|
|
# object expects the original decorated function as the arg.
|
|
return functools.partial(_disable_dynamo, recursive=recursive)
|