mirror of
https://github.com/pytorch/pytorch.git
synced 2025-10-20 21:14:14 +08:00
e7a9b0d62f
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/23269 Test Plan: Imported from OSS Differential Revision: D16454878 fbshipit-source-id: b1e840fc2d3901955280d141e5ad6efd5e9d66af
34 lines
1.8 KiB
Markdown
34 lines
1.8 KiB
Markdown
## Autograd
|
|
|
|
Autograd is a hotspot for PyTorch performance, so most of the heavy lifting is
|
|
implemented in C++. This implies that we have to do some shuffling between
|
|
Python and C++; and in general, we want data to be in a form that is convenient
|
|
to manipulate from C++.
|
|
|
|
Our general model is that for any key data type that autograd manipulates,
|
|
there are two implementations: a C++ type and a Python object type. For
|
|
example, consider variables in autograd: we have both `Variable` in `variable.h`
|
|
(the C++ type) and `THPVariable` in `python_variable.h` (the Python type.)
|
|
(By the way, THP stands for TorcH Python, not to be confused with THPP, TorcH
|
|
C++). `Variable` contains the payload of a variable, while `THPVariable` just
|
|
contains a `shared_ptr` reference to `Variable`, as well as references to other
|
|
Python objects which the Python runtime needs to know about. A lot of
|
|
data accessor implementations in `python_variable.cpp` simply reach through
|
|
to the underlying `Variable` and return the appropriate value.
|
|
|
|
The most complicated application of this principle is Function, which also
|
|
supports users implementing custom behavior in Python. We have the following
|
|
classes:
|
|
|
|
* `Node` in `function.h`, the C++ type.
|
|
* `THPFunction` in `python_function.h`, the Python object type. In
|
|
`python_function.cpp`, you can see the boilerplate that tells the Python
|
|
interpreter about this object.
|
|
* `PyNode` in `python_function.h`, a subclass of `Node` which forwards
|
|
`apply` to a Python `THPFunction`. (NOT a Python object, despite its name!)
|
|
|
|
Outside of `PyNode`, the C++ objects largely avoid referencing Python
|
|
objects (there are a few exceptions, like `pyobj` in `Variable`, and
|
|
`PyNode`, whose whole point is to let C++ call into Python). And `pyobj`
|
|
in `Node` to ensure uniqueness of the associated python wrapper (if it exists).
|