mirror of
https://github.com/pytorch/pytorch.git
synced 2025-10-21 05:34:18 +08:00
Fixes #155027 Converted RST files to Markdown Pull Request resolved: https://github.com/pytorch/pytorch/pull/155252 Approved by: https://github.com/svekars Co-authored-by: Svetlana Karslioglu <svekars@meta.com>
This commit is contained in:
committed by
PyTorch MergeBot
parent
3d82a1dfb5
commit
d41f62b7a0
@ -1,8 +1,14 @@
|
|||||||
torch.mps
|
# torch.mps
|
||||||
===================================
|
|
||||||
.. automodule:: torch.mps
|
|
||||||
.. currentmodule:: torch.mps
|
|
||||||
|
|
||||||
|
```{eval-rst}
|
||||||
|
.. automodule:: torch.mps
|
||||||
|
```
|
||||||
|
|
||||||
|
```{eval-rst}
|
||||||
|
.. currentmodule:: torch.mps
|
||||||
|
```
|
||||||
|
|
||||||
|
```{eval-rst}
|
||||||
.. autosummary::
|
.. autosummary::
|
||||||
:toctree: generated
|
:toctree: generated
|
||||||
:nosignatures:
|
:nosignatures:
|
||||||
@ -19,9 +25,11 @@ torch.mps
|
|||||||
driver_allocated_memory
|
driver_allocated_memory
|
||||||
recommended_max_memory
|
recommended_max_memory
|
||||||
compile_shader
|
compile_shader
|
||||||
|
```
|
||||||
|
|
||||||
MPS Profiler
|
## MPS Profiler
|
||||||
------------
|
|
||||||
|
```{eval-rst}
|
||||||
.. autosummary::
|
.. autosummary::
|
||||||
:toctree: generated
|
:toctree: generated
|
||||||
:nosignatures:
|
:nosignatures:
|
||||||
@ -33,17 +41,27 @@ MPS Profiler
|
|||||||
profiler.is_capturing_metal
|
profiler.is_capturing_metal
|
||||||
profiler.is_metal_capture_enabled
|
profiler.is_metal_capture_enabled
|
||||||
profiler.metal_capture
|
profiler.metal_capture
|
||||||
|
```
|
||||||
|
|
||||||
MPS Event
|
## MPS Event
|
||||||
------------
|
|
||||||
|
```{eval-rst}
|
||||||
.. autosummary::
|
.. autosummary::
|
||||||
:toctree: generated
|
:toctree: generated
|
||||||
:nosignatures:
|
:nosignatures:
|
||||||
|
|
||||||
event.Event
|
event.Event
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
.. This module needs to be documented. Adding here in the meantime
|
% This module needs to be documented. Adding here in the meantime
|
||||||
.. for tracking purposes
|
|
||||||
|
% for tracking purposes
|
||||||
|
|
||||||
|
```{eval-rst}
|
||||||
.. py:module:: torch.mps.event
|
.. py:module:: torch.mps.event
|
||||||
|
```
|
||||||
|
|
||||||
|
```{eval-rst}
|
||||||
.. py:module:: torch.mps.profiler
|
.. py:module:: torch.mps.profiler
|
||||||
|
```
|
@ -1,11 +1,16 @@
|
|||||||
torch.mtia
|
# torch.mtia
|
||||||
===================================
|
|
||||||
|
|
||||||
The MTIA backend is implemented out of the tree, only interfaces are be defined here.
|
The MTIA backend is implemented out of the tree, only interfaces are be defined here.
|
||||||
|
|
||||||
|
```{eval-rst}
|
||||||
.. automodule:: torch.mtia
|
.. automodule:: torch.mtia
|
||||||
.. currentmodule:: torch.mtia
|
```
|
||||||
|
|
||||||
|
```{eval-rst}
|
||||||
|
.. currentmodule:: torch.mtia
|
||||||
|
```
|
||||||
|
|
||||||
|
```{eval-rst}
|
||||||
.. autosummary::
|
.. autosummary::
|
||||||
:toctree: generated
|
:toctree: generated
|
||||||
:nosignatures:
|
:nosignatures:
|
||||||
@ -32,12 +37,15 @@ The MTIA backend is implemented out of the tree, only interfaces are be defined
|
|||||||
set_rng_state
|
set_rng_state
|
||||||
get_rng_state
|
get_rng_state
|
||||||
DeferredMtiaCallError
|
DeferredMtiaCallError
|
||||||
|
```
|
||||||
|
|
||||||
Streams and events
|
## Streams and events
|
||||||
------------------
|
|
||||||
|
```{eval-rst}
|
||||||
.. autosummary::
|
.. autosummary::
|
||||||
:toctree: generated
|
:toctree: generated
|
||||||
:nosignatures:
|
:nosignatures:
|
||||||
|
|
||||||
Event
|
Event
|
||||||
Stream
|
Stream
|
||||||
|
```
|
@ -1,13 +1,19 @@
|
|||||||
torch.mtia.memory
|
# torch.mtia.memory
|
||||||
===================================
|
|
||||||
|
|
||||||
The MTIA backend is implemented out of the tree, only interfaces are be defined here.
|
The MTIA backend is implemented out of the tree, only interfaces are be defined here.
|
||||||
|
|
||||||
|
```{eval-rst}
|
||||||
.. automodule:: torch.mtia.memory
|
.. automodule:: torch.mtia.memory
|
||||||
.. currentmodule:: torch.mtia.memory
|
```
|
||||||
|
|
||||||
|
```{eval-rst}
|
||||||
|
.. currentmodule:: torch.mtia.memory
|
||||||
|
```
|
||||||
|
|
||||||
|
```{eval-rst}
|
||||||
.. autosummary::
|
.. autosummary::
|
||||||
:toctree: generated
|
:toctree: generated
|
||||||
:nosignatures:
|
:nosignatures:
|
||||||
|
|
||||||
memory_stats
|
memory_stats
|
||||||
|
```
|
@ -1,136 +1,139 @@
|
|||||||
:orphan:
|
---
|
||||||
|
orphan: true
|
||||||
|
---
|
||||||
|
|
||||||
.. _multiprocessing-doc:
|
(multiprocessing-doc)=
|
||||||
|
|
||||||
Multiprocessing package - torch.multiprocessing
|
# Multiprocessing package - torch.multiprocessing
|
||||||
===============================================
|
|
||||||
|
|
||||||
|
```{eval-rst}
|
||||||
.. automodule:: torch.multiprocessing
|
.. automodule:: torch.multiprocessing
|
||||||
|
```
|
||||||
|
|
||||||
|
```{eval-rst}
|
||||||
.. currentmodule:: torch.multiprocessing
|
.. currentmodule:: torch.multiprocessing
|
||||||
|
```
|
||||||
|
|
||||||
.. warning::
|
:::{warning}
|
||||||
|
If the main process exits abruptly (e.g. because of an incoming signal),
|
||||||
|
Python's `multiprocessing` sometimes fails to clean up its children.
|
||||||
|
It's a known caveat, so if you're seeing any resource leaks after
|
||||||
|
interrupting the interpreter, it probably means that this has just happened
|
||||||
|
to you.
|
||||||
|
:::
|
||||||
|
|
||||||
If the main process exits abruptly (e.g. because of an incoming signal),
|
## Strategy management
|
||||||
Python's ``multiprocessing`` sometimes fails to clean up its children.
|
|
||||||
It's a known caveat, so if you're seeing any resource leaks after
|
|
||||||
interrupting the interpreter, it probably means that this has just happened
|
|
||||||
to you.
|
|
||||||
|
|
||||||
Strategy management
|
|
||||||
-------------------
|
|
||||||
|
|
||||||
|
```{eval-rst}
|
||||||
.. autofunction:: get_all_sharing_strategies
|
.. autofunction:: get_all_sharing_strategies
|
||||||
|
```
|
||||||
|
|
||||||
|
```{eval-rst}
|
||||||
.. autofunction:: get_sharing_strategy
|
.. autofunction:: get_sharing_strategy
|
||||||
|
```
|
||||||
|
|
||||||
|
```{eval-rst}
|
||||||
.. autofunction:: set_sharing_strategy
|
.. autofunction:: set_sharing_strategy
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
.. _multiprocessing-cuda-sharing-details:
|
(multiprocessing-cuda-sharing-details)=
|
||||||
|
|
||||||
Sharing CUDA tensors
|
## Sharing CUDA tensors
|
||||||
--------------------
|
|
||||||
|
|
||||||
Sharing CUDA tensors between processes is supported only in Python 3, using
|
Sharing CUDA tensors between processes is supported only in Python 3, using
|
||||||
a ``spawn`` or ``forkserver`` start methods.
|
a `spawn` or `forkserver` start methods.
|
||||||
|
|
||||||
|
|
||||||
Unlike CPU tensors, the sending process is required to keep the original tensor
|
Unlike CPU tensors, the sending process is required to keep the original tensor
|
||||||
as long as the receiving process retains a copy of the tensor. The refcounting is
|
as long as the receiving process retains a copy of the tensor. The refcounting is
|
||||||
implemented under the hood but requires users to follow the next best practices.
|
implemented under the hood but requires users to follow the next best practices.
|
||||||
|
|
||||||
.. warning::
|
:::{warning}
|
||||||
If the consumer process dies abnormally to a fatal signal, the shared tensor
|
If the consumer process dies abnormally to a fatal signal, the shared tensor
|
||||||
could be forever kept in memory as long as the sending process is running.
|
could be forever kept in memory as long as the sending process is running.
|
||||||
|
:::
|
||||||
|
|
||||||
1. Release memory ASAP in the consumer.
|
1. Release memory ASAP in the consumer.
|
||||||
|
|
||||||
::
|
```
|
||||||
|
## Good
|
||||||
|
x = queue.get()
|
||||||
|
# do somethings with x
|
||||||
|
del x
|
||||||
|
```
|
||||||
|
|
||||||
## Good
|
```
|
||||||
x = queue.get()
|
## Bad
|
||||||
# do somethings with x
|
x = queue.get()
|
||||||
del x
|
# do somethings with x
|
||||||
|
# do everything else (producer have to keep x in memory)
|
||||||
::
|
```
|
||||||
|
|
||||||
## Bad
|
|
||||||
x = queue.get()
|
|
||||||
# do somethings with x
|
|
||||||
# do everything else (producer have to keep x in memory)
|
|
||||||
|
|
||||||
2. Keep producer process running until all consumers exits. This will prevent
|
2. Keep producer process running until all consumers exits. This will prevent
|
||||||
the situation when the producer process releasing memory which is still in use
|
the situation when the producer process releasing memory which is still in use
|
||||||
by the consumer.
|
by the consumer.
|
||||||
|
|
||||||
::
|
```
|
||||||
|
## producer
|
||||||
|
# send tensors, do something
|
||||||
|
event.wait()
|
||||||
|
```
|
||||||
|
|
||||||
## producer
|
```
|
||||||
# send tensors, do something
|
## consumer
|
||||||
event.wait()
|
# receive tensors and use them
|
||||||
|
event.set()
|
||||||
|
```
|
||||||
::
|
|
||||||
|
|
||||||
## consumer
|
|
||||||
# receive tensors and use them
|
|
||||||
event.set()
|
|
||||||
|
|
||||||
3. Don't pass received tensors.
|
3. Don't pass received tensors.
|
||||||
|
|
||||||
::
|
```
|
||||||
|
# not going to work
|
||||||
|
x = queue.get()
|
||||||
|
queue_2.put(x)
|
||||||
|
```
|
||||||
|
|
||||||
# not going to work
|
```
|
||||||
x = queue.get()
|
# you need to create a process-local copy
|
||||||
queue_2.put(x)
|
x = queue.get()
|
||||||
|
x_clone = x.clone()
|
||||||
|
queue_2.put(x_clone)
|
||||||
|
```
|
||||||
|
|
||||||
|
```
|
||||||
|
# putting and getting from the same queue in the same process will likely end up with segfault
|
||||||
|
queue.put(tensor)
|
||||||
|
x = queue.get()
|
||||||
|
```
|
||||||
|
|
||||||
::
|
## Sharing strategies
|
||||||
|
|
||||||
# you need to create a process-local copy
|
|
||||||
x = queue.get()
|
|
||||||
x_clone = x.clone()
|
|
||||||
queue_2.put(x_clone)
|
|
||||||
|
|
||||||
|
|
||||||
::
|
|
||||||
|
|
||||||
# putting and getting from the same queue in the same process will likely end up with segfault
|
|
||||||
queue.put(tensor)
|
|
||||||
x = queue.get()
|
|
||||||
|
|
||||||
|
|
||||||
Sharing strategies
|
|
||||||
------------------
|
|
||||||
|
|
||||||
This section provides a brief overview into how different sharing strategies
|
This section provides a brief overview into how different sharing strategies
|
||||||
work. Note that it applies only to CPU tensor - CUDA tensors will always use
|
work. Note that it applies only to CPU tensor - CUDA tensors will always use
|
||||||
the CUDA API, as that's the only way they can be shared.
|
the CUDA API, as that's the only way they can be shared.
|
||||||
|
|
||||||
File descriptor - ``file_descriptor``
|
### File descriptor - `file_descriptor`
|
||||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
||||||
|
|
||||||
|
:::{note}
|
||||||
.. note::
|
This is the default strategy (except for macOS and OS X where it's not
|
||||||
|
supported).
|
||||||
This is the default strategy (except for macOS and OS X where it's not
|
:::
|
||||||
supported).
|
|
||||||
|
|
||||||
This strategy will use file descriptors as shared memory handles. Whenever a
|
This strategy will use file descriptors as shared memory handles. Whenever a
|
||||||
storage is moved to shared memory, a file descriptor obtained from ``shm_open``
|
storage is moved to shared memory, a file descriptor obtained from `shm_open`
|
||||||
is cached with the object, and when it's going to be sent to other processes,
|
is cached with the object, and when it's going to be sent to other processes,
|
||||||
the file descriptor will be transferred (e.g. via UNIX sockets) to it. The
|
the file descriptor will be transferred (e.g. via UNIX sockets) to it. The
|
||||||
receiver will also cache the file descriptor and ``mmap`` it, to obtain a shared
|
receiver will also cache the file descriptor and `mmap` it, to obtain a shared
|
||||||
view onto the storage data.
|
view onto the storage data.
|
||||||
|
|
||||||
Note that if there will be a lot of tensors shared, this strategy will keep a
|
Note that if there will be a lot of tensors shared, this strategy will keep a
|
||||||
large number of file descriptors open most of the time. If your system has low
|
large number of file descriptors open most of the time. If your system has low
|
||||||
limits for the number of open file descriptors, and you can't raise them, you
|
limits for the number of open file descriptors, and you can't raise them, you
|
||||||
should use the ``file_system`` strategy.
|
should use the `file_system` strategy.
|
||||||
|
|
||||||
File system - ``file_system``
|
### File system - `file_system`
|
||||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
||||||
|
|
||||||
This strategy will use file names given to ``shm_open`` to identify the shared
|
This strategy will use file names given to `shm_open` to identify the shared
|
||||||
memory regions. This has a benefit of not requiring the implementation to cache
|
memory regions. This has a benefit of not requiring the implementation to cache
|
||||||
the file descriptors obtained from it, but at the same time is prone to shared
|
the file descriptors obtained from it, but at the same time is prone to shared
|
||||||
memory leaks. The file can't be deleted right after its creation, because other
|
memory leaks. The file can't be deleted right after its creation, because other
|
||||||
@ -139,28 +142,27 @@ crash, or are killed, and don't call the storage destructors, the files will
|
|||||||
remain in the system. This is very serious, because they keep using up the
|
remain in the system. This is very serious, because they keep using up the
|
||||||
memory until the system is restarted, or they're freed manually.
|
memory until the system is restarted, or they're freed manually.
|
||||||
|
|
||||||
To counter the problem of shared memory file leaks, :mod:`torch.multiprocessing`
|
To counter the problem of shared memory file leaks, {mod}`torch.multiprocessing`
|
||||||
will spawn a daemon named ``torch_shm_manager`` that will isolate itself from
|
will spawn a daemon named `torch_shm_manager` that will isolate itself from
|
||||||
the current process group, and will keep track of all shared memory allocations.
|
the current process group, and will keep track of all shared memory allocations.
|
||||||
Once all processes connected to it exit, it will wait a moment to ensure there
|
Once all processes connected to it exit, it will wait a moment to ensure there
|
||||||
will be no new connections, and will iterate over all shared memory files
|
will be no new connections, and will iterate over all shared memory files
|
||||||
allocated by the group. If it finds that any of them still exist, they will be
|
allocated by the group. If it finds that any of them still exist, they will be
|
||||||
deallocated. We've tested this method and it proved to be robust to various
|
deallocated. We've tested this method and it proved to be robust to various
|
||||||
failures. Still, if your system has high enough limits, and ``file_descriptor``
|
failures. Still, if your system has high enough limits, and `file_descriptor`
|
||||||
is a supported strategy, we do not recommend switching to this one.
|
is a supported strategy, we do not recommend switching to this one.
|
||||||
|
|
||||||
Spawning subprocesses
|
## Spawning subprocesses
|
||||||
---------------------
|
|
||||||
|
|
||||||
.. note::
|
:::{note}
|
||||||
|
Available for Python >= 3.4.
|
||||||
|
|
||||||
Available for Python >= 3.4.
|
This depends on the `spawn` start method in Python's
|
||||||
|
`multiprocessing` package.
|
||||||
This depends on the ``spawn`` start method in Python's
|
:::
|
||||||
``multiprocessing`` package.
|
|
||||||
|
|
||||||
Spawning a number of subprocesses to perform some function can be done
|
Spawning a number of subprocesses to perform some function can be done
|
||||||
by creating ``Process`` instances and calling ``join`` to wait for
|
by creating `Process` instances and calling `join` to wait for
|
||||||
their completion. This approach works fine when dealing with a single
|
their completion. This approach works fine when dealing with a single
|
||||||
subprocess but presents potential issues when dealing with multiple
|
subprocess but presents potential issues when dealing with multiple
|
||||||
processes.
|
processes.
|
||||||
@ -170,27 +172,48 @@ sequentially. If they don't, and the first process does not terminate,
|
|||||||
the process termination will go unnoticed. Also, there are no native
|
the process termination will go unnoticed. Also, there are no native
|
||||||
facilities for error propagation.
|
facilities for error propagation.
|
||||||
|
|
||||||
The ``spawn`` function below addresses these concerns and takes care
|
The `spawn` function below addresses these concerns and takes care
|
||||||
of error propagation, out of order termination, and will actively
|
of error propagation, out of order termination, and will actively
|
||||||
terminate processes upon detecting an error in one of them.
|
terminate processes upon detecting an error in one of them.
|
||||||
|
|
||||||
|
```{eval-rst}
|
||||||
.. automodule:: torch.multiprocessing.spawn
|
.. automodule:: torch.multiprocessing.spawn
|
||||||
|
```
|
||||||
|
|
||||||
|
```{eval-rst}
|
||||||
.. currentmodule:: torch.multiprocessing.spawn
|
.. currentmodule:: torch.multiprocessing.spawn
|
||||||
|
```
|
||||||
|
|
||||||
|
```{eval-rst}
|
||||||
.. autofunction:: spawn
|
.. autofunction:: spawn
|
||||||
|
```
|
||||||
|
|
||||||
|
```{eval-rst}
|
||||||
.. currentmodule:: torch.multiprocessing
|
.. currentmodule:: torch.multiprocessing
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
```{eval-rst}
|
||||||
.. class:: SpawnContext
|
.. class:: SpawnContext
|
||||||
|
|
||||||
Returned by :func:`~spawn` when called with ``join=False``.
|
Returned by :func:`~spawn` when called with ``join=False``.
|
||||||
|
|
||||||
.. automethod:: join
|
.. automethod:: join
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
.. This module needs to be documented. Adding here in the meantime
|
% This module needs to be documented. Adding here in the meantime
|
||||||
.. for tracking purposes
|
|
||||||
|
% for tracking purposes
|
||||||
|
|
||||||
|
```{eval-rst}
|
||||||
.. py:module:: torch.multiprocessing.pool
|
.. py:module:: torch.multiprocessing.pool
|
||||||
|
```
|
||||||
|
|
||||||
|
```{eval-rst}
|
||||||
.. py:module:: torch.multiprocessing.queue
|
.. py:module:: torch.multiprocessing.queue
|
||||||
|
```
|
||||||
|
|
||||||
|
```{eval-rst}
|
||||||
.. py:module:: torch.multiprocessing.reductions
|
.. py:module:: torch.multiprocessing.reductions
|
||||||
|
```
|
@ -1,11 +1,12 @@
|
|||||||
|
```{eval-rst}
|
||||||
.. currentmodule:: torch
|
.. currentmodule:: torch
|
||||||
|
```
|
||||||
|
|
||||||
.. _name_inference_reference-doc:
|
(name_inference_reference-doc)=
|
||||||
|
|
||||||
Named Tensors operator coverage
|
# Named Tensors operator coverage
|
||||||
===============================
|
|
||||||
|
|
||||||
Please read :ref:`named_tensors-doc` first for an introduction to named tensors.
|
Please read {ref}`named_tensors-doc` first for an introduction to named tensors.
|
||||||
|
|
||||||
This document is a reference for *name inference*, a process that defines how
|
This document is a reference for *name inference*, a process that defines how
|
||||||
named tensors:
|
named tensors:
|
||||||
@ -17,11 +18,13 @@ Below is a list of all operations that are supported with named tensors
|
|||||||
and their associated name inference rules.
|
and their associated name inference rules.
|
||||||
|
|
||||||
If you don't see an operation listed here, but it would help your use case, please
|
If you don't see an operation listed here, but it would help your use case, please
|
||||||
`search if an issue has already been filed <https://github.com/pytorch/pytorch/issues?q=is%3Aopen+is%3Aissue+label%3A%22module%3A+named+tensor%22>`_ and if not, `file one <https://github.com/pytorch/pytorch/issues/new/choose>`_.
|
[search if an issue has already been filed](https://github.com/pytorch/pytorch/issues?q=is%3Aopen+is%3Aissue+label%3A%22module%3A+named+tensor%22) and if not, [file one](https://github.com/pytorch/pytorch/issues/new/choose).
|
||||||
|
|
||||||
.. warning::
|
:::{warning}
|
||||||
The named tensor API is experimental and subject to change.
|
The named tensor API is experimental and subject to change.
|
||||||
|
:::
|
||||||
|
|
||||||
|
```{eval-rst}
|
||||||
.. csv-table:: Supported Operations
|
.. csv-table:: Supported Operations
|
||||||
:header: API, Name inference rule
|
:header: API, Name inference rule
|
||||||
:widths: 20, 20
|
:widths: 20, 20
|
||||||
@ -244,226 +247,221 @@ If you don't see an operation listed here, but it would help your use case, plea
|
|||||||
:meth:`Tensor.zero_`,None
|
:meth:`Tensor.zero_`,None
|
||||||
:func:`torch.zeros`,:ref:`factory-doc`
|
:func:`torch.zeros`,:ref:`factory-doc`
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
.. _keeps_input_names-doc:
|
(keeps_input_names-doc)=
|
||||||
|
|
||||||
Keeps input names
|
## Keeps input names
|
||||||
^^^^^^^^^^^^^^^^^
|
|
||||||
|
|
||||||
All pointwise unary functions follow this rule as well as some other unary functions.
|
All pointwise unary functions follow this rule as well as some other unary functions.
|
||||||
|
|
||||||
- Check names: None
|
- Check names: None
|
||||||
- Propagate names: input tensor's names are propagated to the output.
|
- Propagate names: input tensor's names are propagated to the output.
|
||||||
|
|
||||||
::
|
```
|
||||||
|
>>> x = torch.randn(3, 3, names=('N', 'C'))
|
||||||
|
>>> x.abs().names
|
||||||
|
('N', 'C')
|
||||||
|
```
|
||||||
|
|
||||||
>>> x = torch.randn(3, 3, names=('N', 'C'))
|
(removes_dimensions-doc)=
|
||||||
>>> x.abs().names
|
|
||||||
('N', 'C')
|
|
||||||
|
|
||||||
.. _removes_dimensions-doc:
|
## Removes dimensions
|
||||||
|
|
||||||
Removes dimensions
|
All reduction ops like {meth}`~Tensor.sum` remove dimensions by reducing
|
||||||
^^^^^^^^^^^^^^^^^^
|
over the desired dimensions. Other operations like {meth}`~Tensor.select` and
|
||||||
|
{meth}`~Tensor.squeeze` remove dimensions.
|
||||||
All reduction ops like :meth:`~Tensor.sum` remove dimensions by reducing
|
|
||||||
over the desired dimensions. Other operations like :meth:`~Tensor.select` and
|
|
||||||
:meth:`~Tensor.squeeze` remove dimensions.
|
|
||||||
|
|
||||||
Wherever one can pass an integer dimension index to an operator, one can also pass
|
Wherever one can pass an integer dimension index to an operator, one can also pass
|
||||||
a dimension name. Functions that take lists of dimension indices can also take in a
|
a dimension name. Functions that take lists of dimension indices can also take in a
|
||||||
list of dimension names.
|
list of dimension names.
|
||||||
|
|
||||||
- Check names: If :attr:`dim` or :attr:`dims` is passed in as a list of names,
|
- Check names: If {attr}`dim` or {attr}`dims` is passed in as a list of names,
|
||||||
check that those names exist in :attr:`self`.
|
check that those names exist in {attr}`self`.
|
||||||
- Propagate names: If the dimensions of the input tensor specified by :attr:`dim`
|
- Propagate names: If the dimensions of the input tensor specified by {attr}`dim`
|
||||||
or :attr:`dims` are not present in the output tensor, then the corresponding names
|
or {attr}`dims` are not present in the output tensor, then the corresponding names
|
||||||
of those dimensions do not appear in ``output.names``.
|
of those dimensions do not appear in `output.names`.
|
||||||
|
|
||||||
::
|
```
|
||||||
|
>>> x = torch.randn(1, 3, 3, 3, names=('N', 'C', 'H', 'W'))
|
||||||
|
>>> x.squeeze('N').names
|
||||||
|
('C', 'H', 'W')
|
||||||
|
|
||||||
>>> x = torch.randn(1, 3, 3, 3, names=('N', 'C', 'H', 'W'))
|
>>> x = torch.randn(3, 3, 3, 3, names=('N', 'C', 'H', 'W'))
|
||||||
>>> x.squeeze('N').names
|
>>> x.sum(['N', 'C']).names
|
||||||
('C', 'H', 'W')
|
('H', 'W')
|
||||||
|
|
||||||
>>> x = torch.randn(3, 3, 3, 3, names=('N', 'C', 'H', 'W'))
|
# Reduction ops with keepdim=True don't actually remove dimensions.
|
||||||
>>> x.sum(['N', 'C']).names
|
>>> x = torch.randn(3, 3, 3, 3, names=('N', 'C', 'H', 'W'))
|
||||||
('H', 'W')
|
>>> x.sum(['N', 'C'], keepdim=True).names
|
||||||
|
('N', 'C', 'H', 'W')
|
||||||
|
```
|
||||||
|
|
||||||
# Reduction ops with keepdim=True don't actually remove dimensions.
|
(unifies_names_from_inputs-doc)=
|
||||||
>>> x = torch.randn(3, 3, 3, 3, names=('N', 'C', 'H', 'W'))
|
|
||||||
>>> x.sum(['N', 'C'], keepdim=True).names
|
|
||||||
('N', 'C', 'H', 'W')
|
|
||||||
|
|
||||||
|
## Unifies names from inputs
|
||||||
.. _unifies_names_from_inputs-doc:
|
|
||||||
|
|
||||||
Unifies names from inputs
|
|
||||||
^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
||||||
|
|
||||||
All binary arithmetic ops follow this rule. Operations that broadcast still
|
All binary arithmetic ops follow this rule. Operations that broadcast still
|
||||||
broadcast positionally from the right to preserve compatibility with unnamed
|
broadcast positionally from the right to preserve compatibility with unnamed
|
||||||
tensors. To perform explicit broadcasting by names, use :meth:`Tensor.align_as`.
|
tensors. To perform explicit broadcasting by names, use {meth}`Tensor.align_as`.
|
||||||
|
|
||||||
- Check names: All names must match positionally from the right. i.e., in
|
- Check names: All names must match positionally from the right. i.e., in
|
||||||
``tensor + other``, ``match(tensor.names[i], other.names[i])`` must be true for all
|
`tensor + other`, `match(tensor.names[i], other.names[i])` must be true for all
|
||||||
``i`` in ``(-min(tensor.dim(), other.dim()) + 1, -1]``.
|
`i` in `(-min(tensor.dim(), other.dim()) + 1, -1]`.
|
||||||
- Check names: Furthermore, all named dimensions must be aligned from the right.
|
- Check names: Furthermore, all named dimensions must be aligned from the right.
|
||||||
During matching, if we match a named dimension ``A`` with an unnamed dimension
|
During matching, if we match a named dimension `A` with an unnamed dimension
|
||||||
``None``, then ``A`` must not appear in the tensor with the unnamed dimension.
|
`None`, then `A` must not appear in the tensor with the unnamed dimension.
|
||||||
- Propagate names: unify pairs of names from the right from both tensors to
|
- Propagate names: unify pairs of names from the right from both tensors to
|
||||||
produce output names.
|
produce output names.
|
||||||
|
|
||||||
For example,
|
For example,
|
||||||
|
|
||||||
::
|
```
|
||||||
|
# tensor: Tensor[ N, None]
|
||||||
# tensor: Tensor[ N, None]
|
# other: Tensor[None, C]
|
||||||
# other: Tensor[None, C]
|
>>> tensor = torch.randn(3, 3, names=('N', None))
|
||||||
>>> tensor = torch.randn(3, 3, names=('N', None))
|
>>> other = torch.randn(3, 3, names=(None, 'C'))
|
||||||
>>> other = torch.randn(3, 3, names=(None, 'C'))
|
>>> (tensor + other).names
|
||||||
>>> (tensor + other).names
|
('N', 'C')
|
||||||
('N', 'C')
|
```
|
||||||
|
|
||||||
Check names:
|
Check names:
|
||||||
|
|
||||||
- ``match(tensor.names[-1], other.names[-1])`` is ``True``
|
- `match(tensor.names[-1], other.names[-1])` is `True`
|
||||||
- ``match(tensor.names[-2], tensor.names[-2])`` is ``True``
|
- `match(tensor.names[-2], tensor.names[-2])` is `True`
|
||||||
- Because we matched ``None`` in :attr:`tensor` with ``'C'``,
|
- Because we matched `None` in {attr}`tensor` with `'C'`,
|
||||||
check to make sure ``'C'`` doesn't exist in :attr:`tensor` (it does not).
|
check to make sure `'C'` doesn't exist in {attr}`tensor` (it does not).
|
||||||
- Check to make sure ``'N'`` doesn't exists in :attr:`other` (it does not).
|
- Check to make sure `'N'` doesn't exists in {attr}`other` (it does not).
|
||||||
|
|
||||||
Finally, the output names are computed with
|
Finally, the output names are computed with
|
||||||
``[unify('N', None), unify(None, 'C')] = ['N', 'C']``
|
`[unify('N', None), unify(None, 'C')] = ['N', 'C']`
|
||||||
|
|
||||||
More examples::
|
More examples:
|
||||||
|
|
||||||
# Dimensions don't match from the right:
|
```
|
||||||
# tensor: Tensor[N, C]
|
# Dimensions don't match from the right:
|
||||||
# other: Tensor[ N]
|
# tensor: Tensor[N, C]
|
||||||
>>> tensor = torch.randn(3, 3, names=('N', 'C'))
|
# other: Tensor[ N]
|
||||||
>>> other = torch.randn(3, names=('N',))
|
>>> tensor = torch.randn(3, 3, names=('N', 'C'))
|
||||||
>>> (tensor + other).names
|
>>> other = torch.randn(3, names=('N',))
|
||||||
RuntimeError: Error when attempting to broadcast dims ['N', 'C'] and dims
|
>>> (tensor + other).names
|
||||||
['N']: dim 'C' and dim 'N' are at the same position from the right but do
|
RuntimeError: Error when attempting to broadcast dims ['N', 'C'] and dims
|
||||||
not match.
|
['N']: dim 'C' and dim 'N' are at the same position from the right but do
|
||||||
|
not match.
|
||||||
|
|
||||||
# Dimensions aren't aligned when matching tensor.names[-1] and other.names[-1]:
|
# Dimensions aren't aligned when matching tensor.names[-1] and other.names[-1]:
|
||||||
# tensor: Tensor[N, None]
|
# tensor: Tensor[N, None]
|
||||||
# other: Tensor[ N]
|
# other: Tensor[ N]
|
||||||
>>> tensor = torch.randn(3, 3, names=('N', None))
|
>>> tensor = torch.randn(3, 3, names=('N', None))
|
||||||
>>> other = torch.randn(3, names=('N',))
|
>>> other = torch.randn(3, names=('N',))
|
||||||
>>> (tensor + other).names
|
>>> (tensor + other).names
|
||||||
RuntimeError: Misaligned dims when attempting to broadcast dims ['N'] and
|
RuntimeError: Misaligned dims when attempting to broadcast dims ['N'] and
|
||||||
dims ['N', None]: dim 'N' appears in a different position from the right
|
dims ['N', None]: dim 'N' appears in a different position from the right
|
||||||
across both lists.
|
across both lists.
|
||||||
|
```
|
||||||
|
|
||||||
.. note::
|
:::{note}
|
||||||
|
In both of the last examples, it is possible to align the tensors by names
|
||||||
|
and then perform the addition. Use {meth}`Tensor.align_as` to align
|
||||||
|
tensors by name or {meth}`Tensor.align_to` to align tensors to a custom
|
||||||
|
dimension ordering.
|
||||||
|
:::
|
||||||
|
|
||||||
In both of the last examples, it is possible to align the tensors by names
|
(permutes_dimensions-doc)=
|
||||||
and then perform the addition. Use :meth:`Tensor.align_as` to align
|
|
||||||
tensors by name or :meth:`Tensor.align_to` to align tensors to a custom
|
|
||||||
dimension ordering.
|
|
||||||
|
|
||||||
.. _permutes_dimensions-doc:
|
## Permutes dimensions
|
||||||
|
|
||||||
Permutes dimensions
|
Some operations, like {meth}`Tensor.t()`, permute the order of dimensions. Dimension names
|
||||||
^^^^^^^^^^^^^^^^^^^
|
|
||||||
|
|
||||||
Some operations, like :meth:`Tensor.t()`, permute the order of dimensions. Dimension names
|
|
||||||
are attached to individual dimensions so they get permuted as well.
|
are attached to individual dimensions so they get permuted as well.
|
||||||
|
|
||||||
If the operator takes in positional index :attr:`dim`, it is also able to take a dimension
|
If the operator takes in positional index {attr}`dim`, it is also able to take a dimension
|
||||||
name as :attr:`dim`.
|
name as {attr}`dim`.
|
||||||
|
|
||||||
- Check names: If :attr:`dim` is passed as a name, check that it exists in the tensor.
|
- Check names: If {attr}`dim` is passed as a name, check that it exists in the tensor.
|
||||||
- Propagate names: Permute dimension names in the same way as the dimensions that are
|
- Propagate names: Permute dimension names in the same way as the dimensions that are
|
||||||
being permuted.
|
being permuted.
|
||||||
|
|
||||||
::
|
```
|
||||||
|
>>> x = torch.randn(3, 3, names=('N', 'C'))
|
||||||
|
>>> x.transpose('N', 'C').names
|
||||||
|
('C', 'N')
|
||||||
|
```
|
||||||
|
|
||||||
>>> x = torch.randn(3, 3, names=('N', 'C'))
|
(contracts_away_dims-doc)=
|
||||||
>>> x.transpose('N', 'C').names
|
|
||||||
('C', 'N')
|
|
||||||
|
|
||||||
.. _contracts_away_dims-doc:
|
## Contracts away dims
|
||||||
|
|
||||||
Contracts away dims
|
|
||||||
^^^^^^^^^^^^^^^^^^^
|
|
||||||
|
|
||||||
Matrix multiply functions follow some variant of this. Let's go through
|
Matrix multiply functions follow some variant of this. Let's go through
|
||||||
:func:`torch.mm` first and then generalize the rule for batch matrix multiplication.
|
{func}`torch.mm` first and then generalize the rule for batch matrix multiplication.
|
||||||
|
|
||||||
For ``torch.mm(tensor, other)``:
|
For `torch.mm(tensor, other)`:
|
||||||
|
|
||||||
- Check names: None
|
- Check names: None
|
||||||
- Propagate names: result names are ``(tensor.names[-2], other.names[-1])``.
|
- Propagate names: result names are `(tensor.names[-2], other.names[-1])`.
|
||||||
|
|
||||||
::
|
```
|
||||||
|
>>> x = torch.randn(3, 3, names=('N', 'D'))
|
||||||
>>> x = torch.randn(3, 3, names=('N', 'D'))
|
>>> y = torch.randn(3, 3, names=('in', 'out'))
|
||||||
>>> y = torch.randn(3, 3, names=('in', 'out'))
|
>>> x.mm(y).names
|
||||||
>>> x.mm(y).names
|
('N', 'out')
|
||||||
('N', 'out')
|
```
|
||||||
|
|
||||||
Inherently, a matrix multiplication performs a dot product over two dimensions,
|
Inherently, a matrix multiplication performs a dot product over two dimensions,
|
||||||
collapsing them. When two tensors are matrix-multiplied, the contracted dimensions
|
collapsing them. When two tensors are matrix-multiplied, the contracted dimensions
|
||||||
disappear and do not show up in the output tensor.
|
disappear and do not show up in the output tensor.
|
||||||
|
|
||||||
:func:`torch.mv`, :func:`torch.dot` work in a similar way: name inference does not
|
{func}`torch.mv`, {func}`torch.dot` work in a similar way: name inference does not
|
||||||
check input names and removes the dimensions that are involved in the dot product:
|
check input names and removes the dimensions that are involved in the dot product:
|
||||||
|
|
||||||
::
|
```
|
||||||
|
>>> x = torch.randn(3, 3, names=('N', 'D'))
|
||||||
|
>>> y = torch.randn(3, names=('something',))
|
||||||
|
>>> x.mv(y).names
|
||||||
|
('N',)
|
||||||
|
```
|
||||||
|
|
||||||
>>> x = torch.randn(3, 3, names=('N', 'D'))
|
Now, let's take a look at `torch.matmul(tensor, other)`. Assume that `tensor.dim() >= 2`
|
||||||
>>> y = torch.randn(3, names=('something',))
|
and `other.dim() >= 2`.
|
||||||
>>> x.mv(y).names
|
|
||||||
('N',)
|
|
||||||
|
|
||||||
Now, let's take a look at ``torch.matmul(tensor, other)``. Assume that ``tensor.dim() >= 2``
|
|
||||||
and ``other.dim() >= 2``.
|
|
||||||
|
|
||||||
- Check names: Check that the batch dimensions of the inputs are aligned and broadcastable.
|
- Check names: Check that the batch dimensions of the inputs are aligned and broadcastable.
|
||||||
See :ref:`unifies_names_from_inputs-doc` for what it means for the inputs to be aligned.
|
See {ref}`unifies_names_from_inputs-doc` for what it means for the inputs to be aligned.
|
||||||
- Propagate names: result names are obtained by unifying the batch dimensions and removing
|
- Propagate names: result names are obtained by unifying the batch dimensions and removing
|
||||||
the contracted dimensions:
|
the contracted dimensions:
|
||||||
``unify(tensor.names[:-2], other.names[:-2]) + (tensor.names[-2], other.names[-1])``.
|
`unify(tensor.names[:-2], other.names[:-2]) + (tensor.names[-2], other.names[-1])`.
|
||||||
|
|
||||||
Examples::
|
Examples:
|
||||||
|
|
||||||
# Batch matrix multiply of matrices Tensor['C', 'D'] and Tensor['E', 'F'].
|
```
|
||||||
# 'A', 'B' are batch dimensions.
|
# Batch matrix multiply of matrices Tensor['C', 'D'] and Tensor['E', 'F'].
|
||||||
>>> x = torch.randn(3, 3, 3, 3, names=('A', 'B', 'C', 'D'))
|
# 'A', 'B' are batch dimensions.
|
||||||
>>> y = torch.randn(3, 3, 3, names=('B', 'E', 'F'))
|
>>> x = torch.randn(3, 3, 3, 3, names=('A', 'B', 'C', 'D'))
|
||||||
>>> torch.matmul(x, y).names
|
>>> y = torch.randn(3, 3, 3, names=('B', 'E', 'F'))
|
||||||
('A', 'B', 'C', 'F')
|
>>> torch.matmul(x, y).names
|
||||||
|
('A', 'B', 'C', 'F')
|
||||||
|
```
|
||||||
|
|
||||||
|
Finally, there are fused `add` versions of many matmul functions. i.e., {func}`addmm`
|
||||||
|
and {func}`addmv`. These are treated as composing name inference for i.e. {func}`mm` and
|
||||||
|
name inference for {func}`add`.
|
||||||
|
|
||||||
Finally, there are fused ``add`` versions of many matmul functions. i.e., :func:`addmm`
|
(factory-doc)=
|
||||||
and :func:`addmv`. These are treated as composing name inference for i.e. :func:`mm` and
|
|
||||||
name inference for :func:`add`.
|
|
||||||
|
|
||||||
.. _factory-doc:
|
## Factory functions
|
||||||
|
|
||||||
Factory functions
|
Factory functions now take a new {attr}`names` argument that associates a name
|
||||||
^^^^^^^^^^^^^^^^^
|
|
||||||
|
|
||||||
|
|
||||||
Factory functions now take a new :attr:`names` argument that associates a name
|
|
||||||
with each dimension.
|
with each dimension.
|
||||||
|
|
||||||
::
|
```
|
||||||
|
>>> torch.zeros(2, 3, names=('N', 'C'))
|
||||||
|
tensor([[0., 0., 0.],
|
||||||
|
[0., 0., 0.]], names=('N', 'C'))
|
||||||
|
```
|
||||||
|
|
||||||
>>> torch.zeros(2, 3, names=('N', 'C'))
|
(out_function_semantics-doc)=
|
||||||
tensor([[0., 0., 0.],
|
|
||||||
[0., 0., 0.]], names=('N', 'C'))
|
|
||||||
|
|
||||||
.. _out_function_semantics-doc:
|
## out function and in-place variants
|
||||||
|
|
||||||
out function and in-place variants
|
A tensor specified as an `out=` tensor has the following behavior:
|
||||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
||||||
|
|
||||||
A tensor specified as an ``out=`` tensor has the following behavior:
|
|
||||||
|
|
||||||
- If it has no named dimensions, then the names computed from the operation
|
- If it has no named dimensions, then the names computed from the operation
|
||||||
get propagated to it.
|
get propagated to it.
|
||||||
@ -473,13 +471,13 @@ A tensor specified as an ``out=`` tensor has the following behavior:
|
|||||||
All in-place methods modify inputs to have names equal to the computed names
|
All in-place methods modify inputs to have names equal to the computed names
|
||||||
from name inference. For example:
|
from name inference. For example:
|
||||||
|
|
||||||
::
|
```
|
||||||
|
>>> x = torch.randn(3, 3)
|
||||||
|
>>> y = torch.randn(3, 3, names=('N', 'C'))
|
||||||
|
>>> x.names
|
||||||
|
(None, None)
|
||||||
|
|
||||||
>>> x = torch.randn(3, 3)
|
>>> x += y
|
||||||
>>> y = torch.randn(3, 3, names=('N', 'C'))
|
>>> x.names
|
||||||
>>> x.names
|
('N', 'C')
|
||||||
(None, None)
|
```
|
||||||
|
|
||||||
>>> x += y
|
|
||||||
>>> x.names
|
|
||||||
('N', 'C')
|
|
Reference in New Issue
Block a user