mirror of
https://github.com/pytorch/pytorch.git
synced 2025-10-21 05:34:18 +08:00
414ad47045
The dtype documentation has not been updated in awhile, let's do a revamp.
1. combine the duplicated docs for dtypes from `tensors.rst` and `tensor_attributes.rst` to live in `tensor_attributes.rst`, and link to that page from `tensors.rst`
2. split the dtype table into floating point and integer dtypes
3. add the definition of shell dtype
4. add the float8 and MX dtypes as shell dtypes to the dtype table
5. remove legacy quantized dtypes from the table
6. add the definition of various dtype suffixes ("fn", etc)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/156087
Approved by: https://github.com/albanD
354 lines
15 KiB
ReStructuredText
354 lines
15 KiB
ReStructuredText
.. currentmodule:: torch
|
|
|
|
.. _tensor-attributes-doc:
|
|
|
|
Tensor Attributes
|
|
=================
|
|
|
|
Each ``torch.Tensor`` has a :class:`torch.dtype`, :class:`torch.device`, and :class:`torch.layout`.
|
|
|
|
.. _dtype-doc:
|
|
|
|
torch.dtype
|
|
-----------
|
|
|
|
.. class:: dtype
|
|
|
|
A :class:`torch.dtype` is an object that represents the data type of a
|
|
:class:`torch.Tensor`. PyTorch has several different data types:
|
|
|
|
**Floating point dtypes**
|
|
|
|
========================================= ===============================
|
|
dtype description
|
|
========================================= ===============================
|
|
``torch.float32`` or ``torch.float`` 32-bit floating point, as defined in https://en.wikipedia.org/wiki/IEEE_754
|
|
``torch.float64`` or ``torch.double`` 64-bit floating point, as defined in https://en.wikipedia.org/wiki/IEEE_754
|
|
``torch.float16`` or ``torch.half`` 16-bit floating point, as defined in https://en.wikipedia.org/wiki/IEEE_754, S-E-M 1-5-10
|
|
``torch.bfloat16`` 16-bit floating point, sometimes referred to as Brain floating point, S-E-M 1-8-7
|
|
``torch.complex32`` or ``torch.chalf`` 32-bit complex with two `float16` components
|
|
``torch.complex64`` or ``torch.cfloat`` 64-bit complex with two `float32` components
|
|
``torch.complex128`` or ``torch.cdouble`` 128-bit complex with two `float64` components
|
|
``torch.float8_e4m3fn`` [shell]_, [1]_ 8-bit floating point, S-E-M 1-4-3, from https://arxiv.org/abs/2209.05433
|
|
``torch.float8_e5m2`` [shell]_ 8-bit floating point, S-E-M 1-5-2, from https://arxiv.org/abs/2209.05433
|
|
``torch.float8_e4m3fnuz`` [shell]_, [1]_ 8-bit floating point, S-E-M 1-4-3, from https://arxiv.org/pdf/2206.02915
|
|
``torch.float8_e5m2fnuz`` [shell]_, [1]_ 8-bit floating point, S-E-M 1-5-2, from https://arxiv.org/pdf/2206.02915
|
|
``torch.float8_e8m0fnu`` [shell]_, [1]_ 8-bit floating point, S-E-M 0-8-0, from https://www.opencompute.org/documents/ocp-microscaling-formats-mx-v1-0-spec-final-pdf
|
|
``torch.float4_e2m1fn_x2`` [shell]_, [1]_ packed 4-bit floating point, S-E-M 1-2-1, from https://www.opencompute.org/documents/ocp-microscaling-formats-mx-v1-0-spec-final-pdf
|
|
========================================= ===============================
|
|
|
|
**Integer dtypes**
|
|
|
|
========================================= ===============================
|
|
dtype description
|
|
========================================= ===============================
|
|
``torch.uint8`` 8-bit integer (unsigned)
|
|
``torch.int8`` 8-bit integer (signed)
|
|
``torch.uint16`` [shell]_, [2]_ 16-bit integer (unsigned)
|
|
``torch.int16`` or ``torch.short`` 16-bit integer (signed)
|
|
``torch.uint32`` [shell]_, [2]_ 32-bit integer (unsigned)
|
|
``torch.int32`` or ``torch.int`` 32-bit integer (signed)
|
|
``torch.uint64`` [shell]_, [2]_ 64-bit integer (unsigned)
|
|
``torch.int64`` or ``torch.long`` 64-bit integer (signed)
|
|
``torch.bool`` Boolean
|
|
========================================= ===============================
|
|
|
|
.. [shell] a shell dtype a specialized dtype with limited op and backend support.
|
|
Specifically, ops that support tensor creation (``torch.empty``, ``torch.fill``, ``torch.zeros``)
|
|
and operations which do not peek inside the data elements (``torch.cat``, ``torch.view``, ``torch.reshape``)
|
|
are supported. Ops that peek inside the data elements such as casting,
|
|
matrix multiplication, nan/inf checks are supported only on a case by
|
|
case basis, depending on maturity and presence of hardware accelerated kernels
|
|
and established use cases.
|
|
|
|
.. [1] The "fn", "fnu" and "fnuz" dtype suffixes mean:
|
|
"f" - finite value encodings only, no infinity;
|
|
"n" - nan value encodings differ from the IEEE spec;
|
|
"uz" - "unsigned zero" only, i.e. no negative zero encoding
|
|
|
|
.. [2]
|
|
Unsigned types asides from ``uint8`` are currently planned to only have
|
|
limited support in eager mode (they primarily exist to assist usage with
|
|
torch.compile); if you need eager support and the extra range is not needed,
|
|
we recommend using their signed variants instead. See
|
|
https://github.com/pytorch/pytorch/issues/58734 for more details.
|
|
|
|
**Note**: legacy constructors such as ``torch.*.FloatTensor``, ``torch.*.DoubleTensor``, ``torch.*.HalfTensor``,
|
|
``torch.*.BFloat16Tensor``, ``torch.*.ByteTensor``, ``torch.*.CharTensor``, ``torch.*.ShortTensor``, ``torch.*.IntTensor``,
|
|
``torch.*.LongTensor``, ``torch.*.BoolTensor`` only remain for backwards compatibility and should no longer be used.
|
|
|
|
To find out if a :class:`torch.dtype` is a floating point data type, the property :attr:`is_floating_point`
|
|
can be used, which returns ``True`` if the data type is a floating point data type.
|
|
|
|
To find out if a :class:`torch.dtype` is a complex data type, the property :attr:`is_complex`
|
|
can be used, which returns ``True`` if the data type is a complex data type.
|
|
|
|
.. _type-promotion-doc:
|
|
|
|
When the dtypes of inputs to an arithmetic operation (`add`, `sub`, `div`, `mul`) differ, we promote
|
|
by finding the minimum dtype that satisfies the following rules:
|
|
|
|
* If the type of a scalar operand is of a higher category than tensor operands
|
|
(where complex > floating > integral > boolean), we promote to a type with sufficient size to hold
|
|
all scalar operands of that category.
|
|
* If a zero-dimension tensor operand has a higher category than dimensioned operands,
|
|
we promote to a type with sufficient size and category to hold all zero-dim tensor operands of
|
|
that category.
|
|
* If there are no higher-category zero-dim operands, we promote to a type with sufficient size
|
|
and category to hold all dimensioned operands.
|
|
|
|
A floating point scalar operand has dtype `torch.get_default_dtype()` and an integral
|
|
non-boolean scalar operand has dtype `torch.int64`. Unlike numpy, we do not inspect
|
|
values when determining the minimum `dtypes` of an operand. Complex types
|
|
are not yet supported. Promotion for shell dtypes is not defined.
|
|
|
|
Promotion Examples::
|
|
|
|
>>> float_tensor = torch.ones(1, dtype=torch.float)
|
|
>>> double_tensor = torch.ones(1, dtype=torch.double)
|
|
>>> complex_float_tensor = torch.ones(1, dtype=torch.complex64)
|
|
>>> complex_double_tensor = torch.ones(1, dtype=torch.complex128)
|
|
>>> int_tensor = torch.ones(1, dtype=torch.int)
|
|
>>> long_tensor = torch.ones(1, dtype=torch.long)
|
|
>>> uint_tensor = torch.ones(1, dtype=torch.uint8)
|
|
>>> bool_tensor = torch.ones(1, dtype=torch.bool)
|
|
# zero-dim tensors
|
|
>>> long_zerodim = torch.tensor(1, dtype=torch.long)
|
|
>>> int_zerodim = torch.tensor(1, dtype=torch.int)
|
|
|
|
>>> torch.add(5, 5).dtype
|
|
torch.int64
|
|
# 5 is an int64, but does not have higher category than int_tensor so is not considered.
|
|
>>> (int_tensor + 5).dtype
|
|
torch.int32
|
|
>>> (int_tensor + long_zerodim).dtype
|
|
torch.int32
|
|
>>> (long_tensor + int_tensor).dtype
|
|
torch.int64
|
|
>>> (bool_tensor + long_tensor).dtype
|
|
torch.int64
|
|
>>> (bool_tensor + uint_tensor).dtype
|
|
torch.uint8
|
|
>>> (float_tensor + double_tensor).dtype
|
|
torch.float64
|
|
>>> (complex_float_tensor + complex_double_tensor).dtype
|
|
torch.complex128
|
|
>>> (bool_tensor + int_tensor).dtype
|
|
torch.int32
|
|
# Since long is a different kind than float, result dtype only needs to be large enough
|
|
# to hold the float.
|
|
>>> torch.add(long_tensor, float_tensor).dtype
|
|
torch.float32
|
|
|
|
When the output tensor of an arithmetic operation is specified, we allow casting to its `dtype` except that:
|
|
* An integral output tensor cannot accept a floating point tensor.
|
|
* A boolean output tensor cannot accept a non-boolean tensor.
|
|
* A non-complex output tensor cannot accept a complex tensor
|
|
|
|
Casting Examples::
|
|
|
|
# allowed:
|
|
>>> float_tensor *= float_tensor
|
|
>>> float_tensor *= int_tensor
|
|
>>> float_tensor *= uint_tensor
|
|
>>> float_tensor *= bool_tensor
|
|
>>> float_tensor *= double_tensor
|
|
>>> int_tensor *= long_tensor
|
|
>>> int_tensor *= uint_tensor
|
|
>>> uint_tensor *= int_tensor
|
|
|
|
# disallowed (RuntimeError: result type can't be cast to the desired output type):
|
|
>>> int_tensor *= float_tensor
|
|
>>> bool_tensor *= int_tensor
|
|
>>> bool_tensor *= uint_tensor
|
|
>>> float_tensor *= complex_float_tensor
|
|
|
|
|
|
.. _device-doc:
|
|
|
|
torch.device
|
|
------------
|
|
|
|
.. class:: device
|
|
|
|
A :class:`torch.device` is an object representing the device on which a :class:`torch.Tensor` is
|
|
or will be allocated.
|
|
|
|
The :class:`torch.device` contains a device type (most commonly "cpu" or
|
|
"cuda", but also potentially :doc:`"mps" <mps>`, :doc:`"xpu" <xpu>`,
|
|
`"xla" <https://github.com/pytorch/xla/>`_ or :doc:`"meta" <meta>`) and optional
|
|
device ordinal for the device type. If the device ordinal is not present, this object will always represent
|
|
the current device for the device type, even after :func:`torch.cuda.set_device()` is called; e.g.,
|
|
a :class:`torch.Tensor` constructed with device ``'cuda'`` is equivalent to ``'cuda:X'`` where X is
|
|
the result of :func:`torch.cuda.current_device()`.
|
|
|
|
A :class:`torch.Tensor`'s device can be accessed via the :attr:`Tensor.device` property.
|
|
|
|
A :class:`torch.device` can be constructed using:
|
|
|
|
* A device string, which is a string representation of the device type and optionally the device ordinal.
|
|
* A device type and a device ordinal.
|
|
* A device ordinal, where the current :ref:`accelerator<accelerators>` type will be used.
|
|
|
|
Via a device string:
|
|
::
|
|
|
|
>>> torch.device('cuda:0')
|
|
device(type='cuda', index=0)
|
|
|
|
>>> torch.device('cpu')
|
|
device(type='cpu')
|
|
|
|
>>> torch.device('mps')
|
|
device(type='mps')
|
|
|
|
>>> torch.device('cuda') # implicit index is the "current device index"
|
|
device(type='cuda')
|
|
|
|
Via a device type and a device ordinal:
|
|
|
|
::
|
|
|
|
>>> torch.device('cuda', 0)
|
|
device(type='cuda', index=0)
|
|
|
|
>>> torch.device('mps', 0)
|
|
device(type='mps', index=0)
|
|
|
|
>>> torch.device('cpu', 0)
|
|
device(type='cpu', index=0)
|
|
|
|
Via a device ordinal:
|
|
|
|
.. note::
|
|
This method will raise a RuntimeError if no accelerator is currently detected.
|
|
|
|
::
|
|
|
|
>>> torch.device(0) # the current accelerator is cuda
|
|
device(type='cuda', index=0)
|
|
|
|
>>> torch.device(1) # the current accelerator is xpu
|
|
device(type='xpu', index=1)
|
|
|
|
>>> torch.device(0) # no current accelerator detected
|
|
Traceback (most recent call last):
|
|
File "<stdin>", line 1, in <module>
|
|
RuntimeError: Cannot access accelerator device when none is available.
|
|
|
|
The device object can also be used as a context manager to change the default
|
|
device tensors are allocated on:
|
|
|
|
::
|
|
|
|
>>> with torch.device('cuda:1'):
|
|
... r = torch.randn(2, 3)
|
|
>>> r.device
|
|
device(type='cuda', index=1)
|
|
|
|
This context manager has no effect if a factory function is passed an explicit,
|
|
non-None device argument. To globally change the default device, see also
|
|
:func:`torch.set_default_device`.
|
|
|
|
.. warning::
|
|
|
|
This function imposes a slight performance cost on every Python
|
|
call to the torch API (not just factory functions). If this
|
|
is causing problems for you, please comment on
|
|
https://github.com/pytorch/pytorch/issues/92701
|
|
|
|
.. note::
|
|
The :class:`torch.device` argument in functions can generally be substituted with a string.
|
|
This allows for fast prototyping of code.
|
|
|
|
>>> # Example of a function that takes in a torch.device
|
|
>>> cuda1 = torch.device('cuda:1')
|
|
>>> torch.randn((2,3), device=cuda1)
|
|
|
|
>>> # You can substitute the torch.device with a string
|
|
>>> torch.randn((2,3), device='cuda:1')
|
|
|
|
.. note::
|
|
Methods which take a device will generally accept a (properly formatted) string
|
|
or an integer device ordinal, i.e. the following are all equivalent:
|
|
|
|
>>> torch.randn((2,3), device=torch.device('cuda:1'))
|
|
>>> torch.randn((2,3), device='cuda:1')
|
|
>>> torch.randn((2,3), device=1) # equivalent to 'cuda:1' if the current accelerator is cuda
|
|
|
|
.. note::
|
|
Tensors are never moved automatically between devices and require an explicit call from the user. Scalar Tensors (with tensor.dim()==0) are the only exception to this rule and they are automatically transferred from CPU to GPU when needed as this operation can be done "for free".
|
|
Example:
|
|
|
|
>>> # two scalars
|
|
>>> torch.ones(()) + torch.ones(()).cuda() # OK, scalar auto-transferred from CPU to GPU
|
|
>>> torch.ones(()).cuda() + torch.ones(()) # OK, scalar auto-transferred from CPU to GPU
|
|
|
|
>>> # one scalar (CPU), one vector (GPU)
|
|
>>> torch.ones(()) + torch.ones(1).cuda() # OK, scalar auto-transferred from CPU to GPU
|
|
>>> torch.ones(1).cuda() + torch.ones(()) # OK, scalar auto-transferred from CPU to GPU
|
|
|
|
>>> # one scalar (GPU), one vector (CPU)
|
|
>>> torch.ones(()).cuda() + torch.ones(1) # Fail, scalar not auto-transferred from GPU to CPU and non-scalar not auto-transferred from CPU to GPU
|
|
>>> torch.ones(1) + torch.ones(()).cuda() # Fail, scalar not auto-transferred from GPU to CPU and non-scalar not auto-transferred from CPU to GPU
|
|
|
|
.. _layout-doc:
|
|
|
|
torch.layout
|
|
------------
|
|
|
|
.. class:: layout
|
|
|
|
.. warning::
|
|
The ``torch.layout`` class is in beta and subject to change.
|
|
|
|
A :class:`torch.layout` is an object that represents the memory layout of a
|
|
:class:`torch.Tensor`. Currently, we support ``torch.strided`` (dense Tensors)
|
|
and have beta support for ``torch.sparse_coo`` (sparse COO Tensors).
|
|
|
|
``torch.strided`` represents dense Tensors and is the memory layout that
|
|
is most commonly used. Each strided tensor has an associated
|
|
:class:`torch.Storage`, which holds its data. These tensors provide
|
|
multi-dimensional, `strided <https://en.wikipedia.org/wiki/Stride_of_an_array>`_
|
|
view of a storage. Strides are a list of integers: the k-th stride
|
|
represents the jump in the memory necessary to go from one element to the
|
|
next one in the k-th dimension of the Tensor. This concept makes it possible
|
|
to perform many tensor operations efficiently.
|
|
|
|
Example::
|
|
|
|
>>> x = torch.tensor([[1, 2, 3, 4, 5], [6, 7, 8, 9, 10]])
|
|
>>> x.stride()
|
|
(5, 1)
|
|
|
|
>>> x.t().stride()
|
|
(1, 5)
|
|
|
|
For more information on ``torch.sparse_coo`` tensors, see :ref:`sparse-docs`.
|
|
|
|
torch.memory_format
|
|
-------------------
|
|
|
|
.. class:: memory_format
|
|
|
|
A :class:`torch.memory_format` is an object representing the memory format on which a :class:`torch.Tensor` is
|
|
or will be allocated.
|
|
|
|
Possible values are:
|
|
|
|
- ``torch.contiguous_format``:
|
|
Tensor is or will be allocated in dense non-overlapping memory. Strides represented by values in decreasing order.
|
|
|
|
- ``torch.channels_last``:
|
|
Tensor is or will be allocated in dense non-overlapping memory. Strides represented by values in
|
|
``strides[0] > strides[2] > strides[3] > strides[1] == 1`` aka NHWC order.
|
|
|
|
- ``torch.channels_last_3d``:
|
|
Tensor is or will be allocated in dense non-overlapping memory. Strides represented by values in
|
|
``strides[0] > strides[2] > strides[3] > strides[4] > strides[1] == 1`` aka NDHWC order.
|
|
|
|
- ``torch.preserve_format``:
|
|
Used in functions like `clone` to preserve the memory format of the input tensor. If input tensor is
|
|
allocated in dense non-overlapping memory, the output tensor strides will be copied from the input.
|
|
Otherwise output strides will follow ``torch.contiguous_format``
|