frozenleaves/pytorch
1
0
Fork 0
mirror of https://github.com/pytorch/pytorch.git synced 2025-10-21 05:34:18 +08:00
Code Issues Packages Projects Releases Wiki Activity

doc: fix fake_quantize_per_tensor_affine docs (#104453)

Browse Source 
Fixes #82800

Fixes wrong `fake_quantize_per_tensor_affine` example and wrong `fake_quantize_per_tensor_affine` formula

Pull Request resolved: https://github.com/pytorch/pytorch/pull/104453
Approved by: https://github.com/kit1980
This commit is contained in:
Oren Leung
2023-06-30 22:58:55 +00:00
committed by PyTorch MergeBot
parent a5ca445f79
commit 3ff111a4b4
1 changed files with 10 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
torch/_torch_docs.py
View File

@ -11962,13 +11962,15 @@ Returns a new tensor with the data in :attr:`input` fake quantized using :attr:`
:attr:`zero_point`, :attr:`quant_min` and :attr:`quant_max`.
.. math::
\text{output} = min(
\text{quant\_max},
max(
\text{quant\_min},
\text{std::nearby\_int}(\text{input} / \text{scale}) + \text{zero\_point}
)
)
\text{output} = (
min(
\text{quant\_max},
max(
\text{quant\_min},
\text{std::nearby\_int}(\text{input} / \text{scale}) + \text{zero\_point}
)
) - \text{zero\_point}
) \times \text{scale}
Args:
input (Tensor): the input value(s), ``torch.float32`` tensor
@ -11988,7 +11990,7 @@ Example::
>>> torch.fake_quantize_per_tensor_affine(x, 0.1, 0, 0, 255)
tensor([0.1000, 1.0000, 0.4000, 0.0000])
>>> torch.fake_quantize_per_tensor_affine(x, torch.tensor(0.1), torch.tensor(0), 0, 255)
tensor([0.6000, 0.4000, 0.0000, 0.0000])
tensor([0.1000, 1.0000, 0.4000, 0.0000])
""",
)