eqy 790763b0fe Add an option to disable reduced precision reductions for FP16 GEMM (#67946) ...

Summary:
https://github.com/pytorch/pytorch/issues/67578 disabled reduced precision reductions for FP16 GEMMs. After benchmarking, we've found that this has substantial performance impacts for common GEMM shapes (e.g., those found in popular instantiations of multiheaded-attention) on architectures such as Volta. As these performance regressions may come as a surprise to current users, this PR adds a toggle to disable reduced precision reductions
`torch.backends.cuda.matmul.allow_fp16_reduced_precision_reduction = `
rather than making it the default behavior.

CC ngimel ptrblck
stas00 Note that the behavior after the previous PR can be replicated with
`torch.backends.cuda.matmul.allow_fp16_reduced_precision_reduction = False`

Pull Request resolved: https://github.com/pytorch/pytorch/pull/67946

Reviewed By: zou3519

Differential Revision: D32289896

Pulled By: ngimel

fbshipit-source-id: a1ea2918b77e27a7d9b391e030417802a0174abe

2021-11-09 17:27:20 -08:00

..

_static

clarify the documentation of torch.meshgrid (#62977)

2021-08-18 04:01:22 -07:00

_templates

Remove master documentation from being indexable by search engines (#58056)

2021-05-18 06:20:09 -07:00

community

Update contribution_guide.rst (#64142)

2021-08-30 19:26:59 -07:00

elastic

(torchelastic) make --max_restarts explicit in the quickstart and runner docs (#65838)

2021-09-29 19:29:01 -07:00

notes

Add an option to disable reduced precision reductions for FP16 GEMM (#67946)

2021-11-09 17:27:20 -08:00

rpc

Support Union in TorchScript (#64234)

2021-09-03 06:12:24 -07:00

scripts

[docs] Add images to some activation functions (#65415)

2021-09-22 11:05:29 -07:00

__config__.rst

Fix __config__ docs (#48557)

2020-11-29 23:57:06 -08:00

amp.rst

rebase for autocast updates to include device_type and dtype flags (#61002)

2021-08-10 20:03:12 -07:00

autograd.rst

Update extending doc to cover forward mode AD (#66962)

2021-10-27 14:18:38 -07:00

backends.rst

Add an option to disable reduced precision reductions for FP16 GEMM (#67946)

2021-11-09 17:27:20 -08:00

benchmark_utils.rst

Expand benchmark utils docs (#51664)

2021-02-04 00:22:41 -08:00

bottleneck.rst

[docs] Clarify more CUDA profiling gotchas in bottleneck docs (#6763)

2018-04-19 13:15:27 -04:00

checkpoint.rst

Stashing checkpointing RNG states based on devices of arg tensors (#14518)

2018-12-11 09:48:45 -08:00

complex_numbers.rst

Grammatical update of tech docs (#61547)

2021-07-14 14:01:59 -07:00

conf.py

[Quant] Add dynamic QAT Linear module (#67325)

2021-11-08 10:24:25 -08:00

cpp_extension.rst

correct some cpp extension code usages and documents (#39766)

2020-06-10 08:31:22 -07:00

cpp_index.rst

Add C++ Landing Page (#38450)

2020-05-14 16:02:01 -07:00

cuda.rst

[CUDA graphs] Beta, not prototype (#65247)

2021-09-20 13:32:36 -07:00

cudnn_persistent_rnn.rst

Remove orphan from cuDNN persistent note (#65160)

2021-09-21 11:09:47 -07:00

cudnn_rnn_determinism.rst

Forbid trailing whitespace (#53406)

2021-03-05 17:22:55 -08:00

data.rst

Add a warning about DataLoader num_workers > 0 "memory leak" (#64337)

2021-09-01 21:49:41 -07:00

ddp_comm_hooks.rst

[DDP Comm Hook] Add debugging communication hooks to ddp_comm_hooks.rst (#64352)

2021-09-01 17:37:19 -07:00

distributed.algorithms.join.rst

Add tutorial link (#62785)

2021-08-05 17:28:02 -07:00

distributed.elastic.rst

[1/n][torch/elastic] Move torchelastic docs *.rst (#148)

2021-05-04 00:57:56 -07:00

distributed.optim.rst

[distributed][docs] Delete distributed optimimzer section from RPC and add reference to namespace docs page (#68068)

2021-11-09 15:01:54 -08:00

distributed.rst

Update distributed.rst to show that CUDA send/recv on GPU is supported (#65601)

2021-09-24 12:30:10 -07:00

distributions.rst

Add sample validation for LKJCholesky.log_prob (#52763)

2021-02-25 16:12:29 -08:00

dlpack.rst

Lint trailing newlines (#54737)

2021-03-30 13:09:52 -07:00

docutils.conf

Revert "Revert D21337640: [pytorch][PR] Split up documentation into subpages and clean up some warnings" (#37778)

2020-05-04 14:32:35 -07:00

fft.rst

C++ API and docs for hfftn (#66127)

2021-10-07 12:48:36 -07:00

futures.rst

Update docs to mention CUDA support for Future (#50048)

2021-05-11 08:26:33 -07:00

fx.rst

fx: Update fx.rst (#68043)

2021-11-09 14:00:45 -08:00

hub.rst

Add a torch.hub.load_local() function that can load models from any local directory with a hubconf.py (#44204)

2020-09-21 14:17:21 -07:00

index.rst

Make _Join, _Joinable, _JoinHook public (#62605)

2021-08-03 12:20:11 -07:00

jit_builtin_functions.rst

Lint trailing newlines (#54737)

2021-03-30 13:09:52 -07:00

jit_language_reference_v2.rst

Document torch.jit.is_tracing() (#67326)

2021-10-28 09:56:09 -07:00

jit_language_reference.rst

Document torch.jit.is_tracing() (#67326)

2021-10-28 09:56:09 -07:00

jit_python_reference.rst

[JIT] improve documentation (#57991)

2021-05-19 11:47:32 -07:00

jit_unsupported.rst

[JIT] Update docs for recently added features (#45232)

2020-09-28 18:17:42 -07:00

jit.rst

Back out "D30740897 Add fusion enabled apis" (#64500)

2021-09-04 20:55:58 -07:00

linalg.rst

Create linalg.matrix_exp (#62715)

2021-10-19 09:07:15 -07:00

math-quantizer-equation.png

adding quantization.rst file for quantization feature (#27559)

2019-10-09 16:45:09 -07:00

mobile_optimizer.rst

Mod lists to neutral+descriptive terms in caffe2/docs (#49803)

2020-12-23 11:37:11 -08:00

model_zoo.rst

add/move a few apis in torch.hub (#18758)

2019-04-10 23:10:39 -07:00

multiprocessing.rst

Forbid trailing whitespace (#53406)

2021-03-05 17:22:55 -08:00

name_inference.rst

Abladawood patch 1 (#58496)

2021-05-20 10:32:18 -07:00

named_tensor.rst

Forbid trailing whitespace (#53406)

2021-03-05 17:22:55 -08:00

nn.functional.rst

Add mish activation function (#58648)

2021-05-25 10:36:21 -07:00

nn.init.rst

Bag of documentation fixes; fix more sphinx warnings (#27850)

2019-10-15 07:31:14 -07:00

nn.rst

Implements the orthogonal parametrization (#62089)

2021-08-30 13:12:07 -07:00

onnx.rst

[Doc] [ONNX]Fix a broken url for ONNXRuntime custom op (#67944)

2021-11-08 15:51:02 -08:00

optim.rst

To add SequentialLR to PyTorch Core Schedulers (#64037)

2021-09-09 09:36:32 -07:00

package.rst

[package] add some docs describing how to debug dependencies (#65704)

2021-09-27 12:14:23 -07:00

pipeline.rst

fixed comments referring fairscale master branch (#65531)

2021-09-23 14:37:58 -07:00

profiler.rst

docs: fix profiler docstring (#55750)

2021-04-13 00:23:14 -07:00

quantization-support.rst

[Quant] Add dynamic QAT Linear module (#67325)

2021-11-08 10:24:25 -08:00

quantization.rst

pytorch quantization: document the custom module APIs (#67449)

2021-10-29 05:22:17 -07:00

random.rst

Remove duplicated entries in random.rst (#39725)

2020-06-10 16:51:15 -07:00

rpc.rst

[distributed][docs] Delete distributed optimimzer section from RPC and add reference to namespace docs page (#68068)

2021-11-09 15:01:54 -08:00

sparse.rst

Add CSR (compressed sparse row) layout for sparse tensors (#50937)

2021-04-12 10:09:12 -07:00

special.rst

[special] special alias for softmax (#62251)

2021-10-01 03:55:32 -07:00

storage.rst

Lint trailing newlines (#54737)

2021-03-30 13:09:52 -07:00

tensor_attributes.rst

Remove legacy constructor calls from pytorch codebase. (#54142)

2021-04-11 15:45:17 -07:00

tensor_view.rst

Add tensor.{adjoint(),H,mT,mH} methods and properties (#64179)

2021-10-13 07:44:43 -07:00

tensorboard.rst

Add method add_hparams to API doc (#27344)

2019-10-03 17:07:45 -07:00

tensors.rst

[numpy] add torch.argwhere (#64257)

2021-10-30 15:26:11 -07:00

testing.rst

[Doc] make_tensor to torch.testing module (#63925)

2021-08-30 12:25:40 -07:00

torch.ao.ns._numeric_suite_fx.rst

Quantization docs: add pages for Numeric Suite (Eager and FX) (#66380)

2021-10-11 18:47:58 -07:00

torch.ao.ns._numeric_suite.rst

Quantization docs: add pages for Numeric Suite (Eager and FX) (#66380)

2021-10-11 18:47:58 -07:00

torch.overrides.rst

Add documentation for torch.overrides submodule. (#48170)

2020-11-30 11:25:31 -08:00

torch.rst

[numpy] add torch.argwhere (#64257)

2021-10-30 15:26:11 -07:00

type_info.rst

clarify that torch.finfo.tiny is the smallest normal number (#63241)

2021-08-18 13:44:52 -07:00