Howard Huang 108a75b454 [PP] Add ZeroBubble schedule (#133467) ...

Zero bubble can be expressed through `ScheduleFlexibleInterleaved1F1B` by setting `enable_zero_bubble=True`. But instead of having to include this flag in schedule initialization we should create a separate ZeroBubbleSchedule and also transition `Interleaved1F1B` to derive from `ScheduleFlexibleInterleaved1F1B`. Then we dont need to expose `ScheduleFlexibleInterleaved1F1B` since the naming is not obvious

Pull Request resolved: https://github.com/pytorch/pytorch/pull/133467
Approved by: https://github.com/wconstab
ghstack dependencies: #132691

2024-08-22 13:32:15 +00:00

..

_static

Clean up distributed/CONTRIBUTING.md (#128450)

2024-06-22 02:41:22 +00:00

_templates

Remove sdp_kernel and replace with sdpa_kernel in attention namespace (#114689)

2024-01-24 22:28:04 +00:00

community

Add Alban and Piotr into Core Maintainers (#130903)

2024-07-20 16:02:42 +00:00

elastic

DOC: add docstring to construct_and_record_rdzv_event() (#128189)

2024-06-10 22:17:33 +00:00

notes

[Doc] fix some typos (found by codespell and typos) (#132544)

2024-08-05 17:21:56 +00:00

rpc

[Doc] fix some typos (found by codespell and typos) (#132544)

2024-08-05 17:21:56 +00:00

scripts

[Doc] fix some typos (found by codespell and typos) (#132544)

2024-08-05 17:21:56 +00:00

amp.rst

add xpu for amp (#127276)

2024-06-20 21:49:35 +00:00

autograd.rst

Add torch.library.register_autograd (#124071)

2024-04-18 12:47:59 +00:00

backends.rst

[sparse] Add cuSPARSELt as a backend (#128534)

2024-08-21 22:06:07 +00:00

benchmark_utils.rst

Adding Compare in torch.utils.benchmark documentation (#125009)

2024-05-03 00:50:54 +00:00

bottleneck.rst

add itt unit test and docstrings (#84848)

2022-09-28 01:39:58 +00:00

checkpoint.rst

[checkpoint] Clean up selective activation checkpoint and make public (#125795)

2024-06-18 18:18:50 +00:00

complex_numbers.rst

Document complex optimizer semantic behavior (#121667)

2024-03-16 00:43:47 +00:00

cond.rst

[Doc] fix some typos (found by codespell and typos) (#132544)

2024-08-05 17:21:56 +00:00

conf.py

Revert "[dtensor] move DTensor to public namespace (#133113)"

2024-08-19 05:00:19 +00:00

config_mod.rst

rename config module file to work with gh pages better

2022-03-10 20:41:44 +00:00

cpp_extension.rst

Check clang++/g++ version when compiling CUDA extensions (#63230)

2022-02-24 08:32:32 +00:00

cpp_index.rst

Add C++ Landing Page (#38450)

2020-05-14 16:02:01 -07:00

cpu.rst

Add current_device() to torch.cpu (#110987)

2023-10-11 05:13:10 +00:00

cuda_environment_variables.rst

Add doc page for environment variables that effect PyTorch Runtime (#119087)

2024-02-15 21:41:38 +00:00

cuda._sanitizer.rst

Fix typos under docs directory (#88033)

2022-10-31 19:31:56 +00:00

cuda.rst

[Reland] Add wrappers for synchronous GPUDirect Storage APIs (#133489)

2024-08-15 17:11:52 +00:00

cuda.tunable.rst

[ROCm] TunableOp improvements (#124362)

2024-06-03 22:30:11 +00:00

cudnn_persistent_rnn.rst

Remove orphan from cuDNN persistent note (#65160)

2021-09-21 11:09:47 -07:00

cudnn_rnn_determinism.rst

Forbid trailing whitespace (#53406)

2021-03-05 17:22:55 -08:00

data.rst

Revert "reseed all Generators in Dataloader's _worker_loop() -- via GC (#107131)"

2023-08-23 17:08:07 +00:00

ddp_comm_hooks.rst

[DOCS][DDP]Fix the simple of saving and reloading PowerSGD state and hook. (#102721)

2023-06-10 00:15:00 +00:00

debugging_environment_variables.rst

Add doc page for environment variables that effect PyTorch Runtime (#119087)

2024-02-15 21:41:38 +00:00

deploy.rst

Delete torch::deploy from pytorch core (#85953)

2022-10-06 07:20:16 +00:00

deterministic.rst

Add torch.utils.deterministic.fill_uninitialized_memory flag (#111377)

2023-11-01 16:10:09 +00:00

distributed.algorithms.join.rst

Add tutorial link (#62785)

2021-08-05 17:28:02 -07:00

distributed.checkpoint.rst

[Doc] fix some typos (found by codespell and typos) (#132544)

2024-08-05 17:21:56 +00:00

distributed.elastic.rst

Reapply "distributed debug handlers (#126601)" (#127805)

2024-06-04 19:44:30 +00:00

distributed.optim.rst

[distributed][docs] Delete distributed optimimzer section from RPC and add reference to namespace docs page (#68068)

2021-11-09 15:01:54 -08:00

distributed.pipelining.rst

[PP] Add ZeroBubble schedule (#133467)

2024-08-22 13:32:15 +00:00

distributed.rst

Revert "[dtensor] move DTensor to public namespace (#133113)"

2024-08-19 05:00:19 +00:00

distributed.tensor.parallel.rst

[tp] doc fixes (#121431)

2024-03-08 17:46:44 +00:00

distributions.rst

Add inverse gamma distribution and fix sign bug in PowerTransform. (#104501)

2023-11-01 02:26:25 +00:00

dlpack.rst

Lint trailing newlines (#54737)

2021-03-30 13:09:52 -07:00

docutils.conf

Revert "Revert D21337640: [pytorch][PR] Split up documentation into subpages and clean up some warnings" (#37778)

2020-05-04 14:32:35 -07:00

export.ir_spec.rst

[export] Remove torch._export.export (#119095)

2024-02-08 21:22:04 +00:00

export.rst

[Doc] fix some typos (found by codespell and typos) (#132544)

2024-08-05 17:21:56 +00:00

fft.rst

Cleanup all module references in doc (#73983)

2022-03-10 22:26:29 +00:00

fsdp.rst

[FSDP][state_dict] Expose optimizer state_dict config (#105949)

2023-08-21 07:29:49 +00:00

func.api.rst

[functorch] linearize (#94173)

2023-02-09 15:45:08 +00:00

func.batch_norm.rst

Fix typo under docs directory (#97202)

2023-03-21 01:24:10 +00:00

func.migrating.rst

[torch.func] Add migration guide from functorch (#91811)

2023-01-17 22:14:42 +00:00

func.rst

Fix typo under docs directory (#92762)

2023-01-23 18:07:22 +00:00

func.ux_limitations.rst

[torch.func] Add docs (#91319)

2022-12-30 02:51:18 +00:00

func.whirlwind_tour.rst

[torch.func] Add docs (#91319)

2022-12-30 02:51:18 +00:00

future_mod.rst

Add swap_tensors path to nn.Module._apply (#117167)

2024-02-07 18:55:44 +00:00

futures.rst

Update docs to mention CUDA support for Future (#50048)

2021-05-11 08:26:33 -07:00

fx.experimental.rst

Only thunkify proxies in some situations (#132421)

2024-08-08 12:03:06 +00:00

fx.rst

Consolidate SymDispatchMode into ProxyTensorMode (#132674)

2024-08-08 12:02:54 +00:00

hub.rst

Fix typo under docs directory (#92762)

2023-01-23 18:07:22 +00:00

index.rst

Revert "[dtensor] move DTensor to public namespace (#133113)"

2024-08-19 05:00:19 +00:00

jit_builtin_functions.rst

Lint trailing newlines (#54737)

2021-03-30 13:09:52 -07:00

jit_language_reference_v2.rst

[Doc] fix some typos (found by codespell and typos) (#132544)

2024-08-05 17:21:56 +00:00

jit_language_reference.rst

[Doc] fix some typos (found by codespell and typos) (#132544)

2024-08-05 17:21:56 +00:00

jit_python_reference.rst

[JIT] improve documentation (#57991)

2021-05-19 11:47:32 -07:00

jit_unsupported.rst

Add support for torch.Generator type in TorchScript (#110413)

2023-11-21 23:07:21 +00:00

jit_utils.rst

Create __init__.py (#78629)

2022-06-03 18:14:21 +00:00

jit.rst

Doc test non packages (#110568)

2023-10-06 14:16:01 +00:00

library.rst

[custom ops] Add register_vmap for custom ops (#130589)

2024-07-23 17:48:38 +00:00

linalg.rst

Add a note on the stability of linalg functions. (#88313)

2022-11-07 22:44:23 +00:00

logging.rst

Change classification to beta for TORCH_LOGS (#118682)

2024-01-31 21:50:55 +00:00

masked.rst

Revert "Add MaskedTensor support to *_like API (#128637)"

2024-08-20 08:26:28 +00:00

math-quantizer-equation.png

…

meta.rst

Add documentation for meta device (#119119)

2024-02-04 01:05:22 +00:00

miscellaneous_environment_variables.rst

[RFC] Add support for device extension autoloading (#127074)

2024-07-09 06:14:13 +00:00

mobile_optimizer.rst

[Reland] Clean Up MobileOptimizerType Rewrite Flags Public API and Documentation (#92081)

2023-01-14 17:06:00 +00:00

model_zoo.rst

…

module_tracker.rst

Add module tracker (#125352)

2024-05-04 18:33:35 +00:00

monitor.rst

torch/monitor: merge Interval and FixedCount stats (#72009)

2022-01-30 23:21:59 +00:00

mps_environment_variables.rst

[MPS] Add mps profiler env vars to docs (#129552)

2024-07-04 06:44:48 +00:00

mps.rst

Add support in Python API for the recommended max working set size. (#128289)

2024-06-12 16:03:57 +00:00

mtia.rst

[Land Internally] MTIA equivalent of torch.cuda.memory_stats (#132007)

2024-07-29 20:47:18 +00:00

multiprocessing.rst

Doc test non packages (#110568)

2023-10-06 14:16:01 +00:00

name_inference.rst

[docs] Properly link register_post_accumulate_grad_hook docs (#108157)

2023-08-29 22:13:33 +00:00

named_tensor.rst

fixing named tensor unflatten example (#106921)

2023-08-22 18:00:10 +00:00

nested.rst

Replace master with main in links and docs/conf.py (#100176)

2023-05-02 18:20:32 +00:00

nn.attention.bias.rst

Remove sdp_kernel and replace with sdpa_kernel in attention namespace (#114689)

2024-01-24 22:28:04 +00:00

nn.attention.flex_attention.rst

[Inductor] Added and_masks and or_masks utilities & make fully masked out rows 0 instead of nan (#131552)

2024-07-25 21:29:46 +00:00

nn.attention.rst

Make FlexAttention API public (#130755)

2024-07-16 16:21:25 +00:00

nn.functional.rst

Add RMSNorm module (#121364)

2024-03-29 18:05:28 +00:00

nn.init.rst

update nn.init doc to reflect the no_grad (#80882)

2022-07-07 17:19:29 +00:00

nn.rst

Make adding Buffers more like adding Parameters (#125971)

2024-07-31 10:32:40 +00:00

onnx_dynamo_onnxruntime_backend.rst

Follow-up #108379 (#108905)

2023-09-09 01:38:36 +00:00

onnx_dynamo.rst

[ez][doc] Fix sample code in onnx_dynamo.rst (#114770)

2023-11-29 19:27:52 +00:00

onnx_torchscript_supported_aten_ops.rst

Refactor torch.onnx documentation (#108379)

2023-09-08 18:23:48 +00:00

onnx_torchscript.rst

[ONNX] New export logic leveraging ExportedProgram and ONNX IR (#132530)

2024-08-21 01:08:42 +00:00

onnx.rst

fix pytorch version for onnx in doc (#124182)

2024-04-17 18:05:15 +00:00

optim.rst

Make optim.swa.util content accessible from the torch.optim doc (#133393)

2024-08-21 00:43:46 +00:00

package.rst

Doc test non packages (#110568)

2023-10-06 14:16:01 +00:00

profiler.rst

Doc test non packages (#110568)

2023-10-06 14:16:01 +00:00

quantization-accuracy-debugging.rst

Fix typo under docs directory (#87583)

2022-10-24 23:52:44 +00:00

quantization-backend-configuration.rst

update quantization doc: add x86 backend as default backend of server inference (#86794)

2022-12-02 02:10:25 +00:00

quantization-support.rst

Add numeric_debugger top level APIs (#130643)

2024-07-18 20:54:18 +00:00

quantization.rst

Cleanup some duplicated placeholder py:module docs (#123244)

2024-04-05 03:18:53 +00:00

random.rst

Remove duplicated entries in random.rst (#39725)

2020-06-10 16:51:15 -07:00

rpc.rst

[BE] RPC is missing RRef docs (#106902)

2023-08-10 16:26:27 +00:00

signal.rst

Nuttall window (#90103)

2022-12-16 09:05:53 +00:00

size.rst

Added a docstring for torch.Size.numel. (#124186)

2024-04-19 09:23:02 +00:00

sparse.rst

SparseCsrCUDA: cuDSS backend for linalg.solve (#129856)

2024-08-22 07:57:30 +00:00

special.rst

[primTorch] special: j0, j1, spherical_j0 (#86049)

2022-10-04 18:21:46 +00:00

storage.rst

Deprecate TypedStorage, its derived classes, and all of their public methods (#85303)

2022-11-08 18:11:01 +00:00

tensor_attributes.rst

Refine the logic of device construction when only device index is given (#129119)

2024-07-15 14:34:29 +00:00

tensor_view.rst

Correcting a minor typo: "Users should pay" instead of "Users should be pay" (#72500)

2022-02-08 23:08:25 +00:00

tensorboard.rst

Cleanup all module references in doc (#73983)

2022-03-10 22:26:29 +00:00

tensors.rst

add xpu to torch.tensors (#127280)

2024-06-11 18:13:01 +00:00

testing.rst

document torch.testing.assert_allclose (#89526)

2022-12-01 11:22:50 +00:00

threading_environment_variables.rst

Add doc page for environment variables that effect PyTorch Runtime (#119087)

2024-02-15 21:41:38 +00:00

torch_cuda_memory.rst

Fix typo under docs directory (#110359)

2023-10-03 16:36:05 +00:00

torch_environment_variables.rst

[Docs][MPS] Add mps environment variable table (#129008)

2024-06-20 03:30:35 +00:00

torch_nccl_environment_variables.rst

[c10d][doc] Add docs for ENV variables TORCH_NCCL_ASYNC_ERROR_HANDLING TORCH_NCCL_TRACE_CPP_STACK and TORCH_NCCL_COORD_CHECK_MILSEC (#132920)

2024-08-09 21:08:20 +00:00

torch.ao.ns._numeric_suite_fx.rst

Quantization docs: add pages for Numeric Suite (Eager and FX) (#66380)

2021-10-11 18:47:58 -07:00

torch.ao.ns._numeric_suite.rst

Quantization docs: add pages for Numeric Suite (Eager and FX) (#66380)

2021-10-11 18:47:58 -07:00

torch.compiler_aot_inductor.rst

[AOTI] docs: add suggestion to turn on freezing on CPU (#128010)

2024-06-07 08:57:02 +00:00

torch.compiler_api.rst

[RFC][dynamo] add decorator to register polyfill for unsupported C++ function to avoid graph break (#133712)

2024-08-21 06:36:41 +00:00

torch.compiler_best_practices_for_backends.rst

Restructure torch.compile docs (#105376)

2023-07-28 20:58:57 +00:00

torch.compiler_cudagraph_trees.rst

[CUDAGraph] add more docs for cudagraph trees (#127963)

2024-06-18 02:07:07 +00:00

torch.compiler_custom_backends.rst

Fix a link in the compiler backend doc (#126079)

2024-05-21 20:16:04 +00:00

torch.compiler_dynamic_shapes.rst

feat: Add min, max ranges to mark_dynamic API (#119737)

2024-03-07 23:26:03 +00:00

torch.compiler_dynamo_deepdive.rst

Stop immediately specializing common constants 0/1 for plain int (#128327)

2024-07-03 16:41:51 +00:00

torch.compiler_dynamo_overview.rst

Rename TorchDynamo -> Dyanamo in the dynamo tutorial doc (#123431)

2024-05-07 05:07:00 +00:00

torch.compiler_fake_tensor.rst

[BE] Reroute all uses of proxy_tensor.maybe_disable_fake_tensor_mode to fake_tensor.unset_fake_temporarily (#132770)

2024-08-08 23:07:23 +00:00

torch.compiler_faq.rst

Fixed broken link and removed unfinished sentence from issue #126367 (#127938)

2024-06-05 07:37:32 +00:00

torch.compiler_fine_grain_apis.rst

[Doc] fix some typos (found by codespell and typos) (#132544)

2024-08-05 17:21:56 +00:00

torch.compiler_get_started.rst

add xpu to torch.compile (#127279)

2024-06-13 21:15:09 +00:00

torch.compiler_inductor_profiling.rst

Restructure torch.compile docs (#105376)

2023-07-28 20:58:57 +00:00

torch.compiler_ir.rst

[export] torch.export landing page (#108783)

2023-09-10 01:40:42 +00:00

torch.compiler_nn_module.rst

Revert "Reland 3rd try [finishing colesbury's PR 100642] Guard on nn.Module dicts and type (#109323)" + Forward fixes + test (#110964)

2023-10-11 05:16:47 +00:00

torch.compiler_performance_dashboard.rst

Restructure torch.compile docs (#105376)

2023-07-28 20:58:57 +00:00

torch.compiler_profiling_torch_compile.rst

[docs] Update PT2+Profiler docs (#122272)

2024-03-28 17:52:28 +00:00

torch.compiler_transformations.rst

Fix typo under docs directory (#110359)

2023-10-03 16:36:05 +00:00

torch.compiler_troubleshooting.rst

Add force_disable_caches to the docs (#126184)

2024-05-15 07:16:08 +00:00

torch.compiler.rst

add xpu to torch.compile (#127279)

2024-06-13 21:15:09 +00:00

torch.overrides.rst

Doc test non packages (#110568)

2023-10-06 14:16:01 +00:00

torch.rst

Autoselect default device in FSDP construction. (#127609)

2024-08-08 05:25:17 +00:00

type_info.rst

ENH: Convert finfo.tiny to finfo.smallest_normal (#76292)

2022-05-20 00:59:48 +00:00

utils.rst

New swap function (#111747)

2023-12-08 18:49:35 +00:00

xpu.rst

[2/2] Intel GPU Runtime Upstreaming for Generator (#118613)

2024-02-28 05:28:11 +00:00