diff --git a/docs/Makefile b/docs/Makefile index 9b11128a1575..1337a1fc5dc0 100644 --- a/docs/Makefile +++ b/docs/Makefile @@ -17,9 +17,6 @@ figures: @$(PYCMD) source/scripts/build_activation_images.py @$(PYCMD) source/scripts/build_lr_scheduler_images.py -onnx: - @$(PYCMD) source/scripts/onnx/build_onnx_torchscript_supported_aten_op_csv_table.py - opset: @$(PYCMD) source/scripts/build_opsets.py diff --git a/docs/source/_static/img/onnx/torch_script_exporter_memory_usage.png b/docs/source/_static/img/onnx/torch_script_exporter_memory_usage.png deleted file mode 100644 index b9c81a71ef3c..000000000000 Binary files a/docs/source/_static/img/onnx/torch_script_exporter_memory_usage.png and /dev/null differ diff --git a/docs/source/onnx.md b/docs/source/onnx.md index 184dc8740c79..22a8f416077b 100644 --- a/docs/source/onnx.md +++ b/docs/source/onnx.md @@ -12,8 +12,6 @@ The exported model can be consumed by any of the many [runtimes that support ONNX](https://onnx.ai/supported-tools.html#deployModel), including Microsoft's [ONNX Runtime](https://www.onnxruntime.ai). -**There are two flavors of ONNX exporter API that you can use, as listed below.** -Both can be called through function {func}`torch.onnx.export`. Next example shows how to export a simple model. ```python @@ -40,39 +38,29 @@ torch.onnx.export( ) ``` -Next sections introduce the two versions of the exporter. +## torch.export-based ONNX Exporter -## TorchDynamo-based ONNX Exporter +*The torch.export-based ONNX exporter is the newest exporter for PyTorch 2.6 and newer* -*The TorchDynamo-based ONNX exporter is the newest (and Beta) exporter for PyTorch 2.1 and newer* +{ref}`torch.export ` engine is leveraged to produce a traced graph representing only the Tensor computation of the function in an +Ahead-of-Time (AOT) fashion. The resulting traced graph (1) produces normalized operators in the functional +ATen operator set (as well as any user-specified custom operators), (2) has eliminated all Python control +flow and data structures (with certain exceptions), and (3) records the set of shape constraints needed to +show that this normalization and control-flow elimination is sound for future inputs, before it is finally +translated into an ONNX graph. -TorchDynamo engine is leveraged to hook into Python's frame evaluation API and dynamically rewrite its -bytecode into an FX Graph. The resulting FX Graph is then polished before it is finally translated into an -ONNX graph. +{doc}`Learn more about the torch.export-based ONNX Exporter ` -The main advantage of this approach is that the [FX graph](https://pytorch.org/docs/stable/fx.html) is captured using -bytecode analysis that preserves the dynamic nature of the model instead of using traditional static tracing techniques. +## Frequently Asked Questions -{doc}`Learn more about the TorchDynamo-based ONNX Exporter ` +Q: I have exported my LLM model, but its input size seems to be fixed? -## TorchScript-based ONNX Exporter + The tracer records the shapes of the example inputs. If the model should accept + inputs of dynamic shapes, set ``dynamic_shapes`` when calling :func:`torch.onnx.export`. -*The TorchScript-based ONNX exporter is available since PyTorch 1.2.0* +Q: How to export models containing loops? -[TorchScript](https://pytorch.org/docs/stable/jit.html) is leveraged to trace (through {func}`torch.jit.trace`) -the model and capture a static computation graph. - -As a consequence, the resulting graph has a couple limitations: - -* It does not record any control-flow, like if-statements or loops; -* Does not handle nuances between `training` and `eval` mode; -* Does not truly handle dynamic inputs - -As an attempt to support the static tracing limitations, the exporter also supports TorchScript scripting -(through {func}`torch.jit.script`), which adds support for data-dependent control-flow, for example. However, TorchScript -itself is a subset of the Python language, so not all features in Python are supported, such as in-place operations. - -{doc}`Learn more about the TorchScript-based ONNX Exporter ` + See {ref}`torch.cond `. ## Contributing / Developing @@ -80,14 +68,58 @@ The ONNX exporter is a community project and we welcome contributions. We follow [PyTorch guidelines for contributions](https://github.com/pytorch/pytorch/blob/main/CONTRIBUTING.md), but you might also be interested in reading our [development wiki](https://github.com/pytorch/pytorch/wiki/PyTorch-ONNX-exporter). + +### torch.onnx API + +```{eval-rst} +.. automodule:: torch.onnx +``` + +### Functions + +```{eval-rst} +.. autofunction:: export + :noindex: +.. autofunction:: is_in_onnx_export + :noindex: +.. autofunction:: enable_fake_mode + :noindex: +``` + +### Classes + +```{eval-rst} +.. autoclass:: ONNXProgram + :noindex: +.. autoclass:: OnnxExporterError + :noindex: +``` + +### Deprecated API + +```{eval-rst} +.. deprecated:: 2.6 + These functions are deprecated and will be removed in a future version. + +.. autofunction:: register_custom_op_symbolic +.. autofunction:: unregister_custom_op_symbolic +.. autofunction:: select_model_mode_for_export + +.. autosummary:: + :toctree: generated + :nosignatures: + :template: classtemplate.rst + + JitScalarType +``` + ```{eval-rst} .. toctree:: :hidden: - onnx_dynamo + onnx_export onnx_ops onnx_verification - onnx_torchscript ```