Preferring dash over underscore in command-line options. Add `--command-arg-name` to the argument parser. The old arguments with underscores `--command_arg_name` are kept for backward compatibility.
Both dashes and underscores are used in the PyTorch codebase. Some argument parsers only have dashes or only have underscores in arguments. For example, the `torchrun` utility for distributed training only accepts underscore arguments (e.g., `--master_port`). The dashes are more common in other command-line tools. And it looks to be the default choice in the Python standard library:
`argparse.BooleanOptionalAction`: 4a9dff0e5a/Lib/argparse.py (L893-L895)
```python
class BooleanOptionalAction(Action):
def __init__(...):
if option_string.startswith('--'):
option_string = '--no-' + option_string[2:]
_option_strings.append(option_string)
```
It adds `--no-argname`, not `--no_argname`. Also typing `_` need to press the shift or the caps-lock key than `-`.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/94505
Approved by: https://github.com/ezyang, https://github.com/seemethere
Changes:
1. `typing_extensions -> typing-extentions` in dependency. Use dash rather than underline to fit the [PEP 503: Normalized Names](https://peps.python.org/pep-0503/#normalized-names) convention.
```python
import re
def normalize(name):
return re.sub(r"[-_.]+", "-", name).lower()
```
2. Import `Literal`, `Protocal`, and `Final` from standard library as of Python 3.8+
3. Replace `Union[Literal[XXX], Literal[YYY]]` to `Literal[XXX, YYY]`.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/94490
Approved by: https://github.com/ezyang, https://github.com/albanD
To fix#78540 I committed #78983 which is reverted due to internal CI failure. Then I comitted #79215 which was only fixing the failure but didn't have the full feature of #78983. This PR is another try.
This PR adds script to dump all operators from test models and automatically write into `lightweight_dispatch_ops.yaml`. This way we don't have to manually update the yaml file.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/80791
Approved by: https://github.com/raziel
With ufmt in place https://github.com/pytorch/pytorch/pull/81157, we can now use it to gradually format all files. I'm breaking this down into multiple smaller batches to avoid too many merge conflicts later on.
This batch (as copied from the current BLACK linter config):
* `tools/**/*.py`
Upcoming batchs:
* `torchgen/**/*.py`
* `torch/package/**/*.py`
* `torch/onnx/**/*.py`
* `torch/_refs/**/*.py`
* `torch/_prims/**/*.py`
* `torch/_meta_registrations.py`
* `torch/_decomp/**/*.py`
* `test/onnx/**/*.py`
Once they are all formatted, BLACK linter will be removed.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/81285
Approved by: https://github.com/suo
This PR introduces selective build to lightweight dispatch CI job. By doing so we can't run the `test_lite_intepreter_runtime` test suite anymore because it requires some other operators.
From now on, if we are adding a new unit test in `test_codegen_unboxing`, we will have to export the operators for the unit test model and add them into `lightweight_dispatch_ops.yaml`. This can be automated by introducing tracing based selective build, but that's for next PR to do.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/78983
Approved by: https://github.com/kit1980
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/74663
In lightweight dispatch, we only need to register root ops. Unlike in the dispatcher world, the transitive closure of the operators doesn't need to go through dispatcher or op registry.
Test Plan: Rely on unit tests
Reviewed By: priyaramani
Differential Revision: D35104401
fbshipit-source-id: 1a2df571880ac3c8625985c01bd89a2bb9566af9
(cherry picked from commit 16207fa18e87908ec5e038a7f60f41893a236749)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/74271
Add operator selection ability to gen_unboxing:
- Adds ability to pass in selected_operators.yaml to gen_aten_unboxing_files target
- Creates a selective builder using the selected_operators.yaml in gen_unboxing.py and uses the selective builder to select native functions for codegen
Test Plan: Used selected_operators.yaml used in oacr_service target and ensured the codegen only generated code for included ops
Reviewed By: larryliu0820
Differential Revision: D34809772
fbshipit-source-id: c8d58e9693730b3d5b55f77ea057ee14a5ee1337
(cherry picked from commit 96f8494c21239ba4da7e61df38ddcf47c0f4db9f)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/74069
RFC: pytorch/rfcs#40
In #69881 we added the ability to generate codegen unboxing source files. Notice that the generated code to register an operator looks like this:
```
// aten::add.Tensor(Tensor self, Tensor other, *, Scalar alpha=1) -> Tensor
OperatorGenerator(
TORCH_SELECTIVE_SCHEMA("aten::add.Tensor(Tensor self, Tensor other, *, Scalar alpha=1) -> Tensor"),
[](Stack & stack) {
RECORD_FUNCTION("add", std::vector<c10::IValue>());
at::unboxing::add_Tensor(stack);
},
aliasAnalysisFromSchema()
),
```
However, this means we have to parse the schema and get back arguments with default values in static init time. As written in the RFC, there's a more performant option: providing these arguments with default values using codegen, then we don't have to do expensive regex pattern matching in parsing. Here's how it looks like:
```
// aten::add.Tensor(Tensor self, Tensor other, *, Scalar alpha=1) -> Tensor
OperatorGenerator(
"aten::add",
"Tensor",
{
c10::Argument("self", nullptr, c10::nullopt, c10::IValue(c10::nullopt)),
c10::Argument("other", nullptr, c10::nullopt, c10::IValue(c10::nullopt)),
c10::Argument("alpha", nullptr, c10::nullopt, c10::IValue(1))
},
{
c10::Argument("")
},
[](Stack & stack) {
RECORD_FUNCTION("add", std::vector<c10::IValue>());
at::unboxing::add_Tensor(stack);
},
aliasAnalysisFromSchema()
),
```
We also added corresponding APIs in `operator.h` to take in the arguments.
Test Plan: Rely on CI
Reviewed By: kimishpatel
Differential Revision: D33077733
fbshipit-source-id: e7f13a2f162c70d4e506b4f64cdbb7afec39f4e6
(cherry picked from commit 08a076935f480d5ab11c75e78d8627a8e2ba7d1a)
Summary:
RFC: https://github.com/pytorch/rfcs/pull/40
This PR (re)introduces python codegen for unboxing wrappers. Given an entry of `native_functions.yaml` the codegen should be able to generate the corresponding C++ code to convert ivalues from the stack to their proper types. To trigger the codegen, run
```
tools/jit/gen_unboxing.py -d cg/torch/share/ATen
```
Merged changes on CI test. In https://github.com/pytorch/pytorch/issues/71782 I added an e2e test for static dispatch + codegen unboxing. The test exports a mobile model of mobilenetv2, load and run it on a new binary for lite interpreter: `test/mobile/custom_build/lite_predictor.cpp`.
## Lite predictor build specifics
1. Codegen: `gen.py` generates `RegisterCPU.cpp` and `RegisterSchema.cpp`. Now with this PR, once `static_dispatch` mode is enabled, `gen.py` will not generate `TORCH_LIBRARY` API calls in those cpp files, hence avoids interaction with the dispatcher. Once `USE_LIGHTWEIGHT_DISPATCH` is turned on, `cmake/Codegen.cmake` calls `gen_unboxing.py` which generates `UnboxingFunctions.h`, `UnboxingFunctions_[0-4].cpp` and `RegisterCodegenUnboxedKernels_[0-4].cpp`.
2. Build: `USE_LIGHTWEIGHT_DISPATCH` adds generated sources into `all_cpu_cpp` in `aten/src/ATen/CMakeLists.txt`. All other files remain unchanged. In reality all the `Operators_[0-4].cpp` are not necessary but we can rely on linker to strip them off.
## Current CI job test coverage update
Created a new CI job `linux-xenial-py3-clang5-mobile-lightweight-dispatch-build` that enables the following build options:
* `USE_LIGHTWEIGHT_DISPATCH=1`
* `BUILD_LITE_INTERPRETER=1`
* `STATIC_DISPATCH_BACKEND=CPU`
This job triggers `test/mobile/lightweight_dispatch/build.sh` and builds `libtorch`. Then the script runs C++ tests written in `test_lightweight_dispatch.cpp` and `test_codegen_unboxing.cpp`. Recent commits added tests to cover as many C++ argument type as possible: in `build.sh` we installed PyTorch Python API so that we can export test models in `tests_setup.py`. Then we run C++ test binary to run these models on lightweight dispatch enabled runtime.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/69881
Reviewed By: iseeyuan
Differential Revision: D33692299
Pulled By: larryliu0820
fbshipit-source-id: 211e59f2364100703359b4a3d2ab48ca5155a023
(cherry picked from commit 58e1c9a25e3d1b5b656282cf3ac2f548d98d530b)
