Changes by apply order:
1. Replace all `".."` and `os.pardir` usage with `os.path.dirname(...)`.
2. Replace nested `os.path.dirname(os.path.dirname(...))` call with `str(Path(...).parent.parent)`.
3. Reorder `.absolute()` ~/ `.resolve()`~ and `.parent`: always resolve the path first.
`.parent{...}.absolute()` -> `.absolute().parent{...}`
4. Replace chained `.parent x N` with `.parents[${N - 1}]`: the code is easier to read (see 5.)
`.parent.parent.parent.parent` -> `.parents[3]`
5. ~Replace `.parents[${N - 1}]` with `.parents[${N} - 1]`: the code is easier to read and does not introduce any runtime overhead.~
~`.parents[3]` -> `.parents[4 - 1]`~
6. ~Replace `.parents[2 - 1]` with `.parent.parent`: because the code is shorter and easier to read.~
Pull Request resolved: https://github.com/pytorch/pytorch/pull/129374
Approved by: https://github.com/justinchuby, https://github.com/malfet
Changes by apply order:
1. Replace all `".."` and `os.pardir` usage with `os.path.dirname(...)`.
2. Replace nested `os.path.dirname(os.path.dirname(...))` call with `str(Path(...).parent.parent)`.
3. Reorder `.absolute()` ~/ `.resolve()`~ and `.parent`: always resolve the path first.
`.parent{...}.absolute()` -> `.absolute().parent{...}`
4. Replace chained `.parent x N` with `.parents[${N - 1}]`: the code is easier to read (see 5.)
`.parent.parent.parent.parent` -> `.parents[3]`
5. ~Replace `.parents[${N - 1}]` with `.parents[${N} - 1]`: the code is easier to read and does not introduce any runtime overhead.~
~`.parents[3]` -> `.parents[4 - 1]`~
6. ~Replace `.parents[2 - 1]` with `.parent.parent`: because the code is shorter and easier to read.~
Pull Request resolved: https://github.com/pytorch/pytorch/pull/129374
Approved by: https://github.com/justinchuby, https://github.com/malfet
Test Plan: This is test code
Reviewed By: tugsbayasgalan
Differential Revision: D33858429
fbshipit-source-id: f5bc687ad81125f89cb281f9aa51dc5ac74d6954
(cherry picked from commit 202dff607e72d538c8b3fcd44382d11a401e52c5)
Summary:
1. Add unit test to ensure the key in `ALL_MODULES` is an instance
2. In `generate_models.py`, add a check to ensure the key is an instance of `torch.nn.Module`.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/72615
ghstack-source-id: 148771575
Test Plan:
```
buck test mode/opt //caffe2/test/jit/fixtures_srcs:test_upgrader_models_generation
```
Reviewed By: tugsbayasgalan
Differential Revision: D34116938
fbshipit-source-id: 36b32e00215bf26d471d984653dda5c866eaebc6
(cherry picked from commit 3a178bafc6e033ce6779b5b96fb7eafac70c9308)
Summary:
The model generation script will check the model version, to ensure the developer run the script before they change operator
Previously, the version use the old model version. However, it's hard for developer to know the old version number. In this change, it use the current max operator version to check. It's less strict, but more developer friendly
Pull Request resolved: https://github.com/pytorch/pytorch/pull/71894
ghstack-source-id: 147769215
Test Plan:
first time run:
```
chenlai@devvm5615:~/fbsource/fbcode(b82243650)$ buck run mode/opt //caffe2/torch/fb/mobile/upgrader_codegen:upgrader_test_models_gen
Parsing buck files: finished in 0.7 sec
Downloaded 0/2 artifacts, 0.00 bytes, 100.0% cache miss (for updated rules)
Building: finished in 21.6 sec (100%) 11547/11547 jobs, 2/11547 updated
Total time: 22.4 sec
BUILD SUCCEEDED
TestVersionedDivTensorExampleV7() aten::div.Tensor
INFO:test.jit.fixtures_srcs.generate_models:Processing TestVersionedDivTensorExampleV7
INFO:test.jit.fixtures_srcs.generate_models:Generating model test_versioned_div_tensor_example_v7 and it's save to /data/users/chenlai/fbsource/fbcode/caffe2/test/jit/fixtures/test_versioned_div_tensor_example_v7.ptl
chenlai@devvm5615:~/fbsource/fbcode(b82243650)$
```
second time run:
```
chenlai@devvm5615:~/fbsource/fbcode(b82243650)$ rm caffe2/test/jit/fixtures/test_versioned_div_tensor_example_v4.ptl
chenlai@devvm5615:~/fbsource/fbcode(b82243650)$ buck run mode/opt //caffe2/torch/fb/mobile/upgrader_codegen:upgrader_test_models_gen
Action graph will be rebuilt because files have been added or removed.
Parsing buck files: finished in 2.0 sec
Building... 17.4 sec (99%) 9289/9290 jobs, 0/9290 updated
TestVersionedDivTensorExampleV7() aten::div.Tensor
INFO:test.jit.fixtures_srcs.generate_models:Processing TestVersionedDivTensorExampleV7
INFO:test.jit.fixtures_srcs.generate_models:Model test_versioned_div_tensor_example_v7 already exists, skipping
chenlai@devvm5615:~/fbsource/fbcode(b82243650)$ jf s
```
Reviewed By: tugsbayasgalan
Differential Revision: D33804737
fbshipit-source-id: 7424b81a700703bdf896ec606c2dac8df6dbf8a6
(cherry picked from commit 44b4e37d30077a3160b8a92209af339a6f2fc885)
Summary:
This change is to automate the process to generate the old models for testing upgraders. Developer will
1. Add a module in `caffe2/test/jit/fixtures_srcs/fixtures_src.py`
2. Register the module in `caffe2/test/jit/fixtures_srcs/generate_models.py`
3. Run `python test/jit/fixtures_src/generate_models.py`
The model will be dumped to `test/jit/fixtures`.
This script also ensure:
1. The output model operator version is as expected
2. The output model will include the changed operator
Example log:
```
(base) chenlai@chenlai-mp pytorch % python3 /Users/chenlai/pytorch/test/jit/fixtures_src/generate_models.py
TestVersionedDivTensorExampleV4() aten::div.Tensor
INFO:__main__:Processing TestVersionedDivTensorExampleV4
INFO:__main__:Generating model test_versioned_div_tensor_example_v4 and it's save to /Users/chenlai/pytorch/test/jit/fixtures/test_versioned_div_tensor_example_v4.ptl
```
The second time to run
```
(base) chenlai@chenlai-mp pytorch % python3 /Users/chenlai/pytorch/test/jit/fixtures_src/generate_models.py
TestVersionedDivTensorExampleV4() aten::div.Tensor
INFO:__main__:Processing TestVersionedDivTensorExampleV4
INFO:__main__:Model test_versioned_div_tensor_example_v4 already exists, skipping
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/70629
ghstack-source-id: 147585826
Test Plan:
OSS
```
python3 /Users/chenlai/pytorch/test/jit/fixtures_src/generate_models.py
```
Internal:
```
buck run mode/opt //caffe2/torch/fb/mobile/upgrader_codegen:upgrader_test_models_gen
```
Reviewed By: iseeyuan, tugsbayasgalan
Differential Revision: D33410841
fbshipit-source-id: 28e2b851a30f12a74e4ac8a539d76e83bbc4fb3a
(cherry picked from commit 6614f1bdf360b69bcf9eb4bca30707e5bd0e8a2b)
