Summary:
After inserting quant dequant nodes in the graph, we need
1. Insert packed param creation and quantized op
2. Create packed_params attribute in the top module. For this we need
graph that inlined except for calculate_qparams method calls. But they
can be inlined too. So perhaps we need to make sure no other callmethods
exist.
3. Insert SetAttr for the packed param
4. Insert GetAttr for the packed param
5. Use GetAttr output for quantized op where applicable, e.g.
linear_dynamic
The above is added to quantize_<method-name> method created inprevious
step. Once the above steps are done clone the method into
quantized_<method-name>
Modify quantize_<method-name>:
1. Remove all outputs from the method.
2. Run dce
3. Remove all inputs from the method except self.
Modify quantized_<method-name>:
1. Remove all packed_param setAttr nodes.
2. Run dce.
This should result in removal of all nodes that generate packed param.
Test Plan: To be written
Reviewers:
Subscribers:
Tasks:
Tags:
Differential Revision: [D38771416](https://our.internmc.facebook.com/intern/diff/D38771416)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/83571
Approved by: https://github.com/jerryzh168
Summary:
Replace for loop with for `irange` loop. Also fix some unused variable warnings in range loop cases
Pull Request resolved: https://github.com/pytorch/pytorch/pull/62928
Reviewed By: driazati
Differential Revision: D30171904
Pulled By: malfet
fbshipit-source-id: 1b437a0f7e3515f4a2e324f3450e93312f1933ae
Summary:
This is an automatic change generated by the following script:
```
#!/usr/bin/env python3
from subprocess import check_output, check_call
import os
def get_compiled_files_list():
import json
with open("build/compile_commands.json") as f:
data = json.load(f)
files = [os.path.relpath(node['file']) for node in data]
for idx, fname in enumerate(files):
if fname.startswith('build/') and fname.endswith('.DEFAULT.cpp'):
files[idx] = fname[len('build/'):-len('.DEFAULT.cpp')]
return files
def run_clang_tidy(fname):
check_call(["python3", "tools/clang_tidy.py", "-c", "build", "-x", fname,"-s"])
changes = check_output(["git", "ls-files", "-m"])
if len(changes) == 0:
return
check_call(["git", "commit","--all", "-m", f"NOLINT stubs for {fname}"])
def main():
git_files = check_output(["git", "ls-files"]).decode("ascii").split("\n")
compiled_files = get_compiled_files_list()
for idx, fname in enumerate(git_files):
if fname not in compiled_files:
continue
if fname.startswith("caffe2/contrib/aten/"):
continue
print(f"[{idx}/{len(git_files)}] Processing {fname}")
run_clang_tidy(fname)
if __name__ == "__main__":
main()
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/56892
Reviewed By: H-Huang
Differential Revision: D27991944
Pulled By: malfet
fbshipit-source-id: 5415e1eb2c1b34319a4f03024bfaa087007d7179
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/42222
This change adds the necessary passes to perform FP16 dynamic quantization.
We skip inserting observers for activations based on the dtype (torch.float16) and only insert the Fp16Observer for weights
Test Plan:
python test/test_quantization.py TestQuantizeJitOps
Imported from OSS
Reviewed By: jerryzh168
Differential Revision: D22849220
fbshipit-source-id: 2c53594ecd2485e9e3dd0b380eceaf7c5ab5fc50
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/40596
Previously the fusion patterns for {add/mul}_scalar is inconsistent since the op pattern
produces a non-quantized tensor and the op replacement graph produces a quantized tensor
Test Plan: Imported from OSS
Differential Revision: D22251072
fbshipit-source-id: e16eb92cf6611578cca1ed8ebde961f8d0610137
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/40100
ELU has a range of [-1, inf]. In the original PR which added
the quantized operator we decided to pass the quantization params
from the input. However, it makes more sense to require observation
for this op.
This PR changes the API to require observation. Next PRs in this stack
will add the eager and graph mode handling.
Test Plan:
```
python test/test_quantization.py TestQuantizedOps.test_qelu
```
Imported from OSS
Differential Revision: D22075083
fbshipit-source-id: 0ea0fd05a00cc7a5f122a2b1de09144bbd586f32
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/39413
Implementing the request from
https://github.com/pytorch/pytorch/pull/39095
WIP so we can align on the API, once it looks good
will amend the PR to apply to all relevant functions.
Test Plan:
```
python test/test_quantization.py TestQuantizeScriptPTSQOps.test_hardswish
```
Imported from OSS
Differential Revision: D21885263
fbshipit-source-id: 029339a99f8c50e45dd1dfb7fd89c20e3188720d
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/39867
Support list of filters in subgraph rewriter, the rewrite will execute only
when the match passes all filter check. this is useful for different matches
to share the same filter.
Test Plan: Imported from OSS
Differential Revision: D22009855
fbshipit-source-id: 67aab8d6326b2011a9061397699dc62ee9ad4e2d
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/39219
We didn't model clamp ops correctly right now, this PR fixes that.
Reason is quantized clamp op quantizes the scalar arguments in the op implementation: https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/quantized/cpu/kernels/QuantizedOpKernels.cpp#L614-L617
So we'll need to model this explicitly in the IR.
When we see a `aten::dequantize - aten::clamp(%x, %min, %max)`
we first make a scalar tensor with `aten::scalar_tensor(%scalar, ...)`, then we quantize the tensor with the same quantization parameters from the input tensor of the `aten::clamp`, dequantize the tensor, then convert the dequantized tensor to scalar using `aten::item`.
Test Plan: Imported from OSS
Differential Revision: D21831350
fbshipit-source-id: d60731459a0465d64946aabc62065d25d92faefc
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/39367
We shouldn't match `%alpha` argument since it could be used by multiple functions
Test Plan: Imported from OSS
Differential Revision: D21829295
fbshipit-source-id: 6daa320a4b56df4e142b8e02e04a3ecb36284d1b
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/39041
reduce_range option restricts the activation tensor to 7 bits instead of 8.
This is necessary to enable per channel quant for RNNs and LSTMs
Test Plan:
python test/test_quantization.py TestDynamicQuantizedLinear
Imported from OSS
Reviewed By: akinh
Differential Revision: D21769691
fbshipit-source-id: ef0e9873367f3c1b34091b0b3af788233ef60c6c
