Summary:
As this diff shows, currently there are a couple hundred instances of raw `noqa` in the codebase, which just ignore all errors on a given line. That isn't great, so this PR changes all existing instances of that antipattern to qualify the `noqa` with respect to a specific error code, and adds a lint to prevent more of this from happening in the future.
Interestingly, some of the examples the `noqa` lint catches are genuine attempts to qualify the `noqa` with a specific error code, such as these two:
```
test/jit/test_misc.py:27: print(f"{hello + ' ' + test}, I'm a {test}") # noqa E999
test/jit/test_misc.py:28: print(f"format blank") # noqa F541
```
However, those are still wrong because they are [missing a colon](https://flake8.pycqa.org/en/3.9.1/user/violations.html#in-line-ignoring-errors), which actually causes the error code to be completely ignored:
- If you change them to anything else, the warnings will still be suppressed.
- If you add the necessary colons then it is revealed that `E261` was also being suppressed, unintentionally:
```
test/jit/test_misc.py:27:57: E261 at least two spaces before inline comment
test/jit/test_misc.py:28:35: E261 at least two spaces before inline comment
```
I did try using [flake8-noqa](https://pypi.org/project/flake8-noqa/) instead of a custom `git grep` lint, but it didn't seem to work. This PR is definitely missing some of the functionality that flake8-noqa is supposed to provide, though, so if someone can figure out how to use it, we should do that instead.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/56272
Test Plan:
CI should pass on the tip of this PR, and we know that the lint works because the following CI run (before this PR was finished) failed:
- https://github.com/pytorch/pytorch/runs/2365189927
Reviewed By: janeyx99
Differential Revision: D27830127
Pulled By: samestep
fbshipit-source-id: d6dcf4f945ebd18cd76c46a07f3b408296864fcb
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/56294
When matching a pattern to `BinaryOpQuantizeHandler`, we need to make
sure we check for dtype support on the base node, instead of the current
node. This is important in cases such as `add-relu` and `mul-relu`,
when the current node is `relu`, but the base node is `add|mul`.
Test Plan:
```
python test/test_quantization.py TestQuantizeFx
```
There is no good test case to check this in current logic. Created an
add-relu model manually, and verified with pdb that the add node was
being used to match against dtypes.
Imported from OSS
Reviewed By: jerryzh168
Differential Revision: D27831070
fbshipit-source-id: 3697f1328dff9fec3eb910bae49a73793ef36d63
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/54813
Previously we have a cat that takes a list of Tensors with different qparams and dequantize them
cacatenate them and requantize with the output qparams. This adds some unnecessary overhead in dequantizing
and quantizing Tensors.
This PR adds an optimization for cat operator, we'll make sure inputs and output of cat
uses same observer/fake_quant and produce a cat that does not do rescaling.
Test Plan: Imported from OSS
Reviewed By: vkuzo
Differential Revision: D27408377
fbshipit-source-id: 6a4bdcfd15e57ea1fe0f7e72d1e1288eb3ece4db
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/56205
Allows for int8 modules to shadow int8 modules. This is useful when
comparing quantized models with different qconfigs.
Test Plan:
```
python test/test_quantization.py TestFXNumericSuiteCoreAPIs.test_int8_shadows_int8
```
Imported from OSS
Reviewed By: jerryzh168
Differential Revision: D27807405
fbshipit-source-id: 10c3bc7ab9bb1e6808aa1af23a34c7cf380465fd
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/56195
This is outdated, removing (forgot to clean up in a previous PR).
Test Plan:
```
python test/test_quantization.py TestFXGraphMatcher
```
Imported from OSS
Reviewed By: jerryzh168
Differential Revision: D27805334
fbshipit-source-id: 3b035945b4928a3c727e96e0f7fe0efe201f42c0
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/56194
Enables the NS graph matcher to also match `call_method` nodes.
These are useful for ops such as `torch.sigmoid`.
Test Plan:
```
python test/test_quantization.py TestFXGraphMatcher.test_methods
```
Imported from OSS
Reviewed By: jerryzh168
Differential Revision: D27805333
fbshipit-source-id: 509ae283db6b245671f11e3eb6b7fcb3a5735ef5
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/55918
Adds coverage for determining I/O dtype for various ops. This will
enable shadowing of these ops.
Test Plan:
```
python test/test_quantization.py TestFXNumericSuiteCoreAPIs.test_op_io_dtype_coverage
```
Imported from OSS
Reviewed By: jerryzh168
Differential Revision: D27740661
fbshipit-source-id: c5ce873ec56bffa50ca46d2fe134c70ed677e37e
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/55859
Adds mappings for ops which can accept either fp32 or int8 input,
such as `F.relu`. A future PR will fill out the op coverage.
Test Plan:
```
python test/test_quantization.py TestFXNumericSuiteCoreAPIs.test_op_with_either_fp32_or_int8_input
```
Imported from OSS
Reviewed By: raghuramank100
Differential Revision: D27740659
fbshipit-source-id: cfc3dd58319b7161ca7f1fe05cd22d9a3ff11141
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/55858
Moves the mappings of input and output dtypes of various ops
into its own file, and makes the variable names more clear. No logic
change.
Test Plan:
```
python test/test_quantization.py TestFXNumericSuiteCoreAPIs
```
Imported from OSS
Reviewed By: raghuramank100
Differential Revision: D27740662
fbshipit-source-id: d384e7e542d9cc868d9cee9c53c2ac2f74a15a48
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/55837
Adds a test that checks that all of the relevant op pairs defined in
`quantization_mappings.py` are also defined as related by Numerical
Suite.
Note: this does not cover all the ops, just the ones in
`quantization_mappings.py`. A future PR will fill out the remainder.
Test Plan:
```
python test/test_quantization.py TestFXGraphMatcher.test_op_relationship_mapping
```
Imported from OSS
Reviewed By: jerryzh168
Differential Revision: D27719979
fbshipit-source-id: 9e852ef94da5f7a653ea15ba52c68a89c8e30208
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/55803
Makes the NS `graph_matcher.get_reversed_fusions` use the fusions
defined the FX quantization code instead of duplicating them.
Test Plan:
```
python test/test_quantization.py TestFXNumericSuiteCoreAPIs
python test/test_quantization.py TestFXNumericSuiteCoreAPIsModels
```
Imported from OSS
Reviewed By: jerryzh168
Differential Revision: D27719980
fbshipit-source-id: 12e3183405181bb9001f10e765cfb4d2ffdfdd88
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/55506
Makes the NS weight extraction tests also test QAT, and fixes
the mappings where necessary to cover all the fusions and make
the tests pass.
Test Plan:
```
python test/test_quantization.py TestFXNumericSuiteCoreAPIs.test_extract_weights_mod_ptq
python test/test_quantization.py TestFXNumericSuiteCoreAPIs.test_extract_weights_mod_qat
```
Imported from OSS
Reviewed By: jerryzh168
Differential Revision: D27650409
fbshipit-source-id: c5bd9268d1bc559afc27d4c5109effd77bf1538a
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/55505
This necessary to add support in NS for QAT modules, to avoid
duplicating logic between NSTracer and QuantizationTracer.
The eng work to expose the custom module and class names to
the user will be in a future PR.
Test Plan:
```
python test/test_quantization.py TestFXNumericSuiteCoreAPIs
python test/test_quantization.py TestFXNumericSuiteCoreAPIsModels
```
Imported from OSS
Reviewed By: jerryzh168
Differential Revision: D27650407
fbshipit-source-id: 431f47c5353b41c11371c5efa79657bfd085459a
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/55434
Before this PR, there was some hacky logic which determined
the input and output types of nodes based on heuristics such
as inspecting `__module__`, or assuming that an op has an
I/O dtype of `torch.float` when the heuristics did not find
any matches. This is problematic because the heuristics were not exact,
and this could result in non-sensical shadow graphs when the heuristics
would return an incorrect dtype.
This PR switches the dtype determination to an allowlist system,
where we specify exactly what the dtypes are for the nodes or modules
which are in an allowlist, and we add an `UNKNOWN` type for everything
else. The shadow logic is changed to skip inserting shadows on any
function or module where the I/O dtype is unknown.
The current allowlist only contains functions necessary for the
currently existing tests. Filling out the allowlist with all necessary
torch functions is left for a future PR.
As a result of this, we can do the following (also implemented in this PR):
1. enable graph matching on nodes with equal types (for example,
F.linear and F.linear). The restriction that only nodes with equal types
was in the code as a placeholder, it's better to allow comparisons of
nodes of equal types. One case where this is useful is unshadowed
activations.
2. enable models with user defined modules to be passed to Numeric Suite
APIs without errors.
Test Plan:
```
python test/test_quantization.py TestFXGraphMatcher
python test/test_quantization.py TestFXGraphMatcherModels
python test/test_quantization.py TestFXNumericSuiteCoreAPIs
python test/test_quantization.py TestFXNumericSuiteCoreAPIsModels
```
Imported from OSS
Reviewed By: jerryzh168
Differential Revision: D27622418
fbshipit-source-id: 40dcba0222c01154c141467640c1eb89725f33a7
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/55431
Fixes a bug in the test cases, returning early resulted
in some tests not being run. Adds logic for `nni.LinearReLU`,
which was unmasked by making the tests run
Test Plan:
```
python test/test_quantization.py TestFXNumericSuiteCoreAPIs.test_extract_weights_mod
```
Imported from OSS
Reviewed By: hx89
Differential Revision: D27622415
fbshipit-source-id: 79d9e3125e5d881d9d13645abbe4bd007a5e1d44
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/55311
Before this PR, `F.conv1d` was matched by FX graph mode quant patterns
but the prepacking was happening inline. There was also a bug with
argument type mismatch.
This PR fixes both issues and adds a test. Thanks jerryzh168 for the
code tip.
Test Plan:
```
python test/test_quantization.py TestQuantizeFx.test_functional_not_reference
```
Imported from OSS
Reviewed By: jerryzh168
Differential Revision: D27575422
fbshipit-source-id: 42301e23cb101a9e64e46800813bc771317e233e
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/55287
Adds support for extracting weights from F.conv2d and F.conv3d.
F.conv1d and the fused variants are saved for future PRs.
Test Plan:
```
python test/test_quantization.py TestFXNumericSuiteCoreAPIs.test_extract_weights_conv_fun
```
Imported from OSS
Reviewed By: hx89
Differential Revision: D27575424
fbshipit-source-id: e945912d7d0ab320f47cab30d00d60ddb7497158
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/55154
Adds functionality to NS to allow matching nodes which have the
same signature across dtypes. For now, only the skeleton is added,
we can fill out the rest of the ops later. This is to unblock
the work to change `cat` to have the same signature for fp32 and int8,
and keep the testing we have for `cat` in NS.
For context, the main reason we are not matching nodes with equal types,
for now, is user defined types for which we do not know the signature.
For now, the design is strictly allowlist of everything. In the future,
we may adjust the design to safely match user defined types.
Test Plan:
```
python test/test_quantization.py TestFXNumericSuiteCoreAPIs.test_ops_with_same_fp32_and_int8_signature
python test/test_quantization.py TestFXGraphMatcher.test_nodes_with_equal_types_do_not_get_matched
```
Imported from OSS
Reviewed By: raghuramank100
Differential Revision: D27504624
fbshipit-source-id: 4f8eb4f3258caf6f99aa373ca7ba516ebbcf4779
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/55727
number of dequantize for fp16 reference pattern was incorrect before, this
PR fixes the problem
Test Plan: Imported from OSS
Reviewed By: vkuzo
Differential Revision: D27713390
fbshipit-source-id: 72b8d4cda0bdcea74abe27a76f918d1b47819b01
Summary:
Generally wildcard imports are bad for the reasons described here: https://www.flake8rules.com/rules/F403.html
This PR replaces wildcard imports with an explicit list of imported items where possible, and adds a `# noqa: F403` comment in the other cases (mostly re-exports in `__init__.py` files).
This is a prerequisite for https://github.com/pytorch/pytorch/issues/55816, because currently [`tools/codegen/dest/register_dispatch_key.py` simply fails if you sort its imports](https://github.com/pytorch/pytorch/actions/runs/742505908).
Pull Request resolved: https://github.com/pytorch/pytorch/pull/55838
Test Plan: CI. You can also run `flake8` locally.
Reviewed By: jbschlosser
Differential Revision: D27724232
Pulled By: samestep
fbshipit-source-id: 269fb09cb4168f8a51fd65bfaacc6cda7fb87c34
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/55710
In the current code, there is an edge case which leads to an error
after the prepare step:
1. have a pattern like this:
```
user_func_unmatched_to_qhandler -> node_matched_to_copy_node_qhandler
```
2. the user function returns a type which is not observable (i.e. not a
Tensor)
3. if this is run through `prepare_fx`, calibrating it with data leads
to a runtime error, because observers cannot observe non-tensor types.
This PR fixes the issue. If a node matched to `CopyNodeQuantizeHandler`
is after an unmatched node, we delete the observer.
Test Plan:
```
python test/test_quantization.py TestQuantizeFx.test_no_obs_between_unmatched_node_and_copy_node
```
Imported from OSS
Reviewed By: jerryzh168
Differential Revision: D27686811
fbshipit-source-id: 320be41b1f383c6352ff89fb39a9f480822a3bb2
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/55529
x.shape outputs a non-Tensor, add this to the all_node_args_have_no_tensors function
to avoid inserting observer for the getattr "shape" node.
Test Plan: Imported from OSS
Reviewed By: wat3rBro
Differential Revision: D27628145
fbshipit-source-id: 4729294ab80c0a1e72440396d31e7e82257b1092
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/55429
Previously we special case copy operator in normal insert observer code, this PR tries to split the
special case logic to a separate function and keep the rest of the code clean.
Test Plan:
Imported from OSS
Imported from OSS
Reviewed By: vkuzo
Differential Revision: D27609972
fbshipit-source-id: 378f6aa70f18c0b477b62b6efe236648748aae7e
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/55388
temporarily revert D27314678 (c57541ce06), it appears to cause a perf regression that makes quantization of some models take too long to complete tests.
Reviewed By: houseroad
Differential Revision: D27583809
fbshipit-source-id: e9c088ccbfd3bfb3a1d4c7eafee3eca29ee7717b
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/55078
Fixes a TODO, make sure we iterate through kwargs as well as args
when navigating graphs. We can use `node.all_input_nodes` convenience
property to accomplish this.
Test Plan:
```
python test/test_quantization.py TestFXGraphMatcher
```
Imported from OSS
Reviewed By: jerryzh168
Differential Revision: D27474699
fbshipit-source-id: 8a6e3db5a73328c4f296ac5fce951e81213b6f58
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/55077
Deletes debugging prints from the code, no logic change.
Test Plan:
CI
Imported from OSS
Reviewed By: jerryzh168
Differential Revision: D27474700
fbshipit-source-id: 3d9d73da6615ddffdfdb0df270bcdfd2c4b50be3
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/55060
Removes the previous iteration of Numeric Suite for FX graph mode
quantization, and moves the current iteration into the top level
file.
Test Plan:
```
python test/test_quantization.py TestFXNumericSuiteCoreAPIs
python test/test_quantization.py TestFXGraphMatcher
```
Imported from OSS
Reviewed By: jerryzh168
Differential Revision: D27467725
fbshipit-source-id: 4c22b5a3221857231f9f59cf6d2908820e6a7f12
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/54335
Simple fix to enable weight extraction for nni.ConvReLU2d.
Note: this module only appears if the internal GraphModule APIs are
called, so we add testing for this path.
Test Plan:
```
python test/test_quantization.py TestFXNumericSuiteCoreAPIs.test_extract_weights_mod
```
Imported from OSS
Reviewed By: hx89
Differential Revision: D27192844
fbshipit-source-id: 923cf63e29e4638fd77ca42e69aedb15fb20a330
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/54326
Fixes unshadowed activation input logging for subgraphs where start_node does
not equal end_node. In detail:
* instead of passing around a single list of nodes, pass around a list
of nodes to instrument inputs, and a list of nodes to instrument
outputs. This way we can handle multi-node subgraphs properly, and we
also keep the subgraph instance definition out of the public APIs.
* add a test case
Test Plan:
```
python test/test_quantization.py TestFXNumericSuiteCoreAPIs.test_linear_fp16_activations
```
Imported from OSS
Reviewed By: hx89
Differential Revision: D27190138
fbshipit-source-id: 58e2377c1c128baaf3b760c1ad29098fb21f53d3
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/54275
Adds support for NS shadow activations path for the fp16 emulation
pattern such as
```
... -> dequantize -> linear -> relu -> to(torch.float16) -> ...
```
There are a couple of changes necessary here:
1. removing the restriction on the shadowing graph pass that the B
subgraph is a single node (since this subgraph is four nodes), and
modifying the code to correctly add the relevant inputs versus output
loggers (input loggers and subgraph copy if we are at start_node,
and output logger if we are at end_node)
2. modifying the logic for calculating node input and output type
to work correcty for the `to` and `dequantize` nodes:
2a. make the function return the first input and output, instead of just
the first input
2b. make the function handle `dequantize` correctly by recursively
using the output if its input
2c. make the function handle `to` correctyl by recursively using the
output of its input and the target dtype
3. a bug fix to handle observers in kwargs, while copying subgraphs
Note: input logging for these patterns is not tested yet,
this will be in the next PR.
Test Plan:
```
python test/test_quantization.py TestFXNumericSuiteCoreAPIs.test_linear_fp16
```
Imported from OSS
Reviewed By: jerryzh168
Differential Revision: D27172655
fbshipit-source-id: 3bdc86618b2a5782627fcf303d58af7f47fbc30d
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/54644
Previously we special case copy operator in normal insert observer code, this PR tries to split the
special case logic to a separate function and keep the rest of the code clean.
Test Plan: Imported from OSS
Reviewed By: vkuzo
Differential Revision: D27314678
fbshipit-source-id: d36870ceb3717bc01eaeaa6f3f1532ad562cbaf1
