Commit Graph

11 Commits

Author SHA1 Message Date
634659e262 Update mypy to 1.4.1 (#91983)
Mostly fixes for PEP-484 violation (i.e. when default arg is set to None, but type is not annotated as optional)
Plus few real fixes:
  - Add missing `_get_upgraders_entry_map` to `torch/_C/__init__.pyi`
  - Add missing return statement to `torch._export. deserialize_graph`
  - Fix error message in `torch.ao.ns.fx.weight_utils.get_lstm_mod_weights`
  -
TODO (in followup PR):
  - Fix erroneous `isinstance` check in `torch/ao/quantization/_pt2e/qat_utils.py`
Pull Request resolved: https://github.com/pytorch/pytorch/pull/91983
Approved by: https://github.com/kit1980, https://github.com/ZainRizvi, https://github.com/huydhn, https://github.com/thiagocrepaldi, https://github.com/aaronenyeshi
2023-07-13 16:30:36 +00:00
e2a3817dfd [BE] Enable C419 rule for any all shortcircuiting (#99890)
Apparently https://github.com/pytorch/pytorch/pull/78142 made torch.JIT allow for simple generator expressions which allows us to enable rules that replace unnecessary list comprehensions with generators in any/all. This was originally part of #99280 but I split it off into this PR so that it can be easily reverted should anything break.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/99890
Approved by: https://github.com/justinchuby, https://github.com/kit1980, https://github.com/malfet
2023-04-25 15:02:13 +00:00
7554c10899 Fix typos under tools directory (#97779)
Fix typos under tools directory

Pull Request resolved: https://github.com/pytorch/pytorch/pull/97779
Approved by: https://github.com/clee2000, https://github.com/kit1980
2023-03-30 08:21:35 +00:00
591222f5d9 Fix use-dict-literal lint (#83718)
Fix use-dict-literal pylint suggestions by changing `dict()` to `{}`. This PR should do the change for every Python file except test/jit/test_list_dict.py, where I think the intent is to test the constructor.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/83718
Approved by: https://github.com/albanD
2022-08-24 00:26:46 +00:00
f625bb4bc9 [codemod][usort] apply import merging for fbcode (1 of 11) (#78973)
Summary:
Applies new import merging and sorting from µsort v1.0.

When merging imports, µsort will make a best-effort to move associated
comments to match merged elements, but there are known limitations due to
the diynamic nature of Python and developer tooling. These changes should
not produce any dangerous runtime changes, but may require touch-ups to
satisfy linters and other tooling.

Note that µsort uses case-insensitive, lexicographical sorting, which
results in a different ordering compared to isort. This provides a more
consistent sorting order, matching the case-insensitive order used when
sorting import statements by module name, and ensures that "frog", "FROG",
and "Frog" always sort next to each other.

For details on µsort's sorting and merging semantics, see the user guide:
https://usort.readthedocs.io/en/stable/guide.html#sorting

Test Plan: S271899

Reviewed By: lisroach

Differential Revision: D36402110

Pull Request resolved: https://github.com/pytorch/pytorch/pull/78973
Approved by: https://github.com/osalpekar
2022-06-06 23:44:28 +00:00
737d920b21 Strictly type everything in .github and tools (#59117)
Summary:
This PR greatly simplifies `mypy-strict.ini` by strictly typing everything in `.github` and `tools`, rather than picking and choosing only specific files in those two dirs. It also removes `warn_unused_ignores` from `mypy-strict.ini`, for reasons described in https://github.com/pytorch/pytorch/pull/56402#issuecomment-822743795: basically, that setting makes life more difficult depending on what libraries you have installed locally vs in CI (e.g. `ruamel`).

Pull Request resolved: https://github.com/pytorch/pytorch/pull/59117

Test Plan:
```
flake8
mypy --config mypy-strict.ini
```

Reviewed By: malfet

Differential Revision: D28765386

Pulled By: samestep

fbshipit-source-id: 3e744e301c7a464f8a2a2428fcdbad534e231f2e
2021-06-07 14:49:36 -07:00
f3a79b881f add lcov to oss for beautiful html report (#44568)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/44568

By `lcov`, we can generate beautiful html. It's better than current file report and line report. Therefore in oss gcc, remove `export` code and `file/line level report` code, only use the html report.

But in clang, since such tool is not available, we will still use file-report and line-report generated by ourself.

Test Plan:
Test in docker ubuntu machine.
## Mesurement
1. After running `atest`, it takes about 15 mins to collect code coverage and genrate the report.
```
# gcc code coverage
python oss_coverage.py --run-only=atest
```

## Presentation
**The html result looks like:**

*Top Level:*

{F328330856}

*File Level:*

{F328336709}

Reviewed By: malfet

Differential Revision: D23550784

fbshipit-source-id: 1fff050e7f7d1cc8e86a6a200fd8db04b47f5f3e
2020-09-11 15:29:24 -07:00
f9146b4598 fix lint (#44346)
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/44346

Reviewed By: jamesr66a

Differential Revision: D23589324

Pulled By: eellison

fbshipit-source-id: a4e22b69196909ec200ac3e262f04d2aaf78e9cf
2020-09-08 18:29:44 -07:00
49e979bfde Set default compiler differently according to platform (#43890)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/43890

1. auto-detect `CXX` default compiler type in oss, and `clang` as default compiler type in fbcode (because auto-detecting will say `gcc` is the default compiler on devserver).

2. change `compiler type` from str `"CLANG" "GCC"` to enum type
3. rename function `get_cov_type` to `detect_compiler_type`
4. auto-set the default pytorch folder for users in oss

Test Plan:
on devserver:
```
buck run :coverage //caffe2/c10:
```

on oss:
```
python oss_coverage.py --run-only=atest
```

Reviewed By: malfet

Differential Revision: D23420034

fbshipit-source-id: c0ea88188578bb1343a286f2090eb8a74cdf3982
2020-09-08 14:57:35 -07:00
e941a462a3 Enable gcc coverage in OSS (#43883)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/43883

Check the result of GCC coverage in OSS is reasonable and ready to ship.

The amount of executable lines are not the same between `gcc` and `clang` because of the following reasons:
* Lines following are counted in `clang` but not in `gcc`:
1. empty line or line with only “{” or “}”
3. some comments are counted in clang but not in gcc
5. `#define ...` -- not supported by gcc according to official documentation

* Besides, a statement that explains to more than one line will be counted as only one executable line in gcc, but several lines in clang

## Advantage of `gcc` coverage
1. Much faster
- code coverage tool runtime is onle **4 min** (*ammazzzing!!*) by `gcc`, compared to **3 hours!!** by `clang`, to analyze all the tests' artifacts
2. Use less disk
- `Clang`'s artifacts will take as large as 170G, but `GCC` is 980M

Besides, also update `README.md`.

Test Plan:
Compare the result in OSS `clang` and OSS `gcc` with the same command:
```
python oss_coverage.py --run-only atest test_nn.py --interested-folder=aten
```

----

## GCC
**Summary**
> time: 0:15:45
summary percentage: 44.85%

**Report and Log**
[File Coverage Report](P140825162)
[Line Coverage Report](P140825196)
[Log](P140825385)

------

## CLANG

**Summary**
> time: 0:21:35
summary percentage: 44.08%

**Report and Log**
[File Coverage Report](P140825845)
[Line Coverage Report](P140825923)
[Log](P140825950)

----------

# Run all tests
```
# run all tests and get coverage over Pytorch
python oss_coverage.py
```
**Summary**
> time: 1:27:20. ( time to run tests:  1:23:33)
summary percentage: 56.62%

**Report and Log**
[File Coverage Report](P140837175)
[Log](P140837121)

Reviewed By: malfet

Differential Revision: D23416772

fbshipit-source-id: a6810fa4d8199690f10bd0a4f58a42ab2a22182b
2020-08-31 16:11:33 -07:00
0564d7a652 Land code coverage tool for OSS (#43778)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/43778

Move code_coverage_tool from experimental folder to caffe2/tools folder.

Delete `TODO` and fb-related code.

Test Plan: Test locally

Reviewed By: malfet

Differential Revision: D23399983

fbshipit-source-id: 92316fd3cc88409d087d2dc6ed0be674155b3762
2020-08-28 13:56:15 -07:00