Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/44568
By `lcov`, we can generate beautiful html. It's better than current file report and line report. Therefore in oss gcc, remove `export` code and `file/line level report` code, only use the html report.
But in clang, since such tool is not available, we will still use file-report and line-report generated by ourself.
Test Plan:
Test in docker ubuntu machine.
## Mesurement
1. After running `atest`, it takes about 15 mins to collect code coverage and genrate the report.
```
# gcc code coverage
python oss_coverage.py --run-only=atest
```
## Presentation
**The html result looks like:**
*Top Level:*
{F328330856}
*File Level:*
{F328336709}
Reviewed By: malfet
Differential Revision: D23550784
fbshipit-source-id: 1fff050e7f7d1cc8e86a6a200fd8db04b47f5f3e
Summary: Some tests like `test_dataloader.py` are not able to run under `clang` in oss, because it generates too large intermediate files (~40G) that can't be merged by `llvm`. Skip them when user doesn't specify the `--run-only` option
Test Plan: Test locally. But still, not recomend user to run `clang` coverage in default mode, because it takes too much space.
Reviewed By: malfet
Differential Revision: D23549829
fbshipit-source-id: 0737e6e9dcbe3f38de00580ee6007906e743e52f
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/43890
1. auto-detect `CXX` default compiler type in oss, and `clang` as default compiler type in fbcode (because auto-detecting will say `gcc` is the default compiler on devserver).
2. change `compiler type` from str `"CLANG" "GCC"` to enum type
3. rename function `get_cov_type` to `detect_compiler_type`
4. auto-set the default pytorch folder for users in oss
Test Plan:
on devserver:
```
buck run :coverage //caffe2/c10:
```
on oss:
```
python oss_coverage.py --run-only=atest
```
Reviewed By: malfet
Differential Revision: D23420034
fbshipit-source-id: c0ea88188578bb1343a286f2090eb8a74cdf3982
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/43889
1. rename input argunment `interested-folder` to `interest-only` -- be consistent with `run-only`, `coverage-only` and be shorted
Test Plan: Test on devserver and linux docker.
Reviewed By: malfet
Differential Revision: D23417338
fbshipit-source-id: ce9711e75ca3a1c30801ad6bd1a620f3b06819c5
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/43883
Check the result of GCC coverage in OSS is reasonable and ready to ship.
The amount of executable lines are not the same between `gcc` and `clang` because of the following reasons:
* Lines following are counted in `clang` but not in `gcc`:
1. empty line or line with only “{” or “}”
3. some comments are counted in clang but not in gcc
5. `#define ...` -- not supported by gcc according to official documentation
* Besides, a statement that explains to more than one line will be counted as only one executable line in gcc, but several lines in clang
## Advantage of `gcc` coverage
1. Much faster
- code coverage tool runtime is onle **4 min** (*ammazzzing!!*) by `gcc`, compared to **3 hours!!** by `clang`, to analyze all the tests' artifacts
2. Use less disk
- `Clang`'s artifacts will take as large as 170G, but `GCC` is 980M
Besides, also update `README.md`.
Test Plan:
Compare the result in OSS `clang` and OSS `gcc` with the same command:
```
python oss_coverage.py --run-only atest test_nn.py --interested-folder=aten
```
----
## GCC
**Summary**
> time: 0:15:45
summary percentage: 44.85%
**Report and Log**
[File Coverage Report](P140825162)
[Line Coverage Report](P140825196)
[Log](P140825385)
------
## CLANG
**Summary**
> time: 0:21:35
summary percentage: 44.08%
**Report and Log**
[File Coverage Report](P140825845)
[Line Coverage Report](P140825923)
[Log](P140825950)
----------
# Run all tests
```
# run all tests and get coverage over Pytorch
python oss_coverage.py
```
**Summary**
> time: 1:27:20. ( time to run tests: 1:23:33)
summary percentage: 56.62%
**Report and Log**
[File Coverage Report](P140837175)
[Log](P140837121)
Reviewed By: malfet
Differential Revision: D23416772
fbshipit-source-id: a6810fa4d8199690f10bd0a4f58a42ab2a22182b
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/43800
1. Move fbcode related coverage code to fb/ folder and add TARGETS so that we can use buck run to run the tool and solved the import probelm.
2. Write `README.md` to give users guidance about the tool
Test Plan:
On devserver:
```
buck run //caffe2/fb/code_coverage/tool:coverage -- //caffe2/c10:
```
More examples in README.md
Reviewed By: malfet
Differential Revision: D23404988
fbshipit-source-id: 4942cd0e0fb7bd28a5e884d9835b93f00adb7b92
