frozenleaves/pytorch
1
0
Fork 0
mirror of https://github.com/pytorch/pytorch.git synced 2025-10-20 12:54:11 +08:00
Code Issues Packages Projects Releases Wiki Activity

[lint] Move shellcheck to its own step (#58623)

Browse Source 
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/58623

This splits out everything shellcheck related into its own job that generates and checks GHA workflows, then shellchecks those + jenkins scripts. This PR also integrates shellcheck into the changed-only stuff in `actions_local_runner.py` so that shellcheck won't do anything unless someone edits a shell script in their local checkout. This is the final piece to clean up the output of `make quicklint` and speeds it up by a good bit (before it was shellchecking everything which took a few seconds):

```
$ make quicklint -j $(nproc)
✓ quick-checks: Ensure no unqualified noqa
✓ quick-checks: Ensure canonical include
✓ quick-checks: Ensure no unqualified type ignore
✓ quick-checks: Ensure no direct cub include
✓ quick-checks: Ensure no tabs
✓ quick-checks: Ensure no non-breaking spaces
✓ shellcheck: Regenerate workflows
✓ quick-checks: Ensure no versionless Python shebangs
✓ quick-checks: Ensure correct trailing newlines
✓ shellcheck: Assert that regenerating the workflows didn't change them
✓ mypy (skipped typestub generation)
✓ cmakelint: Run cmakelint
✓ quick-checks: Ensure no trailing spaces
✓ flake8
✓ shellcheck: Extract scripts from GitHub Actions workflows
✓ shellcheck: Run Shellcheck
real 0.92
user 6.12
sys 2.45
```

Test Plan: Imported from OSS

Reviewed By: nikithamalgifb

Differential Revision: D28617293

Pulled By: driazati

fbshipit-source-id: af960ed441db797d07697bfb8292aff5010ca45b
This commit is contained in:
driazati
2021-05-21 18:21:57 -07:00
committed by Facebook GitHub Bot
parent b842351b4f
commit 84b6c629d3
5 changed files with 115 additions and 45 deletions
Download Patch File Download Diff File Expand all files Collapse all files

38
Makefile
View File

@ -32,13 +32,33 @@ generate-gha-workflows:
.github/scripts/generate_ci_workflows.py
$(MAKE) shellcheck-gha
shellcheck:
@$(PYTHON) tools/actions_local_runner.py \
--file .github/workflows/lint.yml \
--job 'shellcheck' \
--step "Regenerate workflows"
@$(PYTHON) tools/actions_local_runner.py \
--file .github/workflows/lint.yml \
--job 'shellcheck' \
--step "Assert that regenerating the workflows didn't change them"
@$(PYTHON) tools/actions_local_runner.py \
--file .github/workflows/lint.yml \
--job 'shellcheck' \
--step 'Extract scripts from GitHub Actions workflows'
@$(PYTHON) tools/actions_local_runner.py \
--file-filter '.sh' \
$(CHANGED_ONLY) \
--job 'shellcheck'
setup_lint:
$(PYTHON) tools/actions_local_runner.py --file .github/workflows/lint.yml \
--job 'flake8-py3' --step 'Install dependencies' --no-quiet
--job 'flake8-py3' --step 'Install dependencies' --no-quiet
$(PYTHON) tools/actions_local_runner.py --file .github/workflows/lint.yml \
--job 'cmakelint' --step 'Install dependencies' --no-quiet
--job 'cmakelint' --step 'Install dependencies' --no-quiet
$(PYTHON) tools/actions_local_runner.py --file .github/workflows/lint.yml \
--job 'mypy' --step 'Install dependencies' --no-quiet
--job 'mypy' --step 'Install dependencies' --no-quiet
$(PYTHON) tools/actions_local_runner.py --file .github/workflows/lint.yml \
--job 'shellcheck' --step 'Install Jinja2' --no-quiet
@if [ "$$(uname)" = "Darwin" ]; then \
if [ -z "$$(which brew)" ]; then \
@ -48,16 +68,11 @@ setup_lint:
brew install shellcheck; \
else \
$(PYTHON) tools/actions_local_runner.py --file .github/workflows/lint.yml \
--job 'quick-checks' --step 'Install ShellCheck' --no-quiet; \
--job 'shellcheck' --step 'Install ShellCheck' --no-quiet; \
fi
pip install jinja2
quick_checks:
@$(PYTHON) tools/actions_local_runner.py \
--file .github/workflows/lint.yml \
--job 'quick-checks' \
--step 'Extract scripts from GitHub Actions workflows'
# TODO: This is broken when 'git config submodule.recurse' is 'true' since the
# lints will descend into third_party submodules
@$(PYTHON) tools/actions_local_runner.py \
@ -71,7 +86,6 @@ quick_checks:
--step 'Ensure no unqualified noqa' \
--step 'Ensure no unqualified type ignore' \
--step 'Ensure no direct cub include' \
--step 'Run ShellCheck' \
--step 'Ensure correct trailing newlines'
flake8:
@ -102,7 +116,7 @@ toc:
--job 'toc' \
--step "Regenerate ToCs and check that they didn't change"
lint: flake8 mypy quick_checks cmakelint generate-gha-workflows
lint: flake8 mypy quick_checks cmakelint shellcheck
quicklint: CHANGED_ONLY=--changed-only
quicklint: mypy flake8 mypy quick_checks cmakelint generate-gha-workflows
quicklint: mypy flake8 mypy quick_checks cmakelint shellcheck