> [!WARNING]
> We are [immigrating to `ruff` as the linter and formatter and
`pre-commit` as the managing
tool](https://github.com/volcengine/verl/pull/1010).
>
> If your branch is based on a previous commit using `yapf` and
`pylint`, simply merging might trigger overwhelming linting errors,
while **you are only expected to resolve ones in the files related to
your PR**.
>
> To resolve this issue, please try the following workaround to only
include the files you **really changed** in the PR:
>
> 1. In your branch, fix linting and format with `ruff`: `ruff check
--fix && ruff-format`
> 2. Squash into a single commit in a new branch: `git reset --soft
$(git merge-base main HEAD) && git add -A && git commit -m "feat: ..."`
> 3. Merge with the latest main: `git merge origin/main`
> 4. Force push to your branch: `git push --force`
We add the reminder above to the documentation to tell contributors how
to avoid overwhelming linting errors.
### Motivation
According to dicussion in #896, this PR immigrates from yapf & pylint to
ruff based on pre-commit, which allows unified version control and
automatic hook on committing.
### Summary
The `pre-commit` hook and CI
- checks staged / committed files in commits / PR's
- checks all files each month (This should fail before we fix all the
files by the ruff standard)
### Explanation for the Failing CI Workflow `pre-commit`
For now, we only apply `ruff format` and `ruff check --fix` **without
resolving all the errors**, since there are too many errors to resolve,
which causes the CI workflow `pre-commit` fails.
For resolving the remaining errors, we leave to future commits.
Specifically, the `pre-commit` hook and CI will require every commit to
fix its related files with `ruff`, which will fix all the files
incrementally.
### Reviewing Suggestion
The commit
3d93f51ba8
is huge since we apply `ruff` to all the files. To review the main
changes, please check the commits before and after it.
* [ci] upload several tests
* [ci] add sanity and tensordict utility workflow
* [ci] fix workflow
* try fix import ci
* [dataproto] update repeat and unpad/pad
* fix rollout test to 2GPU
* add a fsdp vllm hybridengine script, which can be launched by torchrun
* fix import test
* update requirement.txt
* draft vllm fsdp test
* update label
* fix
* upload conda
* test conda
* test ci
* use docker
* test ci
* test ci
* test ci
* update ci
* test ci
* fix model loader
* fix model loader
* test ci
* test
* upload e2e digit completion test
* update running script for e2e test
* update test config
* fix path
* test
* fix import to register autotokenizer
* fix tokenizer
* fix create dataset
* fix
* fix reward model validate
* fix reward module of digit_completion
* fix reward module of digit_completion
* fix reward module of digit_completion
* fix reward module of digit_completion
* fix reward module of digit_completion
* can run but seems to have some test issue
* no problem, add check results
* add e2e training
* l20-0 seems has docker permission problem, test later
* fix
* test l20-0 and torchrun
* test l20-0 and torchrun
* fix
* fix
* fix
* fix
* fix
* tolerate difference
* tolerate difference with levenshtein
* lint
* add more test for ray
* delete
* use docker on l20
* use docker on l20
* add upgrade
* update ci
* delete code
* ignore test
* upgrade ray
* fix workerhelper method
* lint
* revert worker changes
* fix
* fix
* fix
* fix worker missing func
* [ci] add linting tests dir
* delete redundant tests
* [ci] change test gpu usage to 2
* [ci] update check ci
* [ci] fix init
* [ci] fix ray test running machine
* [ci] test self-hosted gpu
* [ci] fix yapf lint ci
* [ci] test checkout
* [ci] test origin
* [ci] fix ray test
* [ci] fix ray test
* [ci] add install verl in ray
