mirror of
https://github.com/volcengine/verl.git
synced 2025-10-20 05:33:49 +08:00
4da0d3d318
### What does this PR do? > Add **concise** overview of what this PR aims to achieve or accomplish. Reference related GitHub issues and PRs that help with the review. Fixes Issue with metric name validation when using MLFlow. Keeps replacement of `@` with `_at_` for backward compatibility. Resolves https://github.com/volcengine/verl/issues/1242 ### Checklist Before Starting - [x] Search for similar PRs. Paste at least one query link here: N/A - [x] Format the PR title as `[{modules}] {type}: {description}` (This will be checked by the CI) - `{modules}` include `fsdp`, `megatron`, `sglang`, `vllm`, `rollout`, `trainer`, `ci`, `training_utils`, `recipe`, `hardware`, `deployment`, `ray`, `worker`, `single_controller`, `misc`, `perf`, `model`, `algo`, `env`, `tool`, `ckpt`, `doc`, `data` - If this PR involves multiple modules, separate them with `,` like `[megatron, fsdp, doc]` - `{type}` is in `feat`, `fix`, `refactor`, `chore`, `test` - If this PR breaks any API (CLI arguments, config, function signature, etc.), add `[BREAKING]` to the beginning of the title. - Example: `[BREAKING][fsdp, megatron] feat: dynamic batching` ### Test > For changes that can not be tested by CI (e.g., algorithm implementation, new model support), validate by experiment(s) and show results like training curve plots, evaluation results, etc. Adds a unit tests to validate metric name cleanup ### API and Usage Example > Demonstrate how the API changes if any, and provide usage example(s) if possible. ```python # Add code snippet or script demonstrating how to use this ``` ### Design & Code Changes > Demonstrate the high-level design if this PR is complex, and list the specific changes. ### Checklist Before Submitting > [!IMPORTANT] > Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review. - [x ] Read the [Contribute Guide](https://github.com/volcengine/verl/blob/main/CONTRIBUTING.md). - [x] Apply [pre-commit checks](https://github.com/volcengine/verl/blob/main/CONTRIBUTING.md#code-linting-and-formatting): `pre-commit install && pre-commit run --all-files --show-diff-on-failure --color=always` - [x] Add / Update [the documentation](https://github.com/volcengine/verl/tree/main/docs). - [x] Add unit or end-to-end test(s) to [the CI workflow](https://github.com/volcengine/verl/tree/main/.github/workflows) to cover all the code. If not feasible, explain why: ... - [x] Once your PR is ready for CI, send a message in [the `ci-request` channel](https://verl-project.slack.com/archives/C091TCESWB1) in [the `verl` Slack workspace](https://join.slack.com/t/verl-project/shared_invite/zt-3855yhg8g-CTkqXu~hKojPCmo7k_yXTQ). (If not accessible, please try [the Feishu group (飞书群)](https://applink.larkoffice.com/client/chat/chatter/add_by_link?link_token=772jd4f1-cd91-441e-a820-498c6614126a).)
Tests layout
Each folder under tests/ corresponds to a test category for a sub-namespace in verl. For instance:
tests/trainerfor testing functionality related toverl/trainertests/modelsfor testing functionality related toverl/models- ...
There are a few folders with special_ prefix, created for special purposes:
special_distributed: unit tests that must run with multiple GPUsspecial_e2e: end-to-end tests with training/generation scriptsspecial_npu: tests for NPUsspecial_sanity: a suite of quick sanity testsspecial_standalone: a set of test that are designed to run in dedicated environments
Accelerators for tests
- By default tests are run with GPU available, except for the ones under
special_npu, and any test script whose name ends withon_cpu.py. - For test scripts with
on_cpu.pyname suffix would be tested on CPU resources in linux environment.
Workflow layout
All CI tests are configured by yaml files in .github/workflows/. Here's an overview of all test configs:
- A list of always triggered CPU sanity tests:
check-pr-title.yml,secrets_scan.yml,check-pr-title,yml,pre-commit.yml,doc.yml - Some heavy multi-GPU unit tests, such as
model.yml,vllm.yml,sgl.yml - End-to-end tests:
e2e_*.yml - Unit tests
cpu_unit_tests.yml, run pytest on all scripts with file name patterntests/**/test_*_on_cpu.pygpu_unit_tests.yml, run pytest on all scripts with file without theon_cpu.pysuffix.- Since cpu/gpu unit tests by default runs all tests under
tests, please make sure tests are manually excluded in them when- new workflow yaml is added to
.github/workflows - new tests are added to workflow mentioned in 2.
- new workflow yaml is added to