mirror of
https://github.com/volcengine/verl.git
synced 2025-10-20 13:43:50 +08:00
1cfa2be530
### Checklist Before Starting
- [ ] Search for similar PR(s).
### What does this PR do?
Allow to override of transformer config to enable custom megatron
features like variable PP layers distribution, with CI tests, which is
in need for larger moe models with 94 layers (Qwen3 moe) or 61 layers
(DeepSeek V3)
We will first fix e2e_prime CI by use fused kernels.
**Notice that now the imbalance PP layers distribution only compatible
with dist_ckpt load and save, not support huggingface direct
load/save.**
Also, other megatron arguments can be passed through scripts.
### High-Level Design
> Demonstrate the high-level design if this PR is complex.
### Specific Changes
> List the specific changes.
### API
Breaking APIs:
```py
class MegatronWorker(Worker):
def _init_hf_config_and_tf_config(self, model_path, dtype, override_model_config, override_transformer_config):
# and the models building
```
```yaml
actor:
megatron:
override_transformer_config: {} # common transformer config for all models
```
To avoid trouble of input same transformer config arguments, other
models will reuse actor's config, so just need to input once.
### Usage Example
```bash
run_ppo_trainer_megatron.sh \
+actor_rollout_ref.actor.megatron.override_transformer_config.num_layers_in_first_pipeline_stage=13 \
+actor_rollout_ref.actor.megatron.override_transformer_config.num_layers_in_last_pipeline_stage=11
```
### Test
> For changes that can not be tested by CI (e.g., algorithm
implementation, new model support), validate by experiment(s) and show
results like training curve plots, evaluatuion results, etc.
### Additional Info.
- **Issue Number**: Fixes issue # or discussion # if any.
- **Training**: Megatron
- **Inference**: [Note which backend this PR will affect: vLLM, SGLang,
both, or none]
### Checklist Before Submitting
- [x] Read the [Contribute
Guide](https://github.com/volcengine/verl?tab=readme-ov-file#contribution-guide).
- [x] Apply [pre-commit
checks](https://github.com/volcengine/verl?tab=readme-ov-file#code-linting-and-formatting).
- [x] Add `[BREAKING]` to the PR title if it breaks any API.
- [x] Update the documentation about your changes in the
[docs](https://github.com/volcengine/verl/tree/main/docs).
- [x] Add CI test(s) if neccessary.
15 lines
354 B
JSON
15 lines
354 B
JSON
{
|
|
"[python]": {
|
|
"editor.defaultFormatter": "charliermarsh.ruff",
|
|
"editor.codeActionsOnSave": {
|
|
"source.organizeImports": "always",
|
|
}
|
|
},
|
|
"files.associations": {
|
|
"array": "cpp",
|
|
"string_view": "cpp",
|
|
"initializer_list": "cpp",
|
|
"utility": "cpp"
|
|
},
|
|
"iis.configDir": ""
|
|
} |