UlyssesPlus Docs take 2 (#7332)

bare md urls don't get automatically linked, so fixing that.
This commit is contained in:
Stas Bekman
2025-06-03 12:14:06 -07:00
committed by GitHub
parent 81a47408c3
commit 097f0637d5

View File

@ -12,11 +12,11 @@ tags: training, sequence-parallelism
It enables training on 0.5M long sequences on a single H100 CPU and a 15M-long sequences on LLama-8B on four 8x H100 nodes.
It's already fully integrated into Arctic Training, see: https://github.com/snowflakedb/ArcticTraining/blob/main/projects/sequence-parallelism/
It's already fully integrated into Arctic Training, see [this guide](https://github.com/snowflakedb/ArcticTraining/blob/main/projects/sequence-parallelism/).
The rest of the document explains how to integrated it into other frameworks or your own training loop.
The rest of the document explains how to integrate it into other frameworks or your own training loop.
There is another older version of UlyssesSP which only works with Megatron-Deepspeed and can be found here https://www.deepspeed.ai/tutorials/ds-sequence/
There is another older version of UlyssesSP which only works with Megatron-Deepspeed and can be found [here](https://www.deepspeed.ai/tutorials/ds-sequence/).
## Part 1: Ulysses Sequence Parallelism for HF Transformers