frozenleaves/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-10-20 17:13:56 +08:00
Code Issues Packages Projects Releases Wiki Activity
Files
f64354e89a55a456e024b301468da2dbb10e11b3
transformers/docs/source/en/main_classes/deepspeed.md
Yuanyuan Chen f64354e89a Format empty lines and white space in markdown files. (#41100) 
* Remove additional white space and empty lines from markdown files

Signed-off-by: Yuanyuan Chen <cyyever@outlook.com>

* Add empty lines around code

Signed-off-by: Yuanyuan Chen <cyyever@outlook.com>

---------

Signed-off-by: Yuanyuan Chen <cyyever@outlook.com>
2025-09-23 16:20:01 -07:00

1.5 KiB
Raw Blame History

DeepSpeed

DeepSpeed, powered by Zero Redundancy Optimizer (ZeRO), is an optimization library for training and fitting very large models onto a GPU. It is available in several ZeRO stages, where each stage progressively saves more GPU memory by partitioning the optimizer state, gradients, parameters, and enabling offloading to a CPU or NVMe. DeepSpeed is integrated with the [Trainer] class and most of the setup is automatically taken care of for you.

However, if you want to use DeepSpeed without the [Trainer], Transformers provides a [HfDeepSpeedConfig] class.

Learn more about using DeepSpeed with [Trainer] in the DeepSpeed guide.

HfDeepSpeedConfig

autodoc integrations.HfDeepSpeedConfig - all