Guanhua Wang 7e480ea02f offload++ blog (#623) (#4637) ...

This PR is the blog for ZeRO-Offload++, it describes the details of how
our new Twin-Flow feature works and its performance numbers on both
DGX-A100 and DGX-H100 machines.

Corresponding code PR is
https://github.com/microsoft/DeepSpeed/pull/4636

cc @jeffra @awan-10 @tjruwase @mrwyattii

---------

Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

2023-11-06 14:17:15 -08:00

41 KiB

546x364px

Raw History