[pipelining] Add pipeline schedules (#125975)

1. Add pipeline schedules:
- GPipe
- 1F1B
- Interleaved 1F1B
- LoopedBFS

2. Add basic forward and backward tests:
test_schedule.py

Pull Request resolved: https://github.com/pytorch/pytorch/pull/125975
Approved by: https://github.com/wconstab
ghstack dependencies: #125729

torch/distributed/pipelining/microbatch.py

@@ -22,10 +22,11 @@ class _CustomReducer:
     reduces losses of multiple microbatches into one value.
 
     Example:
-        >>> sum_reducer = _CustomReducer(
-        >>>     torch.tensor(0.0),
-        >>>     lambda a, b: a + b
-        >>> )
+    >>> # xdoctest: +SKIP
+    >>> sum_reducer = _CustomReducer(
+    >>>     torch.tensor(0.0),
+    >>>     lambda a, b: a + b
+    >>> )
     """
 
     def __init__(self, init_value, reduce_fn):