frozenleaves/pytorch
1
0
Fork 0
mirror of https://github.com/pytorch/pytorch.git synced 2025-10-20 21:14:14 +08:00
Code Issues Packages Projects Releases Wiki Activity

[functorch] removed unnecessary note

Browse Source
This commit is contained in:
Horace He
2021-09-07 21:15:48 -07:00
committed by Jon Janzen
parent fcd08e1adc
commit b23c9b907c
1 changed files with 1 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
functorch/examples/ensembling/parallel_train.py
View File

@ -16,8 +16,6 @@ from functorch import make_functional, grad_and_value, vmap, combine_state_for_e
# GOAL: Demonstrate that it is possible to use eager-mode vmap
# to parallelize training over models.
# NB: this code runs off of a branch on zou3519/pytorch:dynlayer
DEVICE = 'cpu'
# Step 1: Make some spirals
@ -110,7 +108,7 @@ def step6():
step6()
# Step 7: Now, the flaw with step 6 is that we were training on the same exact
# data. This can lead to all of the models in the ensemble overfitting in the
# data. This can lead to all of the models in the ensemble overfitting in the
# same way. The solution that http://willwhitney.com/parallel-training-jax.html
# applies is to randomly subset the data in a way that the models do not recieve
# exactly the same data in each training step!