Recovery Noloco
Two preprints out of the Gensyn research team: NoLoCo, a low-communication training method that avoids all-reduce, and All is Not Lost, on recovering LLM training without checkpoints.
Two preprints out of the Gensyn research team: NoLoCo, a low-communication training method that avoids all-reduce, and All is Not Lost, on recovering LLM training without checkpoints.