Megatron-LM is an open-source framework by NVIDIA designed for training large-scale transformer-based language models efficiently across multiple GPUs and nodes.
How Megatron-LM helps
Megatron-LM is an open-source framework by NVIDIA designed for training large-scale transformer-based language models efficiently across multiple GPUs and nodes.