Torchtitan

TorchTitan is an open-source PyTorch-native platform for distributed training of large language models with multi-dimensional composable parallelism.