DeepSpeed
Offers distributed training with ZeRO and 3D parallelism, serving as a competitor in large-scale LLM training.
Megatron-LM
Provides tensor, pipeline, and data parallelism for LLM training, similar in scope to TorchTitan.
Colossal-AI
Focuses on composable parallelism for large models, comparable to TorchTitan's approach.