Transformer Engine is an open-source library developed by NVIDIA designed to accelerate Transformer model training and inference on NVIDIA GPUs. It supports FP8 precision on Hopper, Ada, and Blackwell GPU architectures, which reduces memory usage while maintaining performance.