Transformerengine

Transformer Engine is an NVIDIA open-source library that accelerates Transformer models on supported GPUs using FP8 precision.