Jamba combines the strengths of both Mamba (SSM) and Transformer architectures, enabling high throughput and performance while maintaining a large context window.
Process and analyze extremely long documents, such as financial reports, legal contracts, or entire codebases, without losing context.
Jamba uses an MoE architecture with 16 experts, of which 2 are active per token, to optimize performance and efficiency.
Jamba is an open-source model released under the Apache 2.0 license, allowing for self-hosting and custom fine-tuning.