Nemotron 3 Ultra is a hybrid Mamba-Attention MoE model with 550B total parameters and 55B active parameters, designed for long-running agents. It is pre-trained on 20 trillion text tokens and supports a 1M token context length. The hybrid Mamba-Attention architecture improves throughput efficiency by reducing attention costs and KV cache footprint, while the MoE design enhances accuracy per active parameter, enabling up to 5x throughput and 30% cost reduction compared to other open models.
For RTL workflows, efficiency is critical as agent contexts grow rapidly. A single debug iteration may include specification, module code, generated RTL, simulation output, assertion failures, and previous fix attempts. Efficient context reasoning directly impacts the practicality of these agents in real engineering.
Nemotron 3 Ultra's RTL capabilities benefit from the synthetic data generation pipeline proposed in the ACE-RTL paper. This pipeline aims to create high-quality RTL training data that reflects how hardware engineers and RTL agents actually work: not only generating code from specifications but also modifying existing implementations and debugging failing designs.