Meta Expands Custom Silicon Strategy from Compute to Networking

Meta has introduced MTIA 300, its first custom accelerator optimized for training ranking and recommendation models. Unlike LLM training, recommendation models spend more time communicating between accelerators because embedding tables can contain over 99% of the model parameters. When training across hundreds of accelerators, frequent AllReduce, AllToAll, and AllGather operations make the network as important as the compute itself.

This design stems from an in-depth analysis of the workload: communication pressure comes mainly from embedding tables, not floating-point operations. Therefore, MTIA 300 integrates network interfaces directly into the accelerator package, featuring two network chiplets, each with six 800 Gbps RDMA NICs, totaling 12 NICs, providing 1.2 TB/s of total I/O bandwidth without going through the PCIe bus. These NICs support both intra-rack and inter-rack scaling, with bandwidth allocation dynamically adjustable without redesigning the chip.

Moving NICs closer to compute does not eliminate all contention: on traditional GPUs, collective communication consumes processing resources needed for training. MTIA 300 includes 16 dedicated message engines that handle communication independently from the main compute grid, along with near-memory hardware for reduction operations. Meta reports that this separation results in less than 0.5% degradation in compute throughput when large matrix operations and collective communication run concurrently, whereas the compared GPU architecture degrades by over 20% when overlapped.

The hardware is co-designed with Meta's collective communication library, HCCL. HCCL compiles collective operations into subgraphs that are executed autonomously by MTIA 300's message engines, with the host CPU no longer orchestrating communication operation-by-operation. Once instructions reach the accelerator, the host is no longer involved in driving communication. In production, HCCL achieves up to 940 GB/s of intra-rack communication bandwidth.

On a 150-billion-parameter recommendation model with 40 accelerators, Meta reports that MTIA 300 reduces total communication time by 3.9x compared to an equivalent GPU cluster. This figure comes from Meta's testing and is a company claim, not independently verified.

MTIA 300 is part of Meta's broader custom chip initiative. The company says it has deployed hundreds of thousands of MTIA accelerators for inference and plans to release four new chip generations over the next two years, covering ranking, recommendation, and generative AI workloads. Meta has also expanded its partnership with Broadcom to co-develop future MTIA generations, while continuing to source accelerators from vendors like AMD and NVIDIA as part of its portfolio strategy.

This strategy reflects a shift among hyperscalers toward workload-specific AI chips: Google is expanding TPUs, Amazon is developing Trainium, and Microsoft is working on Maia. MTIA 300 demonstrates Meta taking specialization to the extreme: rather than treating the network as peripheral infrastructure, it co-designs compute, communication hardware, and collective software around workloads where data movement is as important as processing.