Moonshot AI released Kimi K3, a 2.8 trillion parameter vision-language model using a mixture-of-experts architecture, activating 16 experts per token (out of 896), with an estimated 50 billion active parameters. The model supports text, image, and video input (up to 1 million tokens), outputs text (up to 1 million tokens at 62.0 tokens/sec), and offers adjustable reasoning levels, tool calling, structured output, and automatic context caching.
On the Artificial Analysis Intelligence Index, Kimi K3 scores 57, ranking first among open-source models, behind only GPT-5.6 Sol (59) and Claude Fable 5 (60). On AutomationBench-AA, Kimi K3 leads all models with a 53% success rate; on GDPval-AA v2, its Elo rating of 1668 is second only to Claude Fable 5 (1760) and GPT-5.6 Sol (1743). Additionally, Kimi K3 tops the Arena.ai Code Arena WebDev leaderboard.
Kimi K3's architectural innovations include Kimi Delta Attention (KDA) and Attention Residuals. KDA is a linear attention mechanism that replaces per-token comparison with fixed-size memory updates, reducing memory usage by 75% and increasing output speed by 6x in 2025 experiments. Attention Residuals allow each layer to selectively attend to outputs from previous layers rather than simple summation, saving 20% training computation in experiments. Moonshot claims these improvements boost training efficiency about 2.5 times over the previous generation, but technical details await further reports.