Thinking Machines Lab, founded by former OpenAI CTO Mira Murati, released Inkling, a 975-billion-parameter multimodal open-weights model. It leads U.S. open-weights models on the Artificial Analysis Intelligence Index but trails top Chinese open models on some tasks. Priced at $1.87 per million input tokens, it is positioned as a base for fine-tuning.
SynthePulse Insight · AI deep reading
Version 1 · 1 source
Thinking Machines releases Inkling, a 975B total parameter, 41B active parameter MoE model with native support for image, text, and audio inputs, a 1M context window, and innovative architectures including relative attention, hybrid attention, and short convolutions. This article provides an in-depth analysis of its technical features, inference deployment options, and impact on the open-source ecosystem.
Inkling, released by Thinking Machines, has 975B total parameters and 41B active parameters, employing a Mixture-of-Experts (MoE) architecture with 256 experts. The model is trained on 45 trillion tokens covering text, images, audio, and video data. It is the first open-source large model to natively support image, text, and audio inputs, with a context window of 1M tokens.
The model is massive: the BF16 full checkpoint requires 2TB VRAM, and the NVFP4 quantized version requires 600GB VRAM. The NVFP4 version is well-calibrated and can run on Blackwell GPUs.
Inkling uses relative attention instead of the common RoPE position encoding. Each attention layer learns an additional relative feature R projection that adjusts based on the distance between query and key, injected into the attention module. This design allows the model to directly learn positional information.
The model employs hybrid attention: decoder layers alternate between sliding window attention (fixed context window) and global attention (full context length) at a 5:1 ratio. The last layer uses global attention to generate rich feature representations. Additionally, the model applies short 1D convolutions (SConv) on hidden states, reading the current token and the previous W-1 hidden states to help handle local dependencies, relieving the attention and MoE modules from local representation burdens.
Inkling's MoE routing scores both routed experts and shared experts simultaneously. Top-k selection picks from 6 routed experts, and additionally, 2 shared experts are always activated. This design combines the efficiency of sparse activation with the stability of shared experts.
Inkling's multimodal input processing uses simple modules rather than separate encoders. Images are processed through a hierarchical MLP patchifier: multiple linear layers progressively merge pixels, ultimately generating an embedding per patch. Audio inputs are processed via discretized mel spectrograms: each 100ms audio chunk is converted to mel scale and then classified into precise mel spectrogram bins. Image inputs also include an extra temporal dimension to support video processing, though official video performance out of the box has not been evaluated.
This design allows the model to natively handle any combination of modalities without external encoders, facilitating fine-tuning for downstream tasks.
Inkling is supported upon release by transformers 5.14.0's any-to-any pipeline, as well as SGLang, vLLM, and llama.cpp. Users can deploy via Hugging Face's serverless inference routing or local ggml quantization.
During inference, the reasoning_effort parameter is supported, offering seven levels: 'none', 'minimal', 'low', 'medium', 'high', 'xhigh', and 'max', allowing users to adjust reasoning depth based on task requirements. The model also includes speculative MTP (Multi-Token Prediction) layers to accelerate inference.
As the first native multimodal open-source large model, Inkling fills a gap in the open-source community for unified multimodal understanding. Its simple multimodal tower design lowers the fine-tuning barrier, while the MoE architecture makes inference relatively efficient (41B active parameters). However, the model's scale leads to extremely high deployment costs: the BF16 version requires 2TB VRAM, and even the quantized version requires 600GB VRAM, limiting widespread community use.
Additionally, the model's video processing capabilities have not been officially evaluated, and audio processing relies on discretized mel spectrograms, which may result in information loss for high-fidelity audio tasks. The actual effectiveness of innovative architectures like relative attention and short convolutions awaits further community validation.
The information in this article primarily comes from the official blog post released by Thinking Machines on Hugging Face, which is a product announcement. Technical details (such as parameter counts and architecture design) are from official descriptions and are highly credible. However, the model's actual performance and inference speed have not been provided with independent benchmarks and require subsequent community verification.
Inkling is a significant milestone for open-source multimodal large models. Its native multimodal capabilities, 1M context window, and MoE architecture provide a powerful foundation model for the community, but high deployment costs and dependence on specific hardware remain major barriers to widespread adoption.
Primary report
Primary source
Also covered by 4 sources