Thinking Machines Welcomes Inkling

Inkling, released by Thinking Machines, has 975B total parameters and 41B active parameters, employing a Mixture-of-Experts (MoE) architecture with 256 experts. The model is trained on 45 trillion tokens covering text, images, audio, and video data. It is the first open-source large model to natively support image, text, and audio inputs, with a context window of 1M tokens.

The model is massive: the BF16 full checkpoint requires 2TB VRAM, and the NVFP4 quantized version requires 600GB VRAM. The NVFP4 version is well-calibrated and can run on Blackwell GPUs.

Inkling uses relative attention instead of the common RoPE position encoding. Each attention layer learns an additional relative feature R projection that adjusts based on the distance between query and key, injected into the attention module. This design allows the model to directly learn positional information.

The model employs hybrid attention: decoder layers alternate between sliding window attention (fixed context window) and global attention (full context length) at a 5:1 ratio. The last layer uses global attention to generate rich feature representations. Additionally, the model applies short 1D convolutions (SConv) on hidden states, reading the current token and the previous W-1 hidden states to help handle local dependencies, relieving the attention and MoE modules from local representation burdens.

Inkling's MoE routing scores both routed experts and shared experts simultaneously. Top-k selection picks from 6 routed experts, and additionally, 2 shared experts are always activated. This design combines the efficiency of sparse activation with the stability of shared experts.

Inkling's multimodal input processing uses simple modules rather than separate encoders. Images are processed through a hierarchical MLP patchifier: multiple linear layers progressively merge pixels, ultimately generating an embedding per patch. Audio inputs are processed via discretized mel spectrograms: each 100ms audio chunk is converted to mel scale and then classified into precise mel spectrogram bins. Image inputs also include an extra temporal dimension to support video processing, though official video performance out of the box has not been evaluated.

This design allows the model to natively handle any combination of modalities without external encoders, facilitating fine-tuning for downstream tasks.

Inkling is supported upon release by transformers 5.14.0's any-to-any pipeline, as well as SGLang, vLLM, and llama.cpp. Users can deploy via Hugging Face's serverless inference routing or local ggml quantization.

During inference, the reasoning_effort parameter is supported, offering seven levels: 'none', 'minimal', 'low', 'medium', 'high', 'xhigh', and 'max', allowing users to adjust reasoning depth based on task requirements. The model also includes speculative MTP (Multi-Token Prediction) layers to accelerate inference.

As the first native multimodal open-source large model, Inkling fills a gap in the open-source community for unified multimodal understanding. Its simple multimodal tower design lowers the fine-tuning barrier, while the MoE architecture makes inference relatively efficient (41B active parameters). However, the model's scale leads to extremely high deployment costs: the BF16 version requires 2TB VRAM, and even the quantized version requires 600GB VRAM, limiting widespread community use.

Additionally, the model's video processing capabilities have not been officially evaluated, and audio processing relies on discretized mel spectrograms, which may result in information loss for high-fidelity audio tasks. The actual effectiveness of innovative architectures like relative attention and short convolutions awaits further community validation.