Laptop Runs 744B-Parameter GLM Without a GPU? SSD-as-VRAM Project Explodes on GitHub

The core idea of Colibrì is to reverse the order of running a large model: instead of first figuring out how to fit the entire model into VRAM or memory and then computing, it loads whichever expert is needed on the spot from the SSD. The author JustVugg likens this approach to a JIT for model weights—rather than compiling the whole program in advance, it observes which code is actually running and then handles the hot paths.

The premise that makes this work is the MoE architecture. Take GLM-5.2: it has 744B total parameters, but each token actually activates only about 40B; the Router first decides which experts to hand it to, and then only a small subset is activated. Since the vast majority of experts are not needed at the moment, there is no reason to keep them occupying high-speed memory all the time.

So the model is split into two parts: the Dense portion that must be used on every inference—Attention, Embedding, shared experts, and so on—is about 17B parameters, taking up only about 9.9GB after int4, and stays resident in RAM; what really takes up space is the 19,456 routed experts, which after int4 still total about 370GB, all placed on NVMe SSD. According to the source, GLM-5.2 after int4 processing has about 372GB of weights and can run on a machine with a minimum of 16GB and a recommended 24GB or so of RAM, and a GPU is not even a necessity.

If every generated token required finding experts from the SSD on the spot, the speed would be very poor. On Colibrì's earliest development machine, a 12-core CPU + 25GB RAM, GLM-5.2 with a cold cache was indeed only about 0.05–0.1 token/s—it can run, but it is so slow as to be almost unusable.

So the project focuses its optimization on "how to fetch experts from the SSD as little as possible." It organizes VRAM, RAM, and NVMe SSD into a tiered memory system, with the principle that the more frequently used an expert is, the closer it lives: those already in VRAM or RAM are computed directly, those used most recently are kept in the RAM cache as much as possible, and only the truly infrequently used ones remain on the SSD. An LRU cache keeps recently called experts in RAM with priority; during operation it also records the usage count of each expert, and high-frequency experts get higher cache priority.

Going further, it does not wait for the Router to finish calling names before acting. According to the project's tests, there is a clear correlation between expert routing in adjacent layers, and predicting experts one layer ahead reaches 71.6% predictability, so while the current layer is still computing, the background can read ahead the experts the next layer may need, overlapping computation with SSD I/O as much as possible. In effect, a pure-CPU desktop with 128GB RAM can cache more experts and reaches about 1.8 token/s; if you stack up to 6 RTX 5090s so that all experts stay resident in the high-speed storage tier, decoding speed can reach 5.8–6.8 token/s.

Colibrì has an important design principle: where an expert is placed only determines speed and does not change the model itself. Whether an expert is already in VRAM or is temporarily read from the SSD, the Router's choice will not change because of it, and the weight precision used is exactly the same; it will not secretly compute fewer experts or switch to a different routing scheme just because the machine has less memory. The developer calls this approach AI memory multitiering.

There are also stackable engineering measures: if the machine has two SSDs, Colibrì supports placing a second copy of the model to spread the expert-reading workload across different drives and use the bandwidth of both in parallel. The barrier to entry is not high either—for GLM-5.2, for example, you need a few hundred KB of the Colibrì program and about 372GB of the model; the project provides precompiled versions for Linux, macOS, and Windows, building from source only requires gcc or clang together with OpenMP, a single coli chat gets you into a conversation, and the Web Dashboard lets you view token generation speed, the time taken by each stage, and the distribution of experts across VRAM, RAM, and disk in real time.

It should be made clear that the above speed figures and the 71.6% predictability all come from the project's own statements or the source's retelling, are source_claim, and have not yet been independently verified by a third party; actual performance may vary greatly across different hardware, models, and cache states.

Colibrì originally set out for GLM-5.2, and now covers 9 model families. The smaller ones include Qwen3.8-Flash-Next, Qwen3.6, and OLMoE; DeepSeek V4 Flash is 284B; GLM-5.2/5.3 is 744B; GLM-5.3-Flash is 321B; Thinking Machines' Inkling is 975B; and the largest is Moonshot's Kimi K3, with 2.8T total parameters and 104B activated parameters.

According to the configuration given by the source, Kimi K3's weights require about 1.6TB of storage space, but it can start running with 32GB+ of RAM, and a GPU is likewise not mandatory. Each model has its own separate set of C adaptation files, while common components such as the underlying IO, cache, and tokenizer reuse the same core. The author also welcomes community participation in experiments to find more efficient approaches.

This model list and the configuration requirements likewise come from the source's retelling and are source_claim; among them, Kimi K3's 1.6TB storage and 32GB RAM starting requirement have not been independently confirmed by first-party sources.