2026-09-15 16:20 UTC
The 0.4.0 release of llama.cpp just added the n-cpu-ffn flag for offloading weights for dense models to the CPU to complement the existing n-cpu-moe flag for MoE models. If you have a 16GB GPU and some extra RAM to spare, you might even be able to run Qwen 3.8 27B at a usable speed after some experimentation. I don’t own a 16GB card, so I haven’t tested it myself, and it might not work as well as I am imagining.
Replies (0)
No replies.