2026-09-15 16:05 UTC
This is great info! I’ve been using OpenCode Go because I only have 16GB to work with and their limits are pretty generous. Looking forward to tiny, competent local models. Thanks o7
Replies (2)
-
@melfie@lemmy.zip 2026-09-15 16:20
The 0.4.0 release of llama.cpp just added the n-cpu-ffn flag for offloading weights for dense models to the CPU to complement the existing n-cpu-moe flag for MoE models. If you have a 16GB GPU and some extra RAM to spare, you might even be able to run Qwen 3.8 27B at a usable speed after some experimentation. I don’t own a 16GB card, so I haven’t tested it myself, and it might not work as well as I am imagining.
-
@sudoer777@lemmy.ml 2026-09-16 07:40
I’m going to stick with OpenCode Go, but the available models keep getting more expensive and the DeepSeek peak hours is annoying to deal with