2026-09-15 10:38 UTC
You may be overestimating the hardware requirements for current open weights. I am running Qwen3.8-27b with 52.000 tokens context on a RX 7900 XTX from 2022 with 15 - 30 tokens / second. Sure, it was a high end card back then but for a graphics card today it’s quite affordable when you’re looking for a used one. Also it’s not blazingly fast, but it gets the job done.
Replies (1)
-
@NihilsineNefas@slrpnk.net 2026-09-15 11:15
“Just run it local, you can use cheap hardware” is running it on a $1200 card and is still struggling to get it done quickly Keep on sucking up that slop, I’m sure paying a company to do your thinking for you will pay off in the end, as we’ve been seeing.