2026-02-08 10:12 UTC
@dzervas@infosec.exchange @oli@social.tchncs.de the main question is: what is good enough for a given task? As we know since GPT 4.5, pure parameter count isn't the main enabler for "better" LLM results. And then we have Unsloth's Dynamic 2.0 quantization, which is extremely helpful to bring down the VRAM requirements while still keeping the accuracy at a very high level. So yes, pack two or three RTX 6000 Blackwell GPUs into a decent server, and you're good to go at a decent price.
Replies (1)
-
@dzervas@infosec.exchange 2026-02-08 11:23
@oli@social.tchncs.de @riaschissl@sigmoid.social I agree with you that pure parameter count is not what we're after but "a few rtx 6000" translates to roughly 30-40k + supporting hardware around it + electricity. That's nowhere near what 20 people would spend on AI subscriptions and the performance of the model you'll be running is simply worse - maybe by a lot maybe by a little, currently is by a lot but that could change soon and in 5 years you would still have to buy new hardware, doing a similar (or bigger) investment. I fail to see how that makes any kind of economical sense