2026-02-07 16:16 UTC
@oli@social.tchncs.de @riaschissl@sigmoid.social even for 20 people, what would run and on what hardware? even deepseek ~0.5T parameter model needs a tremendous amount of vram (around 1G/B params) and the price of it is in the ballpark of "let's buy a house that will be useless in 5 years"
Replies (1)
-
@riaschissl@sigmoid.social 2026-02-08 10:12
@dzervas@infosec.exchange @oli@social.tchncs.de the main question is: what is good enough for a given task? As we know since GPT 4.5, pure parameter count isn't the main enabler for "better" LLM results. And then we have Unsloth's Dynamic 2.0 quantization, which is extremely helpful to bring down the VRAM requirements while still keeping the accuracy at a very high level. So yes, pack two or three RTX 6000 Blackwell GPUs into a decent server, and you're good to go at a decent price.