Post by @dielan@shitposter.world

2026-10-09 23:44 UTC

@sun@shitposter.world @mrsaturday@shitposter.world if I knew this AI stuff was coming like this in 2024 i would have gotten a slightly better card. I'm stuck at 20 GB VRAM with my 7900XT. So i can run 27B models with like no context window, or i can run smaller models one might recommend for a 16GB card with a big dick context window. It's not a bad life but damn i wish i had 24GB VRAM

Replies (4)

@sun@shitposter.world 2026-10-09 23:45

@dielan@shitposter.world @mrsaturday@shitposter.world I have a 24GiB 4090 and it's not enough for most cool models now :sadge:

Open ##5233605
@mrsaturday@shitposter.world 2026-10-09 23:47

@dielan@shitposter.world @sun@shitposter.world I managed to luck my way into getting a computer with a 4080 for $1700. I can gen images with Anima but I haven't tried self-hosting LLMs or running my own agents or anything.

Open ##5233627
@nerthos@shitposter.world 2026-10-09 23:47

@dielan@shitposter.world @sun@shitposter.world @mrsaturday@shitposter.world I would personally kill everyone involved in AI with a wooden club if that'd get GPU prices back to normal. Even my friends.

Open ##5233629
@bruh_momento@shota.house 2026-10-10 13:42

@dielan@shitposter.world @sun@shitposter.world @mrsaturday@shitposter.world You should be able to utilize larger models soon, with the latest techniques offloading less-critical things to system RAM or SSD, not to mention MoE architecture getting popular. People with 16 GB VRAM cards are squeezing in some big-ahh models fr fr.

Open ##5233635