Post by @Architeuthis@awful.systems

2026-10-30 14:56 UTC

Uh-oh, renowned everyman Sam Altman opened his mouth again. Local inference is a terrible allocation of scarce resources. A lot of people want local inference, and somehow they assume that the problem is solved now because we have open-source models, we’re going to be able to run them on our computers, and so we’re not going to need all the stuff in the data centre but from a global allocation perspective, actually it’s quite terrible to have everybody, like, have a bunch of the world’s limited supply of DRAM sitting in their own computer and being used five percent of the time you want maximum utilisation in the cloud if you’re trying to deal with this very supply-constrained environment, somehow many people’s intuition is the opposite of that

Replies (7)

@Soyweiser@awful.systems 2026-10-30 15:14

5% mister? I run firefox.

Open ##5344949
@sailor_sega_saturn@awful.systems 2026-10-30 15:17

Much better for RAM to be sitting behind a network hop with 100 milliseconds of latency amirite? if you’re trying to deal with this very supply-constrained environment, We’re all trying to find the guy who did this .jpg

Open ##5345045
@istewart@awful.systems 2026-10-30 16:19

Not that Altman’s company is ahead in the AI coding race, but everyone’s PCs will still need increasing amounts of DRAM to be able to run the next generation of slop-coded JavaScript apps.

Open ##5351038
@mardor@ruhr.social 2026-10-30 20:02

@Architeuthis@awful.systems @BlueMonday1984@awful.systems Now let’s talk about privately owned cars versus public transport, Sam.

Open ##5363815
@cyclopentane@rheinneckar.social 2026-10-30 20:21

YOU CAN TAKE MY LOCAL COMPUTE FROM MY COLD DEAD HANDS

Open ##5364310
@swlabr@awful.systems 2026-10-30 20:44

I’ll donate my RAM to SAlty if he comes to collect it in person, and only if I can deliver it straight to his dome

Open ##5364921
@BioMan@awful.systems 2026-11-01 20:17

And here I thought I was building servers two years ago because the next Asian war would disrupt supply chains.

Open ##5436696