Post by @BioMan@awful.systems

2026-11-01 17:08 UTC

Am I part of this category? I have a rather powerful home server that, among other things, I use to: Run a decent coding LLM that helps me vomit up simple python functions for moving around lots of data between CSVs and graphing stuff, set up new docker containers with less trial and error, and run ML experiments on my GPU that previously took me hours to set up. I’m a biologist and previously I could run a commandline without breaking a computer but was very slow and non-native, now I can do simple non-security-critical things faster and actually understand novel error messages. Of note I never have that hooked up in a way that lets it run things just produce text that I can then do things with and maybe run myself. Run my own local RAG system with embedding models and a small shitty LLM to search a hundred gigabytes of scientific papers and other PDFs semantically for things relevant to roughly paragraph length queries. Run my own web metasearch engine with RAG and embedding models pointed at the internet, such that when I remember a paper I read fifteen years ago and provide a three sentence description it reliably finds it or finds lots of relevant old research (or old news articles I remember) when I give it a query and let it churn for five minutes. I feel like the important thing here is that I almost exclusively use them as information channels that take something at one end and spit it back out the other in a different or filtered form rather than making stuff, or effectively a piece of reference material for something that is formally verifiable and NOT CRITICAL FOR A SYSTEM TO FUNCTION.

Replies (2)

@YourNetworkIsHaunted@awful.systems 2026-11-01 17:47

I don’t find myself having nearly as many occasions to make use of them as you do, but I do think that all the misplaced AGI hype makes it hard to evaluate the actually technological advancement that LLMs represent. Like, being able to statistically model language to the degree these things do is neat, and I would be more surprised if it wasn’t possible to use that to do some useful and powerful things. But just like how it’s impossible to separate the underlying math of the cryptocurrency sphere from the ideology of technolibertarian griftocracy, everything that the LLM space has produced is poisoned on some level by millenarian technofascism. Like, the degree to which our economy and culture have been warped by these things is entirely a product of the people who think that they’re going to spit out a God that makes whoever does so first king of the world forever. All the actual useful things people are getting from it are around the edges and happening in spite of the demands to remove guardrails and quality checks and human accountability from these systems. But that doesn’t necessarily mean that the underlying mathematics is without value and can’t be part of good and useful systems in the future. Machine Learning as a whole had already demonstrated the value in automated pattern identification and reproduction in a variety of fields from medical diagnostics to pastry sales. Don’t get me wrong, I’m strongly in favor of throwing out all this bathwater even if there might be a baby in there somewhere. I just think that “hard to evaluate independently” isn’t necessarily the same thing as “without value in other contexts”.

Open ##5433595
@jaschop@awful.systems 2026-11-01 18:30

Can you answer a question about RAG for me? My understanding is based entirely on this diagram from Wikipedia If RAG takes into account vector similarity between query and reference material, do you need an LLM to regurgitate the found best vectors back to you? Could you get back a list of documents that best matched your query, or are the results in an inscrutable LLM-internal representation that need to be transformered back into readable text?

Open ##5434251