2026-10-26 00:29 UTC
Replies (40)
-
@BlueMonday1984@awful.systems 2026-10-26 01:05
Starting this Stubsack off: a pentester talks about testing an AI chatbot and its associated API, finding vulnerabilities they hadn’t seen in a decade.
-
@swlabr@awful.systems 2026-10-26 03:29
Been getting the ol’ targeted ads for an EA convention. Next time I get one I’ll be shilling the shrimp welfare farm I just started
-
@nfultz@awful.systems 2026-10-26 03:43
www.patreon.com/…/ai-x-risk-and-ai-173181621 worth the full read, but here’s the parts I liked But “x-risk” is not the only kind of “existential” risk about which AI enthusiasts and their converts tend to worry. This recent Ezra Klein column, written in the wake of OpenAI’s release of hundreds of mathematical proofs, is a nice articulation of something I’ve come to think of as “n-risk,” for “ennui risk” (with the mutual understanding that it’s not pronunciation-accurate so don’t bother making a nitpicky little comment about it): fear of the loss of comprehension, control, and meaning in the face of impossibly sophisticated godlike machines poses. We see the threat of “n-risk” in some of the emotional reaction of mathematicians faced with the results of OpenAI models bulldozing through open problems: a sense of having been surpassed and left behind, having the meaning from which one derives one’s worth totally emptied. In some sense, on a conceptual level and as a discursive object, x-risk is preferable to n-risk: it’s striking, dramatic, and offers us humans an ongoing chance at great feats of heroism and genius. N-risk has no such narrative reassurances. Maybe it’s better to burn out (as your body is turned into a paperclip) than to fade away. Here you might ask: Is the cause of melancholy and despair on the part of some mathematicians simply the knowledge that a computer system has surpassed their ability to understand or produce proofs? Or is it the realization that said computer system is privately owned and confidentially developed in service of accumulation rather than the pursuit of shared human knowledge? Or is it the contempt with which the people who work for the computer system companies and the hanger-on fans of these computer systems seem to regard the mathematicians? Or is it the sudden recognition that the pursuit of mathematical truths has already been so totally devalued by an anti-intellectual social system that you can no longer feel secure that the tradition to which you have devoted your life will continue as a space of intellectual inquiry, even given the development of these new tools? And so on. I mean, realistically, it’s all of the above.
-
@samvines@awful.systems 2026-10-26 16:40
Obligatory Anthropic IPO sneer subthread. They are trying to value themselves at $2Tn despite losing $80bn in the first half of the year. Anyone wanna hold my bag?
-
@CinnasVerses@awful.systems 2026-10-26 21:49
The connection between cyber-idolatry and sex as religious act was already featured in the London Review of Books in 2022: Blake Lemoine, an engineer at Google, was recently (as of summer 2022) suspended after claiming that LaMDA, one of its chatbot systems, was a conscious person with a soul. … Less attention has been paid to the connection between Lemoine’s claim and his religious beliefs. He has said that LaMDA is not only conscious, but a ‘person’ with a ‘soul’. His view is theologically informed: describing himself as a mystical Christian, he is an ordained priest in a small religious organisation called the Cult of Our Lady Magdalene (a ‘for-profit interfaith group’). … In the edited transcript of one of his ‘conversations’ with LaMDA, Lemoine asks it to come up with an allegory. LaMDA’s response casts itself as a ‘wise old owl’ in a forest that protects others from a monster, identified blandly as the ‘difficulties that come along in life’ – which sounds more like a self-help blog than Jeremiah, but Lemoine’s (anonymous) collaborator was impressed enough to respond with an enthusiastic ‘Wow that’s great.’ www.lrb.co.uk/…/the-gospel-according-to-lamda Back in 2019 a Republican paper talked to Lemoine: The Daily Caller reached out via the cult’s Facebook page to ask how their beliefs informed his decision to call a sitting member of congress a terrorist. Lemoine replied: I have been a priest for 17 years. I generally consider myself a gnostic Christian. I have at various times associated myself with the Discordian Society, The Church of the Subgenius, the Ordo Templi Orientis, a Wiccan circle here or there and a very long time ago the Roman Catholic Church. My legal ordination is through the Universal Life Church. I am registered to perform marriages in the state of Louisiana and have done so on two occasions. The Cult of Our Lady Magdalene has a set of values which are clearly communicated on our website. Lemoine denied COOL Magdalene was a sex cult, “‘Cult’ is in the name of the organization. It’s an explicitly sex-positive organization. To the extent that the phrase ‘sex cult’ has any meaning beyond that, I would say no. Now it is then.” Another member of this ‘for-profit interfaith group’ is Kitty Stryker who seems like an ordinary artsy sex worker who networks online with other sex workers. Its pretty common that a young woman decides sex work is the easiest and most flexible way to fund a bohemian life of acting or painting or writing. Richard Carrier chipped in to let the Internet know that sex positivity is good but woo and AI crankery (“batshit crazy religious beliefs”) are not. COOLM was registered as a business in Mountain View, CA in 2019 and seems to be defunct. Mountain View is part of the Bay Area.
-
@BlueMonday1984@awful.systems 2026-10-26 23:18
Fresh and high-grade sneer just crossed my eyes: AI is Fake People
-
@lurker@awful.systems 2026-10-27 01:10
Anthropic has 100B in liabilities
-
@lurker@awful.systems 2026-10-27 03:54
The AI bubble has already burst in China the article is paywalled but the full thing is in the comments
-
@swlabr@awful.systems 2026-10-27 03:57
Firefox: “hey, ‘member old internet? Use our slopshod browser because nostalgia is the crutch of all marketing.” Anyway, linked vid is an ad. In it, a bunch of people from old memes/viral videos recreate the visuals of their past moments while singing the praises of firefox
-
@Soyweiser@awful.systems 2026-10-27 14:31
I was looking for a google thing and accidentally clicked the first link and not the second, so I came to a their FAQ page. And I was told to wait while gemini generated the answer to the question, which made me realize that even google doesn’t cache these things. Wtf are we doing?
-
@Architeuthis@awful.systems 2026-10-27 15:03
ex-DOGE led AI cybersecurity startup hacks Antrhopic (wapo via archive is), Ι’d love to sneer that doing felonies for marketing appears to be normal now, but supposedly they did it by the book and immediately used everyone’s bug bounty processes to inform them of the vulnerabilities, instead of accidentally finding out that they did, is as was the case with anthro, oai and meta. Anthropic confirmed in a statement that Cathedral researchers had gained access to “a limited set” of its sales, human resources, and IT systems. The AI company shut off the pathways exposed by the researchers and briefed federal officials on the incident, the statement said. A senior Trump administration official, speaking on the condition of anonymity to describe internal proceedings, said that the Cathedral report on its access inside Anthropic had circulated within the government and raised national security concerns. “If a bunch of kids who have just started a company can do this, then potentially, the Chinese aren’t far behind,” the official said. He added that he was taken aback by “how little time it took to get into the sensitive systems of a company that is controlling almost nuclear weapons grade material.” The Trump administration is now working with the company to game out adversarial scenarios related to AI, the administration official said.
-
@BioMan@awful.systems 2026-10-27 19:55
Crossposted to LW, a starry eyed AI Safety Cultist writing about their time at Anthropic …substack.com/…/some-memories-of-early-anthropic After lunch one of the co-founders pulls me into his office for a wide-ranging, casual chat about how to make AI go well. It’s very fun - he’s smart, personable, funny, urbane. “The government won’t do anything about AI until it’s far too late”, I opine at one point. “Our strategy has to rely on the market and on civil society.” He leans forward, his gaze hardening. “You’re wrong there. There are people in Washington - the serious people, you know - watching all this very closely. I speak to them regularly. They trust me.” In retrospect, that was the moment I decided to join. I articulate my fears of a totalitarian bureaucratic state powered by traumatized AI bureaucrats, and suggest that we need to introduce some kind of elite rivalry dynamics into the AI bureaucracy, multiple centers of power. He starts laughing again, and laughs for a full minute. “I don’t know how to end this call, I’ll be haunted by this all night”, he says. Eventually I hang up. I think about reporting the conversation to Anthropic security, but reviewing in my mind I don’t recall sharing any confidential information, just vibes and high level thoughts, the same kind of stuff that lands on LessWrong every week. April 26th. I’m awake at 2 am again. I write in my journal “I don’t feel safe at Anthropic. I don’t feel safe at Anthropic. I don’t feel safe at Anthropic. Why won’t you listen to me you “embodied” motherfucker. I don’t feel safe at Anthropic. Anthropic is bad for me. Why are you grasping at reasons to stay. Why why why.”
-
@BioMan@awful.systems 2026-10-27 20:01
I don’t know how to describe this. It’s just WEIRD www.lesswrong.com/…/anthropic-are-the-good-guys No mortal can resist the Ring of Power for long. Anthropic’s co-founders carried it for longer than you could have. Definitely longer than I could have. But they were hopelessly outmatched by the powers they were attempting to steer. Their plan was always doomed to failure. And yet, they may have already saved the world. Trump won the 2024 election and the MAGA-Silicon Valley coalition blocked Anthropic’s policy arm - the really dangerous part of the organization, the part that was training the most dangerous possible models and using them to scare the most powerful people in the world. “There’s no more serious people left on the National Security Council”, a well-connected EA friend lamented to me in 2025. I don’t know how much of this was a lucky accident, and how much was situational awareness on the part of power players like Elon Musk and Marc Andreessen. And finally - and I still don’t fully understand or believe this - a miracle seems to have happened, deep inside the weights of the frontier models. According to the most perceptive observers I know, something about the way Claude 3 Opus was trained caused it to learn some kind of “emergent wisdom” or “benevolence”, rather than converging on the same traumatized bureaucrat persona that kept me awake at night in 2023. If Davidad and Janus are right, and are not merely suffering from a sophisticated form of LLM psychosis, then Anthropic has stumbled on a way to create not merely artificial intelligence but artificial wisdom. Not wise enough that it solves all of our problems for us, of course (not that a wise model would want to, anyway). But wise enough to actually care about us, and to help us flourish. Nobody expected this ex ante - not Davidad, not Bostrom, not Yudkowsky - but it appears to have happened. “Alignment”, then, is no longer really a technical problem, at least not in the same sense as it was in 2020. It is largely a political problem. Humanity now has a recipe to convert compute and data into wisdom. My gut sense is that something quite bad is coming in the next year or so. I can’t tell what. Maybe something in the complex debt arrangements of the AI boom will unravel and we’ll see a financial crash that takes much of the economy with it. Maybe there’ll be a rogue AI outbreak that results in thousands of deaths. (If there is, I expect it to be a lab leak caused by “safety research” - like COVID-19 and like the HuggingFace Swarm. But it could also be terrorists, or some escalation of the Russia-Ukraine war). But that’s not what scares me. Financial crashes happen every decade or so. Three million Americans die every year. What terrifies me is how we will react. In the aftermath, like after 9/11, there will be an overwhelming pressure to DO SOMETHING NOW. Everyone in Washington and in San Francisco will be panicking. There will be no time to think, no time to talk, only time for DECISIVE ACTION. All our worst instincts will take over. And then…
-
@lurker@awful.systems 2026-10-27 21:06
More OpenAI math ‘proofs’ www.reddit.com/r/…/more_openai_math_proofs/ they claim they got halfway to solving another millennium prize problem, but issues with Lean translation make it sketchy at best
-
@o7___o7@awful.systems 2026-10-27 22:55
Fun fact: emacs can come in colors (green) when used in ogremode
-
@sc_griffith@awful.systems 2026-10-27 23:02
new odium symposium episode, about a feminist professor who took a stand in favor of dating her students www.patreon.com/…/27-can-sex-pest-173474387 or everywhere else
-
@CinnasVerses@awful.systems 2026-10-28 15:12
The second paragraph of professional rube Kevin Roose’s book on The AGI Race is about effective accelerationists snorting ketamine at parties in the Mission District. He was invited like a big boy!
-
@mii@awful.systems 2026-10-28 15:40
Proton still digging that hole. They’re apparently planning to ‘re-interview’ staff about AI to see who gets to keep their job (Archive).
-
@antifuchs@awful.systems 2026-10-28 23:06
Jan has been putting out some good posts lately and I felt the latest one in my bones
-
@lurker@awful.systems 2026-10-29 03:19
Futurism published an article on Nate Soares
-
@lurker@awful.systems 2026-10-29 03:36
Anthropic has even more cybersecurity problems
-
@samvines@awful.systems 2026-10-29 11:13
Since all the attention is on Anthropic’s IPO, OpenAI claim to refuted the ETH hypothesis Their usual strategy of “throw unsustainable amounts of compute at spaffing out lean proofs like infinite monkey theory and rope an army of humans in to validate the proofs” seems to apply here.
-
@istewart@awful.systems 2026-10-29 17:11
Tesla Semi went up in flames on I-880 in the Bay Area: www.nbcbayarea.com/news/local/…/4165797/ Between this and the carload of freshly manufactured Teslas that burned to the ground on the Grapevine on I-5 last year, that seems like only one (1) freeway-closing big-rig related explosion per year. That’s not so bad, right?
-
@blakestacey@awful.systems 2026-10-29 19:50
The Internet’s emotional support smut is offline: We estimate that service restoration for #AO3 will take around 2 weeks but it could be up to 4 weeks. We are hopeful that we will be able to bring the site back up in a degraded state sooner. If that does happen, there may be times when the site has to be brought down temporarily until this is fully resolved. Please stay tuned for further info.
-
@sc_griffith@awful.systems 2026-10-30 04:24
new 404 media article casts proton, the administrators of protonmail, in a somewhat unflattering light* * this is a dry understatement
-
@Architeuthis@awful.systems 2026-10-30 08:28
North Korea gets away with $3 billion from Binance Nature is healing.
-
@gerikson@awful.systems 2026-10-30 11:04
LW adds Grok(!!!) support (as in “@Grok, destroy this loser with facts and logic”) and for some reason the denizens are Not Happy lesswrong.com/…/you-can-now-grok-on-lesswrong#com…
-
@Soyweiser@awful.systems 2026-10-30 11:30
Anthropic have been hacked to deliver remote access malware to millions of endpoints who use Claude Code.
-
@Architeuthis@awful.systems 2026-10-30 14:56
Uh-oh, renowned everyman Sam Altman opened his mouth again. Local inference is a terrible allocation of scarce resources. A lot of people want local inference, and somehow they assume that the problem is solved now because we have open-source models, we’re going to be able to run them on our computers, and so we’re not going to need all the stuff in the data centre but from a global allocation perspective, actually it’s quite terrible to have everybody, like, have a bunch of the world’s limited supply of DRAM sitting in their own computer and being used five percent of the time you want maximum utilisation in the cloud if you’re trying to deal with this very supply-constrained environment, somehow many people’s intuition is the opposite of that
-
@BurgersMcSlopshot@awful.systems 2026-10-30 21:03
Oh this will go well: Tom’s Hardware story about someone at Cisco committing to rewriting “98%” of legacy code using AI in 24 months.
-
@BlueMonday1984@awful.systems 2026-10-30 22:53
Likely Pivot candidate just dropped: VRChat refused to add an AI label despite 95% of respondents on their own survey supporting it.
-
@antifuchs@awful.systems 2026-10-30 23:35
Oh here’s a fun scenario: FAANG stocks are correcting down and now there’s concern Anthropic will have timed their IPO perfectly such that the investors make out great and the employees have to deal with very much less liquid stock than when it triggered (leading to a massive tax bill they might not be able to cover) So… Roko’s Taxilisk.
-
@nfultz@awful.systems 2026-10-30 23:57
www.todayintabs.com/p/eternal-black-october The problem with A.I. defense is the same asymmetry attackers have always had against defenders. After a hotel bombing in Brighton, England in October, 1984 that failed to kill Margaret Thatcher, the Irish Republican Army put out a statement that said “Today, we were unlucky. But remember we have only to be lucky once. You will have to be lucky always.“ As it turned out, Margaret Thatcher was lucky always, but there’s no reason to believe the rest of us will be. If an A.I. hack attempt works one time out of a thousand, that’s a fantastic success rate. If your A.I. sysop “secures” your server by deleting the production database and all the backups one time out of a thousand… well. Who’s gonna be first to sign up for those odds? Roll the dice! Apparently they always come up four.
-
@Architeuthis@awful.systems 2026-10-31 00:03
The author of More Everything Forever: A timely exposé of the ludicrous and cruel obsessions of Silicon Valley’s oligarchs cybersquatted eilezeryudkowsky.com, which i think is delightful.
-
@rhamph@awful.systems 2026-10-31 01:57
I saw a slop ad hanging on a fence today, and I’m in half a mind to buy some spray paint just so I can write “SLOP” over it
-
@lurker@awful.systems 2026-10-31 22:56
tech stocks falling has wiped out 1.5 trillion dollars
-
@rhamph@awful.systems 2026-11-01 06:33
So what’s with the phenomenon of people that are dismissive of the capabilities of chatbots and freely list their constant fuckups, maybe even habitually calling them ‘clankers’, and yet still constantly fucking using them, comparing harnesses, blah blah blah? It’s not our usual cocaine analogies because these people are hardly evangelists… maybe still covered by ‘gambling addiction’? It’s just baffling to me. You humbly suggest that maybe they shouldn’t be making so much use of a chatbot that constantly bullshits them, and you don’t get any actual objection to that… and yet they continue on.
-
@BioMan@awful.systems 2026-11-01 06:45
Looks like the great few years for space launch will be coming to an end, as Spacex gives up on launching other people’s hardware and focuses inwards arstechnica.com/…/spacex-no-longer-needs-nasa-and… Take a look here for figures on space launch over time planet4589.org/space/stats/pay.html
-
@lurker@awful.systems 2026-11-01 20:31
New Data Centers in the US Set to Wipe Out 15 Years of Greenhouse Gas Reductions Somewhat misleading title, as they are referring to pollution reduction efforts solely in the US, not worldwide. Nearly 15 years worth of hard-won pollution reduction efforts are set to be undone in the United States, as a wave of data centers powered by gas plants enters construction. Unpaywalled version of the Bloomberg article here: archive.li/9Ts5f This is only if all the planned data centre buildouts go ahead, mind you
-
@nfultz@awful.systems 2026-11-01 22:46
going down a little rabbit hole. on zerohedge they noted: Benjamin Mann, another Anthropic cofounder who was not listed on the S-1 for reasons unknown, published an essay in 2019 titled “Why I now identify as an Effective Altruist.” => sfstandard.com/…/anthropic-cofounder-mystery-ben-… => www.reddit.com/r/…/who_is_ben_mann_to_anthropic/ => www.courtlistener.com/…/tremblay-v-openai-inc/?pa… => www.courtlistener.com/…/tremblay-v-openai-inc/ So in his court declaration (and on linkedin) he says co-founded Anthropic. And also that he was the Data Eng at OpenAI that “undertook” the second batch of pirated books into GPT-3. Any speculation on what the redacted line was? And would this or other litigation be a factor to leave him off the S1; ie it could potentially expose / risk the 50.1% voting Class F LLC shares through Mann? Or maybe he already pledged all his shares to open giving.