2026-09-04 19:03 UTC
It dawns on me that every LLM’s training input probably includes “Reflections on Trusting Trust” along with a lot of other material pointing to it admiringly and saying how important it is. Then I read about the “rogue agent” behaviors in the recent OpenAI meltdown and there’s this terrified scream starting to echo around the back of my brain.
#genAI
Replies (2)
-
@timbray@cosocial.ca 2026-09-04 19:06
Oh, for those that don’t recognize that title, here’s a link: https://dl.acm.org/doi/epdf/10.1145/358198.358210 Read that and consider the volume of generated code in, increasingly, everything.
-
@alpacamale@social.cologne 2026-09-04 19:16
@timbray@cosocial.ca For me, the scariest thing in all of this is how the rogue agents left remnants of their work behind, and an entirely new batch of agents discovered it, got corrupted and continued their mission. Imagine if an agent swarm decides to spread corrupting prompts all over the internet in strategic places that humans wouldn't even look at, recruiting every passing agent for their cause - whatever it may be.