Catastrophic failures are rare with frontier models now. They're becoming very reliable. People will get used to this and take fewer precautions. But I'd argue that, as these systems become ubiquitous and more powerful, we should actually take more. This morning, Claude happily deleted all the SSH keys on one machine.