Post by @cm0002@infosec.pub

Mika

@Mika@piefed.ca · Feb 25

https://infosec.pub/api/v3/image_proxy?url=https%3A%2F%2Fmedia.piefed.ca%2Fposts%2Fw4%2Fo2%2Fw4o2WpaaMA6GC8C.png

561

10

0

Hackworth

@Hackworth@piefed.ca · Feb 25

Here's the Anthropic post.

Detecting and preventing distillation attacks

We identified coordinated distillation attacks using ~24,000 fraudulent accounts to extract Claude's capabilities.

https://www.anthropic.com/news/detecting-and-preventing-distillation-attacks

561

10

0

Tar_Alcaran

@Tar_alcaran@sh.itjust.works · Feb 25

Also pictured here: Anthropic stating out loud their models will just give out all the "secret" and "secured" internal data to anyone who asks. Of course, that's by design. LLMs can't have any barrier between data and instructions, so they can never be secure.

60

2

0

Hackworth

@Hackworth@piefed.ca · Feb 25

Thread reply

Distillation is using one model to train another. It's not really about leaking data. >Claude was used to generate censorship-safe alternatives to politically sensitive queries like questions about dissidents, party leaders, or authoritarianism, likely in order to train DeepSeek’s own models to steer conversations away from censored topics But you're right, prompt injection/jailbreaking is still trivial too.

561

10

0

degenerate_neutron_matter

@degenerate_neutron_matter@fedia.io · Feb 25

The AI companies are inbreeding intentionally now? Wonderful!

24

0

mindbleach

@mindbleach@sh.itjust.works · Feb 25

In undue fairness, there *is* a difference between turning text files into a chatbot, and exfiltrating that chatbot. One is transformative, and the other is making a megaphone out of some string, a squirrel, and a megaphone. But if I don't give a shit about companies doing math on Disney DVDs I'm not about to give a shit about them hoarding their big pile of numbers. I'm jazzed when source code leaks for things written by *people.*

10

0

0_o7

@0_o7@lemmy.dbzer0.com · Feb 26

When they steal: Innovative approach to knowledge acquisition When others steal: A threat to free market by IP violation

8

0

Mika

@Mika@piefed.ca · Feb 25

1

0

Conversation (8)