Post by @wikiemol@mathstodon.xyz

2026-08-30 01:07 UTC

@tao@mathstodon.xyz Is it not the case that the inevitable increase of entropy relative to the training data that went into the AI is a relatively basic result due to the data processing inequality?? I really want an answer to this question and I have no one to ask, except those online, but no one has actually answered it. Now, it feels like you are saying (essentially) the same thing. In other words, just like energy is lost to heat when processed through an engine, an AI processing our repositories of information will lose some of that information to noise. This is only in cases where Tarski's undefinability theorem applies (and its para-complete relative based on the Liar's revenge) such as in the case of an AI attempting to approach 'General Intelligence' (meaning, conservatively, that it can solve any math problem that has a truth value), because only results like that allow you to show that the inequality is a strict one by taking the limit as the AI gets better. If no information was lost, then the AI would approach a [potentially para-]complete truth predicate over time, so the limit will be a self applicable truth relation, which can't exist... (its not merely that its not definable, a self applicable truth relation can't exist). So, if an AI manages to produce 'technically correct' results, there must be residual information contained in our existing repositories of information, e.g. in how things are written or potentially simply what is important (similar to how prefix-free codes can contain 'residual' information), that must be currently being destroyed if the information is not being lost to 'technical incorrectness'.

Replies (1)

@wikiemol@mathstodon.xyz 2026-08-30 01:33

@tao@mathstodon.xyz I want to clarify, although I am less sure about how to formalize this part, I do mean actually destroyed (not merely that the AI is 'wrong' and that wrongness is localized). To explain what I mean, it is analogous to how historians gather information about the ancient past. One might think that the more manuscripts a historian finds that are verified to be from a certain period the more information the historian has about that period. But this isn't necessarily true. Lets say a historian finds a manuscript A and B that tells us something about that period. And lets say for the sake of argument that what A and B says are true (but the historian has no way of proving that). The historian takes as evidence for the truth of A and B that A and B were written independently, and yet agree with each-other. But later, a new manuscript C and D are discovered, and they claim something about the event that contradicts something in B, and agree with each-other. In reality, A and B are true, but C and D contain untrue rumors that spread about the event. In this instance, the historian has more information than before, but is less certain about the truth of A and B, since C and D undermined the validity of B. So, the discovery of C and D destroyed some information the historian previously had about the event, since it made the historian more uncertain. Similarly, when an AI adds a widespread amount of noise, it becomes less and less clear what is true/important/valuable etc, destroying previous repositories of information.

Open ##5159044