Post by @tao@mathstodon.xyz

2026-08-27 19:07 UTC

The advent of capable AI tools has highighted a variant of Simpson&#39;s paradox https://en.wikipedia.org/wiki/Simpson%27s_paradox : a technological advance can improve the quality and volume of each individual&#39;s output, and yet the average quality (signal-to-noise ratio) of the aggregate output can deteriorate as a result. I can illustrate this phenomenon with a toy numerical model (all numbers here are made up to simplify the exposition). Let&#39;s take the task of solving a mathematical problem, and then writing it up properly to a professional standard. (Here for simplicity I ignore the intermediate stage of verifying the proof.) Before the advent of an AI, suppose that it took on the order of six months to generate a solution to the problem, and then an additional month to write things up. This was a lot of work, and so relatively few projects would even get as far as the solution generation stage. But, on the other hand, an author who has already invested six months on the problem would usually be willing to invest the additional month to write things up for the added value; few projects would be abandoned between the proof generation and the proof exposition stage. (1/4)

Replies (1)

@tao@mathstodon.xyz 2026-08-27 19:07

Let's say that in a given year, it was possible for a field to generate 100 solutions to various problems, and (by spending the extra month) generate 90 publication-quality writeups of these problems, leaving only 10 solutions that lack such writeups. Then, even if every solved result was announced with only a poorly written description of the result, the signal-to-noise ratio would remain quite high: nine times as many well written solutions as poorly written ones. Only a light amount of curation - such as via a peer-reviewed journal system - would be sufficient for filtering out the 10 results that lack a professional writeup, and obtain a very high quality curated stream of results that form a suitable base for further development of the field. But suppose that advances in AI reduce the problem solving time from six months to a single day, whereas the task of writing things up carefully to professional standards only decreases modestly, from one month to three weeks. So now, one has the choice between spending a day of effort to generate a poorly written solution, or invest a little over three weeks to generate a well written solution. In both cases, the net effort required has gone down, so production volume should go up significantly: for instance, the number of publication-quality writeups might increase from 90 to 500. But in the same time period, the total number of solutions that can be created (with or without a proper writeup) -- which are now far easier to generate still -- might go up from 100 to 5000. (2/4)

Open ##5159028