Post by @tao@mathstodon.xyz

2026-08-23 18:30 UTC

As an experiment, I directed a coding agent to sort through recent #arXiv posts for similarity in topic or keywords to my own research papers, with additional weighting for papers authored by my former collaborators or mentees (such papers are marked with a star in the lists provided below). I also asked the agent to assess the degree of AI assistance in each paper using robot emojis, with one emoji denoting minor assistance, two denoting major assistance, and three denoting near-total automation. (A computer emoji is also used to indicate more traditional computer assistance, e.g., in numerics.) The lists produced by the agent for the months of July and of August respectively are provided below. (It is important to stress that the AI-generated rankings here are based on proximity to my own interests, and should not be regarded as an absolute ranking of importance of the result.) The generation of these listings is highly unscientific and tailored to my own personal preferences (and there was at least one annotation error, in that the author Van Khu Vu was confused with my collaborator Van Ha Vu), but it does indicate to me the increasing adoption of AI assistance, at least in my own fields of interest.

Replies (5)

@eudaemon@mathstodon.xyz 2026-08-23 19:57

@tao@mathstodon.xyz another interesting experiment would be to do the same thing using something like a tf-idf algorithm, and compare the outcome with the AI's one. It would be a nice test of the AI's capabilities at measuring the importance of a word in a document, since tf-idf is (or at least initially was) the core algorithm used in search engines.

Open ##5163052
@manas@nerdculture.de 2026-08-24 16:56

@tao@mathstodon.xyz Why is your primary Mastodon and not Bluesky, any technical reason?

Open ##5163053
@andrewallen@mathstodon.xyz 2026-08-24 23:35

@tao@mathstodon.xyz I don't understand why we don't hear more about serious efforts to use OpenAI-scale compute to essentially brute-force valid LEAN proofs, then feed them into generative AI model training.

Open ##5163054
@human_in_exile@mathstodon.xyz 2026-08-25 02:36

@tao@mathstodon.xyz No worries, as an experiment I will now have to figure out what to do with this, and if it was said simply for the love of ones own voice, or actually to be helpful. The experiment commences!

Open ##5163055
@DispersiveArithmetic@mathstodon.xyz 2026-08-25 16:53

@tao@mathstodon.xyz For what it's worth, my paper listed there was correctly categorized by the agent: no assistance was used. I did not include any AI statement, so I'm not sure if the agent just assumed there was no assistance or if it made inferences based on language used.

Open ##5163056