2026-09-26 18:59 UTC
Replies (3)
-
@eevee@slime.global 2026-09-26 19:05
@zzzeek@fosstodon.org @glyph@mastodon.social @hynek@mastodon.social programming is a special case: you are designing an algorithm, and the resulting artifact is an encoding of that algorithm, at which point you can try it to see if it's correct. (of course it has its own blind spots because computers are hard.) but most tasks aren't like this. most tasks are the *execution* of an algorithm. how do i verify if a chatbot correctly added up some numbers? well i have to add up the numbers myself and see if the results match
-
@glyph@mastodon.social 2026-09-26 19:01
@eevee@slime.global @zzzeek@fosstodon.org @hynek@mastodon.social if I could put a finer point on this, that's not exactly what I'm saying, although the functional output of what I'm saying is the same. Specifically, verification can be cheaper than performing the task sometimes, but the aggregate of A) doing the verification *every* time (you can't skip any instances!) and B) _investigating failed verifications_, re-doing the work in those cases, and C) maintenance of the verification loop itself is indeed *more* work than the work.
-
@glyph@mastodon.social 2026-09-26 19:05
@eevee@slime.global @zzzeek@fosstodon.org @hynek@mastodon.social I hasten to add to this particular point: this is a logical issue, a very basic fact of information theory, and requires no testing of LLMs. when we allow a human to summarize text for us we are not actually expecting a human to do a computational task. what we are doing is establishing a *trust relationship* that we trust their judgement in contextually appropriately removing information that is irrelevant to us based on what they know about our needs