2026-10-07 02:12 UTC
I’m not techno babbling you. I specifically said “if you consider the model + kv cache together,” e.g preserved. A kv cache is not an internal optimization, a model cannot generate text without a kv cache. There’s a separate idea of caching the kv cache between requests as an optimization. That’s not the one I’m talking about.
Or if it helps, consider it just within one generation. The model + kv cache. Does that not match your definition?
Replies (1)
-
@midribbon_action@lemmy.blahaj.zone 2026-10-07 02:21
Definition of what? Do I think that because llms use a kv cache during inference, that proves they can experience things? You’re gonna have to explain your thought process. To me, the particular algorithm used is not the philosophical crux of the issue. There’s a separate idea of caching the kv cache between requests I’ve never heard of this anyways. The kv cache is specific to a particular input, it can’t be reused for a subsequent prompts.