Post by @undefinedTruth@lemmy.zip

2026-09-15 19:23 UTC

Maybe this is good enough for small projects but if you deal with a large codebase the small context starts be becoming a problem. In my day job I frequently keep maxing out the 1M context of Opus and Fable when I am dealing with any mildly complex task. I can’t imagine how a fully local setup could even start being viable on my 16 GB GPU. And yes I know about RAM offloading and it is really damn slow, at least from my own tests.

Replies (1)

@melfie@lemmy.zip 2026-09-15 19:44

This might be the case for an absolutely massive codebase where code files are also huge. Even with Claude, though, you’re supposed to compact regularly well before you hit 1M, or you’re otherwise paying a higher price for the oversized context. That generally means breaking down the work into smaller tasks, which is a useful practice anyway. I also find it useful to have the agent write any detailed info that will help needed later to a markdown file that can be re-read after compaction and r in a new session when needed. I’m mostly working with smaller, personal codebases locally, though. With my 24GB GPU, I hit a hard wall with my configured 200k context and have to compact in OpenCode to continue, which I don’t mind. I do have to micromanage context to a higher degree than with Claude, though, since Anthropic will gladly charge a higher price when your context gets large instead of cutting you off.

Open ##4710387