{"record":{"id":"64c2d31821ecd040","repo":"JuliusBrussee/caveman","slug":"cave-memory-too-large","errorCode":"cave_memory_too_large","errorMessage":"cave_memory_too_large","messagePattern":"cave_memory_too_large","errorType":"error_code","errorClass":"ErrMemoryTooLarge","httpStatus":null,"severity":"error","filePath":"mem/store.go","lineNumber":71,"sourceCode":"// across all its hits. Without it, recall loaded and returned every matching\n// memory whole — a single 2.5 MB memory came back as tokens_added=440,000 in one\n// result. Callers override it per-call through RecallOptions.TokenBudget.\nconst DefaultTokenBudget = 2000\n\n// UnlimitedTokenBudget disables Recall's aggregate token cap. It is explicit:\n// zero retains the safe DefaultTokenBudget for existing Go callers. Public\n// adapters map their documented token_budget=0 sentinel to this value.\nconst UnlimitedTokenBudget = -1\n\n// MaxMemoryBytes is the largest single memory Remember accepts. A memory is a\n// fact or note meant to be recalled and injected, not a file dump; anything\n// larger fails closed with ErrMemoryTooLarge rather than being stored and later\n// blowing the recall budget.\nconst MaxMemoryBytes = 256 * 1024\n\n// ErrMemoryTooLarge is returned when new memory text exceeds MaxMemoryBytes.\n// It carries the cave_ error idiom so callers surface cave_memory_too_large.\nvar ErrMemoryTooLarge = errors.New(\"cave_memory_too_large\")\n\n// tokenCounter is the shared offline BPE estimator (o200k_base). It is read-only\n// and safe for concurrent use, matching the engine's inferred token accounting.\nvar tokenCounter = tokens.Default()\n\n// Store is a cavemem instance: a SQLite memory table plus an engine (with its\n// own CCR store) used to compress recalls.\ntype Store struct {\n\tdb  *sql.DB\n\teng *engine.Engine\n\tccr *ccr.Store\n}\n\n// Options configures Open.\ntype Options struct {\n\t// Dir is the data directory; default ~/.caveman/mem (honoring CAVEMAN_HOME).\n\tDir string\n\t// InMemory uses an ephemeral database for both memories and CCR (tests).","sourceCodeStart":53,"sourceCodeEnd":89,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/27d5a3981a347890211bb1bf2439e5c821a63bc9/mem/store.go#L53-L89","documentation":"cavemem's Store.Remember refuses any single memory whose text exceeds MaxMemoryBytes (256 KiB). Memories are facts/notes meant for recall and injection, not file dumps; oversized input fails closed with ErrMemoryTooLarge (message doubles as the cave_ error idiom) so it can never blow the recall token budget later.","triggerScenarios":"Calling Remember with a text longer than 262144 bytes: pasted file contents, huge stack traces, whole documents, or concatenated logs.","commonSituations":"Agent pipeline auto-storing command output or file reads as memories without size checks; users pasting large blobs; a bug that passes a whole context window dump into Remember.","solutions":["Store a short distilled fact/note instead of the raw blob; summarize before Remember","If the content is a file, keep it out of cavemem and reference it by path","Pre-check len([]byte(text)) > mem.MaxMemoryBytes and split or truncate deliberately at your boundary"],"exampleFix":"// before\nerr := store.Remember(ctx, sessionID, entireFileContents)\n\n// after\nif len(text) > mem.MaxMemoryBytes {\n    text = summarize(entireFileContents) // or store a pointer to the file\n}\nerr := store.Remember(ctx, sessionID, text)","handlingStrategy":"validation","validationCode":"if len(text) > mem.MaxMemoryBytes {\n    text = truncateOrSummarize(text)\n}\nerr := store.Remember(ctx, sessionID, text)","typeGuard":"func memoryFits(text string) bool { return len(text) <= mem.MaxMemoryBytes }","tryCatchPattern":"if err := store.Remember(ctx, sessionID, text); err != nil {\n    if errors.Is(err, mem.ErrMemoryTooLarge) {\n        // summarize and retry once, or decline to store\n    }\n}","preventionTips":["Never pass raw files/command output to Remember; distill first","Enforce a size check at ingestion so oversized blobs never reach the store"],"tags":["mem","go","size-limit","validation","memory-store"],"backgroundTag":null,"analyzedSha":"27d5a3981a347890211bb1bf2439e5c821a63bc9","analyzedAt":"2026-08-15T09:26:11.751Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}