JuliusBrussee/caveman · error · ErrMemoryTooLarge

cave_memory_too_large

cave_memory_too_large

Error message

cave_memory_too_large

What it means

cavemem's Store.Remember refuses any single memory whose text exceeds MaxMemoryBytes (256 KiB). Memories are facts/notes meant for recall and injection, not file dumps; oversized input fails closed with ErrMemoryTooLarge (message doubles as the cave_ error idiom) so it can never blow the recall token budget later.

Source

Thrown at mem/store.go:71

// across all its hits. Without it, recall loaded and returned every matching
// memory whole — a single 2.5 MB memory came back as tokens_added=440,000 in one
// result. Callers override it per-call through RecallOptions.TokenBudget.
const DefaultTokenBudget = 2000

// UnlimitedTokenBudget disables Recall's aggregate token cap. It is explicit:
// zero retains the safe DefaultTokenBudget for existing Go callers. Public
// adapters map their documented token_budget=0 sentinel to this value.
const UnlimitedTokenBudget = -1

// MaxMemoryBytes is the largest single memory Remember accepts. A memory is a
// fact or note meant to be recalled and injected, not a file dump; anything
// larger fails closed with ErrMemoryTooLarge rather than being stored and later
// blowing the recall budget.
const MaxMemoryBytes = 256 * 1024

// ErrMemoryTooLarge is returned when new memory text exceeds MaxMemoryBytes.
// It carries the cave_ error idiom so callers surface cave_memory_too_large.
var ErrMemoryTooLarge = errors.New("cave_memory_too_large")

// tokenCounter is the shared offline BPE estimator (o200k_base). It is read-only
// and safe for concurrent use, matching the engine's inferred token accounting.
var tokenCounter = tokens.Default()

// Store is a cavemem instance: a SQLite memory table plus an engine (with its
// own CCR store) used to compress recalls.
type Store struct {
	db  *sql.DB
	eng *engine.Engine
	ccr *ccr.Store
}

// Options configures Open.
type Options struct {
	// Dir is the data directory; default ~/.caveman/mem (honoring CAVEMAN_HOME).
	Dir string
	// InMemory uses an ephemeral database for both memories and CCR (tests).

View on GitHub (pinned to 27d5a3981a)

Solutions

1. Store a short distilled fact/note instead of the raw blob; summarize before Remember
2. If the content is a file, keep it out of cavemem and reference it by path
3. Pre-check len([]byte(text)) > mem.MaxMemoryBytes and split or truncate deliberately at your boundary

Example fix

// before
err := store.Remember(ctx, sessionID, entireFileContents)

// after
if len(text) > mem.MaxMemoryBytes {
    text = summarize(entireFileContents) // or store a pointer to the file
}
err := store.Remember(ctx, sessionID, text)

if len(text) > mem.MaxMemoryBytes {
    text = truncateOrSummarize(text)
}
err := store.Remember(ctx, sessionID, text)

func memoryFits(text string) bool { return len(text) <= mem.MaxMemoryBytes }

if err := store.Remember(ctx, sessionID, text); err != nil {
    if errors.Is(err, mem.ErrMemoryTooLarge) {
        // summarize and retry once, or decline to store
    }
}

When it happens

Trigger: Calling Remember with a text longer than 262144 bytes: pasted file contents, huge stack traces, whole documents, or concatenated logs.

Common situations: Agent pipeline auto-storing command output or file reads as memories without size checks; users pasting large blobs; a bug that passes a whole context window dump into Remember.

Related errors

• ErrMemoryTooLarge
• history requires memory id
• File too large to compress safely (max 500KB): {filepath}
• cachebench: nil corpus reader
• message exceeds byte limit

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/64c2d31821ecd040. Report an issue: GitHub.