{"record":{"id":"e7ef2d1c6e45b18b","repo":"JuliusBrussee/caveman","slug":"store-recovery-for-oversized-memory-s-w","errorCode":null,"errorMessage":"store recovery for oversized memory %s: %w","messagePattern":"store recovery for oversized memory (.+?): %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"mem/store.go","lineNumber":554,"sourceCode":"\n// headHit builds a single recall hit for a memory whose compressed form exceeds\n// the entire token budget: it truncates the compressed text to a head that fits\n// and guarantees a CCR handle to the byte-exact original. When the engine passed\n// the payload through without compressing (so it left no handle), the original\n// is stored here — a head drops the tail, so the dropped detail must stay\n// recoverable, upholding cavemem's reversible invariant.\nfunc (s *Store) headHit(m Memory, compressedText []byte, handle string, score float64, budget int) (Hit, error) {\n\thead := truncateToTokens(compressedText, budget)\n\tif handle == \"\" {\n\t\th, err := s.ccr.Put(ccr.Recovery{\n\t\t\tContentType:  \"text\",\n\t\t\tCompressor:   \"cavemem-head\",\n\t\t\tTokensBefore: tokenCounter.Count([]byte(m.Text)),\n\t\t\tTokensAfter:  tokenCounter.Count(head),\n\t\t\tOriginal:     []byte(m.Text),\n\t\t})\n\t\tif err != nil {\n\t\t\treturn Hit{}, fmt.Errorf(\"store recovery for oversized memory %s: %w\", m.ID, err)\n\t\t}\n\t\thandle = h\n\t}\n\treturn Hit{\n\t\tID:             m.ID,\n\t\tText:           string(head),\n\t\tScore:          score,\n\t\tTokensAdded:    tokenCounter.Count(head),\n\t\tBasis:          engine.BasisInferred,\n\t\tRecoveryHandle: handle,\n\t}, nil\n}\n\n// truncateToTokens returns the longest byte prefix of b whose inferred token\n// count does not exceed budget, ending on a UTF-8 rune boundary. Recovery of the\n// dropped tail is byte-exact through the hit's CCR handle.\nfunc truncateToTokens(b []byte, budget int) []byte {\n\tif budget <= 0 {","sourceCodeStart":536,"sourceCodeEnd":572,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/3ee70a102609e550bd2e68004bf5990a9341c851/mem/store.go#L536-L572","documentation":"When a single recalled memory exceeds the token budget on its own, Recall falls back to a head-truncated 'cavemem-head' representation and must persist recovery metadata (the Hit with compressed text and a handle) into the store. If that storage write fails, the error is wrapped as \"store recovery for oversized memory %s\", naming the offending memory ID. Recall fails rather than silently returning an unrecoverable hit.","triggerScenarios":"Recall returns a hit whose single memory's TokensBefore exceeds the budget, and the subsequent store write of the recovery Hit record fails (DB closed, locked, corrupted, disk full).","commonSituations":"Recalling a very large memory with a small TokenBudget while the SQLite file is read-only or the database handle was closed; also seen when a concurrent writer holds the write lock.","solutions":["Inspect the wrapped %w error for the underlying SQLite failure.","Ensure the store is open and writable (not closed, file not read-only) when calling Recall.","Resolve DB lock contention from concurrent writers before recalling.","Preemptively supersede or delete the oversized memory so the recovery path is not needed."],"exampleFix":"// before\nhits, err := store.Recall(ctx, q, mem.RecallOptions{TokenBudget: 100})\n// after (verify store health and check wrapped error)\nif err != nil {\n    var id string\n    if strings.Contains(err.Error(), \"store recovery for oversized memory\") {\n        // extract ID, inspect/repair that memory, then retry\n    }\n    return err\n}\nhits, err := store.Recall(ctx, q, mem.RecallOptions{TokenBudget: 100})","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"hits, err := store.Recall(ctx, q, opts)\nif err != nil && strings.Contains(err.Error(), \"store recovery for oversized memory\") {\n    // DB write failed during head-truncation; check store health and retry\n}","preventionTips":["Keep the store writable while Recall runs (don't Close early).","Supersede very large memories proactively so the recovery path isn't hit.","Set a TokenBudget large enough that common hits aren't oversized."],"tags":["database","sqlite","wrapped-error","token-budget"],"backgroundTag":"database-write-failed","analyzedSha":"3ee70a102609e550bd2e68004bf5990a9341c851","analyzedAt":"2026-09-20T15:53:39.229Z","contentChangedAt":"2026-09-20T15:53:39.229Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}