{"record":{"id":"d21cf69d1ea5b026","repo":"JuliusBrussee/caveman","slug":"cachebench-replay-token-summary-overflow","errorCode":null,"errorMessage":"cachebench: replay token summary overflow","messagePattern":"cachebench: replay token summary overflow","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"cacheengine/cachebench/replay.go","lineNumber":295,"sourceCode":"\t\t\tgroup.summary.Successful++\n\t\t\tif record.QualityPassed {\n\t\t\t\tsummary.QualityPassed++\n\t\t\t\tgroup.summary.QualityPassed++\n\t\t\t}\n\t\t} else {\n\t\t\tsummary.Failed++\n\t\t\tgroup.summary.Failed++\n\t\t}\n\t\tif !record.TimingFaithful {\n\t\t\tsummary.TimingFaithful = false\n\t\t}\n\t\tif record.TokenBasis != TokenProviderCounted {\n\t\t\tsummary.InputBudgetClaimedProviderCounted = false\n\t\t}\n\t\tif record.ProviderUsageSHA256 != \"\" {\n\t\t\tinput, output := int64(record.ProviderTotalInputTokens), int64(record.ProviderOutputTokens)\n\t\t\tif input > math.MaxInt64-summary.InputTokens || output > math.MaxInt64-summary.OutputTokens || input > math.MaxInt64-group.summary.InputTokens || output > math.MaxInt64-group.summary.OutputTokens {\n\t\t\t\treturn ReplayEvidenceSummary{}, errors.New(\"cachebench: replay token summary overflow\")\n\t\t\t}\n\t\t\tsummary.InputTokens += input\n\t\t\tsummary.OutputTokens += output\n\t\t\tgroup.summary.InputTokens += input\n\t\t\tgroup.summary.OutputTokens += output\n\t\t}\n\t\tif record.HTTPStatus > 0 {\n\t\t\tlatencies = append(latencies, record.LatencyMilliseconds)\n\t\t\tgroup.latencies = append(group.latencies, record.LatencyMilliseconds)\n\t\t}\n\t}\n\tsummary.Latency = summarizeLatency(latencies)\n\tproviders := make([]string, 0, len(groups))\n\tfor provider := range groups {\n\t\tproviders = append(providers, provider)\n\t}\n\tsort.Strings(providers)\n\tfor _, provider := range providers {","sourceCodeStart":277,"sourceCodeEnd":313,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/27d5a3981a347890211bb1bf2439e5c821a63bc9/cacheengine/cachebench/replay.go#L277-L313","documentation":"While accumulating provider-reported token totals, SummarizeReplayEvidence checks each addition against math.MaxInt64 for both the global and per-provider sums. If any record's input or output token counts would overflow an int64 aggregate, it errors instead of silently wrapping to a negative total.","triggerScenarios":"Replay evidence containing corrupt or adversarial token counts (e.g. ProviderTotalInputTokens near 2^63) that, when summed across records, exceed int64 range. Real usage never approaches this, so it indicates bad data in the records.","commonSituations":"Observation/evidence ingestion mapped the wrong field into token counts (timestamps or byte counts read as tokens); a provider bug or unit mismatch (tokens reported in micro-units); fuzz/synthetic evidence files with random int64 values.","solutions":["Sanitize evidence records: reject any single record with ProviderTotalInputTokens or ProviderOutputTokens above a sane ceiling (e.g. 1e12) before summarizing","Fix field mapping in your evidence importer if the wrong source column feeds token counts","For synthetic tests, bound generated token values to realistic magnitudes"],"exampleFix":"// before\nsummary, err := cachebench.SummarizeReplayEvidence(records) // one record has 9e18 input tokens\n\n// after\nfor _, r := range records {\n    if r.ProviderTotalInputTokens > 1_000_000_000_000 || r.ProviderOutputTokens > 1_000_000_000_000 {\n        return fmt.Errorf(\"implausible token count in request %s; check evidence source\", r.RequestID)\n    }\n}\nsummary, err := cachebench.SummarizeReplayEvidence(records)","handlingStrategy":"validation","validationCode":"const maxPlausibleTokens = int64(1) << 40 // ~1.1T, far above any real request\nfor _, r := range records {\n    if r.ProviderTotalInputTokens > maxPlausibleTokens || r.ProviderOutputTokens > maxPlausibleTokens || r.ProviderTotalInputTokens < 0 || r.ProviderOutputTokens < 0 {\n        return fmt.Errorf(\"implausible token counts in request %s; evidence likely mis-mapped\", r.RequestID)\n    }\n}\nsummary, err := cachebench.SummarizeReplayEvidence(records)","typeGuard":"func plausibleTokenCounts(r ReplayEvidenceRecord) bool {\n    return r.ProviderTotalInputTokens >= 0 && r.ProviderOutputTokens >= 0 &&\n        r.ProviderTotalInputTokens <= 1<<40 && r.ProviderOutputTokens <= 1<<40\n}","tryCatchPattern":"if _, err := cachebench.SummarizeReplayEvidence(records); err != nil {\n    if err.Error() == \"cachebench: replay token summary overflow\" {\n        // find the record with absurd token counts and fix the ingestion mapping\n    }\n}","preventionTips":["Bound per-record token counts at evidence ingestion with a plausibility ceiling","Verify token fields map to provider usage counters, not timestamps or byte sizes","In synthetic/fuzz evidence, generate token values in realistic ranges"],"tags":["go","replay","overflow","data-hygiene"],"backgroundTag":null,"analyzedSha":"27d5a3981a347890211bb1bf2439e5c821a63bc9","analyzedAt":"2026-08-15T09:26:11.751Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}