JuliusBrussee/caveman · error

cachebench: replay token summary overflow

Error message

cachebench: replay token summary overflow

What it means

While accumulating provider-reported token totals, SummarizeReplayEvidence checks each addition against math.MaxInt64 for both the global and per-provider sums. If any record's input or output token counts would overflow an int64 aggregate, it errors instead of silently wrapping to a negative total.

Source

Thrown at cacheengine/cachebench/replay.go:295

			group.summary.Successful++
			if record.QualityPassed {
				summary.QualityPassed++
				group.summary.QualityPassed++
			}
		} else {
			summary.Failed++
			group.summary.Failed++
		}
		if !record.TimingFaithful {
			summary.TimingFaithful = false
		}
		if record.TokenBasis != TokenProviderCounted {
			summary.InputBudgetClaimedProviderCounted = false
		}
		if record.ProviderUsageSHA256 != "" {
			input, output := int64(record.ProviderTotalInputTokens), int64(record.ProviderOutputTokens)
			if input > math.MaxInt64-summary.InputTokens || output > math.MaxInt64-summary.OutputTokens || input > math.MaxInt64-group.summary.InputTokens || output > math.MaxInt64-group.summary.OutputTokens {
				return ReplayEvidenceSummary{}, errors.New("cachebench: replay token summary overflow")
			}
			summary.InputTokens += input
			summary.OutputTokens += output
			group.summary.InputTokens += input
			group.summary.OutputTokens += output
		}
		if record.HTTPStatus > 0 {
			latencies = append(latencies, record.LatencyMilliseconds)
			group.latencies = append(group.latencies, record.LatencyMilliseconds)
		}
	}
	summary.Latency = summarizeLatency(latencies)
	providers := make([]string, 0, len(groups))
	for provider := range groups {
		providers = append(providers, provider)
	}
	sort.Strings(providers)
	for _, provider := range providers {

View on GitHub (pinned to 27d5a3981a)

Solutions

  1. Sanitize evidence records: reject any single record with ProviderTotalInputTokens or ProviderOutputTokens above a sane ceiling (e.g. 1e12) before summarizing
  2. Fix field mapping in your evidence importer if the wrong source column feeds token counts
  3. For synthetic tests, bound generated token values to realistic magnitudes

Example fix

// before
summary, err := cachebench.SummarizeReplayEvidence(records) // one record has 9e18 input tokens

// after
for _, r := range records {
    if r.ProviderTotalInputTokens > 1_000_000_000_000 || r.ProviderOutputTokens > 1_000_000_000_000 {
        return fmt.Errorf("implausible token count in request %s; check evidence source", r.RequestID)
    }
}
summary, err := cachebench.SummarizeReplayEvidence(records)
Defensive patterns

Strategy: validation

Validate before calling

const maxPlausibleTokens = int64(1) << 40 // ~1.1T, far above any real request
for _, r := range records {
    if r.ProviderTotalInputTokens > maxPlausibleTokens || r.ProviderOutputTokens > maxPlausibleTokens || r.ProviderTotalInputTokens < 0 || r.ProviderOutputTokens < 0 {
        return fmt.Errorf("implausible token counts in request %s; evidence likely mis-mapped", r.RequestID)
    }
}
summary, err := cachebench.SummarizeReplayEvidence(records)

Type guard

func plausibleTokenCounts(r ReplayEvidenceRecord) bool {
    return r.ProviderTotalInputTokens >= 0 && r.ProviderOutputTokens >= 0 &&
        r.ProviderTotalInputTokens <= 1<<40 && r.ProviderOutputTokens <= 1<<40
}

Try / catch

if _, err := cachebench.SummarizeReplayEvidence(records); err != nil {
    if err.Error() == "cachebench: replay token summary overflow" {
        // find the record with absurd token counts and fix the ingestion mapping
    }
}

Prevention

When it happens

Trigger: Replay evidence containing corrupt or adversarial token counts (e.g. ProviderTotalInputTokens near 2^63) that, when summed across records, exceed int64 range. Real usage never approaches this, so it indicates bad data in the records.

Common situations: Observation/evidence ingestion mapped the wrong field into token counts (timestamps or byte counts read as tokens); a provider bug or unit mismatch (tokens reported in micro-units); fuzz/synthetic evidence files with random int64 values.

Related errors

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/d21cf69d1ea5b026. Report an issue: GitHub.