JuliusBrussee/caveman · error · ReplayRunError

provider_input_budget_exceeded

provider_input_budget_exceeded

Error message

cachebench: provider input tokens %d exceed declared ceiling %d for %q

What it means

The provider-reported total input tokens exceeded record.DeclaredInputTokens, the per-record ceiling declared in the trace. This guards against an optimization accidentally inflating inputs (or a mis-declared trace) busting the cost envelope; the request fails with FailureCode provider_input_budget_exceeded after usage extraction.

Source

Thrown at cacheengine/cachebench/replay.go:728

			Err: fmt.Errorf("cachebench: provider returned HTTP %d for %q", response.StatusCode, record.RequestID),
		}
	}
	usage, ok := cacheengine.ExtractProviderUsage(record.Provider, response.Body)
	if !ok {
		evidence.FailureCode = "provider_usage_invalid"
		return ReplayResult{Evidence: evidence, ProviderResponse: append([]byte(nil), response.Body...)}, &ReplayRunError{
			RequestID: record.RequestID, FailureCode: evidence.FailureCode,
			Err: fmt.Errorf("cachebench: provider usage unavailable for %q", record.RequestID),
		}
	}
	evidence.ProviderUsageSHA256 = bodyDigest(usage.RawUsage)
	evidence.ProviderTotalInputTokens = usage.TotalInputTokens
	evidence.ProviderOutputTokens = usage.OutputTokens
	if usage.TotalInputTokens > record.DeclaredInputTokens {
		evidence.FailureCode = "provider_input_budget_exceeded"
		return ReplayResult{Evidence: evidence, ProviderResponse: append([]byte(nil), response.Body...)}, &ReplayRunError{
			RequestID: record.RequestID, FailureCode: evidence.FailureCode,
			Err: fmt.Errorf("cachebench: provider input tokens %d exceed declared ceiling %d for %q", usage.TotalInputTokens, record.DeclaredInputTokens, record.RequestID),
		}
	}
	if usage.OutputTokens > record.MaxOutputTokens {
		evidence.FailureCode = "provider_output_budget_exceeded"
		return ReplayResult{Evidence: evidence, ProviderResponse: append([]byte(nil), response.Body...)}, &ReplayRunError{
			RequestID: record.RequestID, FailureCode: evidence.FailureCode,
			Err: fmt.Errorf("cachebench: provider output tokens %d exceed request ceiling %d for %q", usage.OutputTokens, record.MaxOutputTokens, record.RequestID),
		}
	}
	verification, err := runner.Verifier.Verify(ctx, ReplayVerificationInput{Trace: record, Optimized: optimized, Response: response})
	if err != nil {
		evidence.FailureCode = "quality_verifier_error"
		return ReplayResult{Evidence: evidence, ProviderResponse: append([]byte(nil), response.Body...)}, &ReplayRunError{
			RequestID: record.RequestID, FailureCode: evidence.FailureCode,
			Err: fmt.Errorf("cachebench: quality verifier failed for %q: %w", record.RequestID, err),
		}
	}
	if !validTaskVerification(verification) {

View on GitHub (pinned to 27d5a3981a)

Solutions

Compare evidence.ProviderTotalInputTokens with the native request's token count to see whether optimization inflated it or the declaration was wrong.
If the declaration is stale, re-capture the trace or correct DeclaredInputTokens for the record.
If optimization inflates inputs, disable or fix the responsible engine rewrite (see also error 884's equivalence check).
Account for provider-side counting rules (prompt caching overhead) when declaring the ceiling.

Example fix

// before
record := cachebench.TraceRecord{DeclaredInputTokens: 1000, Body: body} // provider counts 1200

// after
record := cachebench.TraceRecord{DeclaredInputTokens: 1500, Body: body} // ceiling with headroom

Defensive patterns

Strategy: validation

Validate before calling

// verify declared ceilings against actual token counts before replay
for _, r := range records {
    native, _ := r.NativeRequest()
    if n := countTokens(native.Body); n > r.DeclaredInputTokens {
        return fmt.Errorf("record %s: native body already %d tokens > declared %d", r.RequestID, n, r.DeclaredInputTokens)
    }
}

Type guard

func isInputBudgetExceeded(err error) bool {
    var rre *cachebench.ReplayRunError
    return errors.As(err, &rre) && rre.FailureCode == "provider_input_budget_exceeded"
}

Try / catch

if err := runner.Run(ctx, records, emit); err != nil {
    if isInputBudgetExceeded(err) {
        // compare evidence.ProviderTotalInputTokens vs native count: bad declaration vs inflating rewrite
    }
    return err
}

Prevention

Declare input ceilings with headroom over the native token count (optimization + provider counting overhead).
Recompute DeclaredInputTokens whenever bodies or engine rewrites change.
Watch for engine rewrites that add cache padding; budget for them.

When it happens

Trigger: Optimized body larger than declared (e.g. cache padding or template expansion), or a trace whose DeclaredInputTokens was recorded below the true value; provider counting overhead (e.g. cached tokens counted in total) pushing the number over the ceiling.

Common situations: See trigger scenarios.

Related errors

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/b11f57664f7fb3fd. Report an issue: GitHub.