cachebench: failed replay evidence needs known failure_code

Error message

cachebench: failed replay evidence needs known failure_code

What it means

Error "cachebench: failed replay evidence needs known failure_code" thrown in JuliusBrussee/caveman.

Source

Thrown at cacheengine/cachebench/replay.go:896

			return errors.New("cachebench: invalid replay provider usage evidence")
		}
	} else if record.ProviderTotalInputTokens != 0 || record.ProviderOutputTokens != 0 {
		return errors.New("cachebench: replay token counters lack provider usage evidence")
	}
	if record.Applied != (record.Decision == cacheengine.DecisionApply) || record.Applied && len(record.OptimizerIDs) == 0 {
		return errors.New("cachebench: inconsistent replay engine decision")
	}
	switch record.Attribution {
	case cacheengine.AttributionNone, cacheengine.AttributionCausal, cacheengine.AttributionAffinity, cacheengine.AttributionOrganic:
	default:
		return errors.New("cachebench: invalid replay attribution")
	}
	if record.Success {
		if record.HTTPStatus < 200 || record.HTTPStatus >= 300 || !validEvidenceSHA256(record.ProviderEvidenceSHA256) || !validEvidenceSHA256(record.ProviderUsageSHA256) || record.ProviderTotalInputTokens <= 0 || !validBoundedText(record.QualityVerifier, 256, false) || !validEvidenceSHA256(record.QualityEvidenceSHA256) || record.FailureCode != "" {
			return errors.New("cachebench: incomplete successful replay evidence")
		}
	} else if !validReplayFailureCode(record.FailureCode) {
		return errors.New("cachebench: failed replay evidence needs known failure_code")
	}
	return nil
}

func validReplayFailureCode(value string) bool {
	switch value {
	case "schedule_drift", "model_visible_mismatch", "engine_not_cacheable", "clock_regression", "transport_error", "provider_response_invalid", "provider_http_status", "provider_usage_invalid", "provider_input_budget_exceeded", "provider_output_budget_exceeded", "quality_verifier_error", "quality_provenance_missing":
		return true
	default:
		return false
	}
}

func absDuration(value time.Duration) time.Duration {
	if value == time.Duration(math.MinInt64) {
		return time.Duration(math.MaxInt64)
	}
	if value < 0 {

View on GitHub (pinned to 27d5a3981a)

Solutions

Set a known failure_code on the failed replay evidence.

When it happens

Trigger: Thrown at cacheengine/cachebench/replay.go:896 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/9cdf228922b5d7c2. Report an issue: GitHub.