JuliusBrussee/caveman · error · ReplayRunError

cachebench: quality provenance missing for %q

Error message

cachebench: quality provenance missing for %q

What it means

The verification result for a replay request is missing provenance — no verifier identity/evidence digest is attached, so the pass/fail outcome cannot be attributed; failure code quality_provenance_missing.

Source

Thrown at cacheengine/cachebench/replay.go:750

		evidence.FailureCode = "provider_output_budget_exceeded"
		return ReplayResult{Evidence: evidence, ProviderResponse: append([]byte(nil), response.Body...)}, &ReplayRunError{
			RequestID: record.RequestID, FailureCode: evidence.FailureCode,
			Err: fmt.Errorf("cachebench: provider output tokens %d exceed request ceiling %d for %q", usage.OutputTokens, record.MaxOutputTokens, record.RequestID),
		}
	}
	verification, err := runner.Verifier.Verify(ctx, ReplayVerificationInput{Trace: record, Optimized: optimized, Response: response})
	if err != nil {
		evidence.FailureCode = "quality_verifier_error"
		return ReplayResult{Evidence: evidence, ProviderResponse: append([]byte(nil), response.Body...)}, &ReplayRunError{
			RequestID: record.RequestID, FailureCode: evidence.FailureCode,
			Err: fmt.Errorf("cachebench: quality verifier failed for %q: %w", record.RequestID, err),
		}
	}
	if !validTaskVerification(verification) {
		evidence.FailureCode = "quality_provenance_missing"
		return ReplayResult{Evidence: evidence, ProviderResponse: append([]byte(nil), response.Body...)}, &ReplayRunError{
			RequestID: record.RequestID, FailureCode: evidence.FailureCode,
			Err: fmt.Errorf("cachebench: quality provenance missing for %q", record.RequestID),
		}
	}
	evidence.QualityPassed = verification.Passed
	evidence.QualityVerifier = verification.Verifier
	evidence.QualityEvidenceSHA256 = bodyDigest(verification.Evidence)
	evidence.Success = true
	observation := NewObservationRecord(record.RequestID, record.Provider, record.Epoch, usage.TotalInputTokens, record.Body, response.Body, optimized, verification, usage.RawUsage)
	return ReplayResult{
		Evidence: evidence, Observation: &observation,
		ProviderResponse:     append([]byte(nil), response.Body...),
		VerificationEvidence: append([]byte(nil), verification.Evidence...),
	}, nil
}

func replayEvidenceBase(record TraceRecord, optimized cacheengine.NativeResult, scale float64, scheduled, started time.Time, tolerance time.Duration) ReplayEvidenceRecord {
	drift := absDuration(started.Sub(scheduled))
	return ReplayEvidenceRecord{
		Schema: ReplayEvidenceSchema, RequestID: record.RequestID,

View on GitHub (pinned to 766dce6b13)

Solutions

1. Ensure the verifier returns a verifier name and evidence payload
2. Wire the verification provenance fields into the replay pipeline

When it happens

Trigger: Thrown at cacheengine/cachebench/replay.go:750 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of JuliusBrussee/caveman@766dce6b13 (2026-08-18). Data as JSON: /api/errors/528dea3f4315b1a2. Report an issue: GitHub.