{"record":{"id":"528dea3f4315b1a2","repo":"JuliusBrussee/caveman","slug":"cachebench-quality-provenance-missing-for-q","errorCode":null,"errorMessage":"cachebench: quality provenance missing for %q","messagePattern":"cachebench: quality provenance missing for %q","errorType":"exception","errorClass":"ReplayRunError","httpStatus":null,"severity":"error","filePath":"cacheengine/cachebench/replay.go","lineNumber":750,"sourceCode":"\t\tevidence.FailureCode = \"provider_output_budget_exceeded\"\n\t\treturn ReplayResult{Evidence: evidence, ProviderResponse: append([]byte(nil), response.Body...)}, &ReplayRunError{\n\t\t\tRequestID: record.RequestID, FailureCode: evidence.FailureCode,\n\t\t\tErr: fmt.Errorf(\"cachebench: provider output tokens %d exceed request ceiling %d for %q\", usage.OutputTokens, record.MaxOutputTokens, record.RequestID),\n\t\t}\n\t}\n\tverification, err := runner.Verifier.Verify(ctx, ReplayVerificationInput{Trace: record, Optimized: optimized, Response: response})\n\tif err != nil {\n\t\tevidence.FailureCode = \"quality_verifier_error\"\n\t\treturn ReplayResult{Evidence: evidence, ProviderResponse: append([]byte(nil), response.Body...)}, &ReplayRunError{\n\t\t\tRequestID: record.RequestID, FailureCode: evidence.FailureCode,\n\t\t\tErr: fmt.Errorf(\"cachebench: quality verifier failed for %q: %w\", record.RequestID, err),\n\t\t}\n\t}\n\tif !validTaskVerification(verification) {\n\t\tevidence.FailureCode = \"quality_provenance_missing\"\n\t\treturn ReplayResult{Evidence: evidence, ProviderResponse: append([]byte(nil), response.Body...)}, &ReplayRunError{\n\t\t\tRequestID: record.RequestID, FailureCode: evidence.FailureCode,\n\t\t\tErr: fmt.Errorf(\"cachebench: quality provenance missing for %q\", record.RequestID),\n\t\t}\n\t}\n\tevidence.QualityPassed = verification.Passed\n\tevidence.QualityVerifier = verification.Verifier\n\tevidence.QualityEvidenceSHA256 = bodyDigest(verification.Evidence)\n\tevidence.Success = true\n\tobservation := NewObservationRecord(record.RequestID, record.Provider, record.Epoch, usage.TotalInputTokens, record.Body, response.Body, optimized, verification, usage.RawUsage)\n\treturn ReplayResult{\n\t\tEvidence: evidence, Observation: &observation,\n\t\tProviderResponse:     append([]byte(nil), response.Body...),\n\t\tVerificationEvidence: append([]byte(nil), verification.Evidence...),\n\t}, nil\n}\n\nfunc replayEvidenceBase(record TraceRecord, optimized cacheengine.NativeResult, scale float64, scheduled, started time.Time, tolerance time.Duration) ReplayEvidenceRecord {\n\tdrift := absDuration(started.Sub(scheduled))\n\treturn ReplayEvidenceRecord{\n\t\tSchema: ReplayEvidenceSchema, RequestID: record.RequestID,","sourceCodeStart":732,"sourceCodeEnd":768,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/766dce6b1394ebb56a3090748d5a0240a5aefb36/cacheengine/cachebench/replay.go#L732-L768","documentation":"The verification result for a replay request is missing provenance — no verifier identity/evidence digest is attached, so the pass/fail outcome cannot be attributed; failure code quality_provenance_missing.","triggerScenarios":"Thrown at cacheengine/cachebench/replay.go:750 when the library encounters an invalid state.","commonSituations":"See trigger scenarios.","solutions":["Ensure the verifier returns a verifier name and evidence payload","Wire the verification provenance fields into the replay pipeline"],"exampleFix":null,"handlingStrategy":"validation","validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"766dce6b1394ebb56a3090748d5a0240a5aefb36","analyzedAt":"2026-08-18T03:14:35.516Z","contentChangedAt":"2026-08-18T03:14:35.516Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}