{"record":{"id":"a2ec2af2295f3c5b","repo":"JuliusBrussee/caveman","slug":"cachebench-no-replay-evidence","errorCode":null,"errorMessage":"cachebench: no replay evidence","messagePattern":"cachebench: no replay evidence","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"cacheengine/cachebench/replay.go","lineNumber":248,"sourceCode":"// ReplayEvidenceSummary aggregates exact validated evidence population.\ntype ReplayEvidenceSummary struct {\n\tSchema                            string                  `json:\"schema\"`\n\tRequests                          int                     `json:\"requests\"`\n\tSuccessful                        int                     `json:\"successful\"`\n\tFailed                            int                     `json:\"failed\"`\n\tQualityPassed                     int                     `json:\"quality_passed\"`\n\tInputTokens                       int64                   `json:\"provider_total_input_tokens\"`\n\tOutputTokens                      int64                   `json:\"provider_output_tokens\"`\n\tTimingFaithful                    bool                    `json:\"timing_faithful\"`\n\tInputBudgetClaimedProviderCounted bool                    `json:\"input_budget_claimed_provider_counted\"`\n\tLatency                           LatencyDistribution     `json:\"latency\"`\n\tProviders                         []ProviderReplaySummary `json:\"providers\"`\n}\n\n// SummarizeReplayEvidence validates and aggregates replay records.\nfunc SummarizeReplayEvidence(records []ReplayEvidenceRecord) (ReplayEvidenceSummary, error) {\n\tif len(records) == 0 {\n\t\treturn ReplayEvidenceSummary{}, errors.New(\"cachebench: no replay evidence\")\n\t}\n\tsummary := ReplayEvidenceSummary{\n\t\tSchema: ReplaySummarySchema, Requests: len(records),\n\t\tTimingFaithful: true, InputBudgetClaimedProviderCounted: true,\n\t}\n\ttype accumulator struct {\n\t\tsummary   ProviderReplaySummary\n\t\tlatencies []int64\n\t}\n\tgroups := map[string]*accumulator{}\n\tseen := map[string]bool{}\n\tlatencies := make([]int64, 0, len(records))\n\tfor index, record := range records {\n\t\tif err := validateReplayEvidence(record); err != nil {\n\t\t\treturn ReplayEvidenceSummary{}, fmt.Errorf(\"cachebench: replay evidence %d: %w\", index, err)\n\t\t}\n\t\tif seen[record.RequestID] {\n\t\t\treturn ReplayEvidenceSummary{}, fmt.Errorf(\"cachebench: duplicate replay evidence request %q\", record.RequestID)","sourceCodeStart":230,"sourceCodeEnd":266,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/27d5a3981a347890211bb1bf2439e5c821a63bc9/cacheengine/cachebench/replay.go#L230-L266","documentation":"SummarizeReplayEvidence aggregates ReplayEvidenceRecord values into a summary and requires at least one record. An empty slice cannot produce token totals, latency distributions, or pass counts, so it errors rather than returning a zero summary that could be mistaken for a real (all-zero) replay.","triggerScenarios":"Calling SummarizeReplayEvidence with an empty records slice — typically the replay run produced no evidence (all requests filtered, JSONL empty, or the replay loop never executed) before summarization.","commonSituations":"Replay pipeline stage skipped due to dry-run flag but summarize still called; evidence file empty after a crashed replay; records filtered by request ID set that matched nothing.","solutions":["Check len(records) > 0 after replay and before summarization; treat zero as a replay failure, not a summary input","Verify the evidence JSONL was written and non-empty after the replay phase","Log replay counts per provider so an empty result is visible with context"],"exampleFix":"// before\nsummary, err := cachebench.SummarizeReplayEvidence(records) // empty\n\n// after\nif len(records) == 0 {\n    return fmt.Errorf(\"replay produced no evidence; check replay filters and evidence output path\")\n}\nsummary, err := cachebench.SummarizeReplayEvidence(records)","handlingStrategy":"validation","validationCode":"if len(records) == 0 {\n    return errors.New(\"replay produced no evidence; refusing to summarize\")\n}\nsummary, err := cachebench.SummarizeReplayEvidence(records)","typeGuard":"func hasReplayEvidence(rs []ReplayEvidenceRecord) bool { return len(rs) > 0 }","tryCatchPattern":"if _, err := cachebench.SummarizeReplayEvidence(records); err != nil {\n    if err.Error() == \"cachebench: no replay evidence\" {\n        // check replay stage logs: filters, dry-run flags, evidence file path\n    }\n}","preventionTips":["Assert evidence file is non-empty immediately after the replay phase","Treat zero evidence records as a replay failure with diagnostics, not a summary input","Log replay record counts per provider before summarization"],"tags":["go","replay","validation","empty-input"],"backgroundTag":null,"analyzedSha":"27d5a3981a347890211bb1bf2439e5c821a63bc9","analyzedAt":"2026-08-15T09:26:11.751Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}