{"record":{"id":"45eb8caf707ecd12","repo":"JuliusBrussee/caveman","slug":"provider-http-status","errorCode":"provider_http_status","errorMessage":"cachebench: provider returned HTTP %d for %q","messagePattern":"cachebench: provider returned HTTP (.+?) for %q","errorType":"http","errorClass":"ReplayRunError","httpStatus":null,"severity":"error","filePath":"cacheengine/cachebench/replay.go","lineNumber":710,"sourceCode":"\t\tevidence.FailureCode = \"transport_error\"\n\t\treturn ReplayResult{Evidence: evidence, ProviderResponse: append([]byte(nil), response.Body...)}, &ReplayRunError{\n\t\t\tRequestID: record.RequestID, FailureCode: evidence.FailureCode,\n\t\t\tErr: fmt.Errorf(\"cachebench: provider transport failed for %q: %w\", record.RequestID, sendErr),\n\t\t}\n\t}\n\tif !validProviderRequestID(response.ProviderRequestID) {\n\t\tevidence.FailureCode = \"provider_response_invalid\"\n\t\treturn ReplayResult{Evidence: evidence, ProviderResponse: append([]byte(nil), response.Body...)}, &ReplayRunError{\n\t\t\tRequestID: record.RequestID, FailureCode: evidence.FailureCode,\n\t\t\tErr: fmt.Errorf(\"cachebench: provider returned invalid request identity for %q\", record.RequestID),\n\t\t}\n\t}\n\tevidence.ProviderRequestID = response.ProviderRequestID\n\tif response.StatusCode < 200 || response.StatusCode >= 300 {\n\t\tevidence.FailureCode = \"provider_http_status\"\n\t\treturn ReplayResult{Evidence: evidence, ProviderResponse: append([]byte(nil), response.Body...)}, &ReplayRunError{\n\t\t\tRequestID: record.RequestID, FailureCode: evidence.FailureCode,\n\t\t\tErr: fmt.Errorf(\"cachebench: provider returned HTTP %d for %q\", response.StatusCode, record.RequestID),\n\t\t}\n\t}\n\tusage, ok := cacheengine.ExtractProviderUsage(record.Provider, response.Body)\n\tif !ok {\n\t\tevidence.FailureCode = \"provider_usage_invalid\"\n\t\treturn ReplayResult{Evidence: evidence, ProviderResponse: append([]byte(nil), response.Body...)}, &ReplayRunError{\n\t\t\tRequestID: record.RequestID, FailureCode: evidence.FailureCode,\n\t\t\tErr: fmt.Errorf(\"cachebench: provider usage unavailable for %q\", record.RequestID),\n\t\t}\n\t}\n\tevidence.ProviderUsageSHA256 = bodyDigest(usage.RawUsage)\n\tevidence.ProviderTotalInputTokens = usage.TotalInputTokens\n\tevidence.ProviderOutputTokens = usage.OutputTokens\n\tif usage.TotalInputTokens > record.DeclaredInputTokens {\n\t\tevidence.FailureCode = \"provider_input_budget_exceeded\"\n\t\treturn ReplayResult{Evidence: evidence, ProviderResponse: append([]byte(nil), response.Body...)}, &ReplayRunError{\n\t\t\tRequestID: record.RequestID, FailureCode: evidence.FailureCode,\n\t\t\tErr: fmt.Errorf(\"cachebench: provider input tokens %d exceed declared ceiling %d for %q\", usage.TotalInputTokens, record.DeclaredInputTokens, record.RequestID),","sourceCodeStart":692,"sourceCodeEnd":728,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/27d5a3981a347890211bb1bf2439e5c821a63bc9/cacheengine/cachebench/replay.go#L692-L728","documentation":"The provider answered with a status outside 200-299. The runner records status, ids, and body digest in evidence, then fails the request with FailureCode provider_http_status and the concrete status code.","triggerScenarios":"4xx/5xx from the provider: 401/403 bad credentials, 404 wrong endpoint in the trace record, 429 rate limiting under concurrent replay, 5xx provider incidents, 400 from a malformed optimized body.","commonSituations":"Expired or wrong API keys; endpoint drift between capture time and replay (renamed models/routes); replay concurrency tripping rate limits; provider outage during the run.","solutions":["Map the status: 401/403 fix credentials; 404 fix record.Endpoint/Model; 429 lower MaxConcurrency or add backoff; 5xx retry later; 400 inspect the optimized body against the provider schema.","Check the emitted evidence's ProviderEvidenceSHA256 against the response body you captured for details.","Re-verify the trace's Endpoint and Model fields against the provider's current API.","For 429s, scale replay rate via TimeScale or concurrency limits."],"exampleFix":"// before\nrunner.MaxConcurrency = 32 // provider returns 429\n\n// after\nrunner.MaxConcurrency = 4\n// and/or add retry policy around transient 429/5xx at the transport layer","handlingStrategy":"retry","validationCode":"// pre-flight one representative request per provider to catch auth/endpoint issues\nif status, err := transport.Probe(ctx, sampleRecord); err != nil || status < 200 || status >= 300 {\n    return fmt.Errorf(\"pre-flight failed with status %d: %w\", status, err)\n}","typeGuard":"func isProviderHTTPStatus(err error) bool {\n    var rre *cachebench.ReplayRunError\n    return errors.As(err, &rre) && rre.FailureCode == \"provider_http_status\"\n}","tryCatchPattern":"if err := runner.Run(ctx, records, emit); err != nil {\n    if isProviderHTTPStatus(err) {\n        // 401/403: fix credentials; 404: fix Endpoint/Model in trace; 429: cut concurrency; 5xx: retry later\n    }\n    return err\n}","preventionTips":["Pre-flight each provider with one request before committing to a full replay.","Keep trace Endpoint/Model fields current with provider API changes.","Cap MaxConcurrency below known rate limits for your account tier."],"tags":["cachebench","http","provider-api","rate-limit"],"backgroundTag":null,"analyzedSha":"27d5a3981a347890211bb1bf2439e5c821a63bc9","analyzedAt":"2026-08-15T09:26:11.751Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}