{"record":{"id":"1564bd5ec27a60b8","repo":"sipeed/picoclaw","slug":"after-d-retries-w","errorCode":null,"errorMessage":"after %d retries: %w","messagePattern":"after (.+?) retries: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"cmd/membench/llm_client.go","lineNumber":175,"sourceCode":"\t\trespBody, lastErr = io.ReadAll(resp.Body)\n\t\t_ = resp.Body.Close()\n\t\tif lastErr != nil {\n\t\t\tcontinue\n\t\t}\n\n\t\tif resp.StatusCode == 429 || resp.StatusCode >= 500 {\n\t\t\tlastErr = fmt.Errorf(\"API error %d: %s\", resp.StatusCode, string(respBody))\n\t\t\tcontinue // rate limit or server error → retry\n\t\t}\n\t\tif resp.StatusCode != 200 {\n\t\t\treturn \"\", fmt.Errorf(\"API error %d: %s\", resp.StatusCode, string(respBody))\n\t\t}\n\n\t\tlastErr = nil\n\t\tbreak\n\t}\n\tif lastErr != nil {\n\t\treturn \"\", fmt.Errorf(\"after %d retries: %w\", c.MaxRetries, lastErr)\n\t}\n\n\tvar chatResp chatResponse\n\tif err := json.Unmarshal(respBody, &chatResp); err != nil {\n\t\treturn \"\", fmt.Errorf(\"parse response: %w\", err)\n\t}\n\tif len(chatResp.Choices) == 0 {\n\t\treturn \"\", fmt.Errorf(\"no choices in response\")\n\t}\n\tcontent := strings.TrimSpace(chatResp.Choices[0].Message.Content)\n\t// Strip any residual <think>...</think> blocks\n\tif idx := strings.Index(content, \"</think>\"); idx >= 0 {\n\t\tcontent = strings.TrimSpace(content[idx+len(\"</think>\"):])\n\t}\n\t// Fallback: GLM/DeepSeek put thinking output in reasoning_content when thinking is enabled\n\tif content == \"\" && chatResp.Choices[0].Message.ReasoningContent != \"\" {\n\t\tcontent = strings.TrimSpace(chatResp.Choices[0].Message.ReasoningContent)\n\t}","sourceCodeStart":157,"sourceCodeEnd":193,"githubUrl":"https://github.com/sipeed/picoclaw/blob/49183d7e8daed0dba89ddbb6fcb60089401d9680/cmd/membench/llm_client.go#L157-L193","documentation":"Retry budget exhausted in membench's LLM client: after MaxRetries re-attempts (initial try + N retries with 1s/2s/4s... backoff) the last error — a network/timeout failure, a 429, or a 5xx — is wrapped as 'after <N> retries: <cause>'. The %w wrap preserves the original error for errors.Is/As inspection, so the root cause is always the suffix of the message.","triggerScenarios":"Sustained 429 rate limiting longer than the backoff ladder; provider outage spanning the whole retry window; dead endpoint where every Do() times out (server down, firewall drop); context deadline shorter than total backoff time.","commonSituations":"Benchmark runs against a quota that resets hourly; local model server crashed mid-eval; VPN/proxy dropping long-lived connections; MaxRetries left at default while the provider throttles aggressively.","solutions":["Unwrap the cause (errors.Unwrap) — fix that first; the retry layer only tells you it never succeeded","Raise MaxRetries and/or the per-request timeout so the ladder outlasts throttling windows","Lower request rate/concurrency if the cause is 429","Check the endpoint is actually up (curl $BASE/models) before rerunning the full eval"],"exampleFix":"// before\nif lastErr != nil {\n    return \"\", fmt.Errorf(\"after %d retries: %w\", c.MaxRetries, lastErr)\n}\n\n// after\nif lastErr != nil {\n    var statusErr *apiStatusError\n    if errors.As(lastErr, &statusErr) && statusErr.Code == 429 {\n        return \"\", fmt.Errorf(\"rate-limited after %d retries; lower concurrency or raise MaxRetries: %w\", c.MaxRetries, lastErr)\n    }\n    return \"\", fmt.Errorf(\"after %d retries: %w\", c.MaxRetries, lastErr)\n}","handlingStrategy":"retry","validationCode":"if err := preflightEndpoint(baseURL); err != nil {\n    return err // avoid burning a long eval on a dead endpoint\n}\n// ensure MaxRetries and timeouts sized to worst-case throttling before the run\nif c.MaxRetries < 5 { c.MaxRetries = 5 }","typeGuard":null,"tryCatchPattern":"if err := llm.Complete(ctx, prompt); err != nil {\n    if strings.Contains(err.Error(), \"after\") && strings.Contains(err.Error(), \"retries\") {\n        cause := errors.Unwrap(err) // the real network/429/5xx failure\n        if isTransient(cause) {\n            time.Sleep(30 * time.Second) // let the throttle window reset\n            return llm.Complete(ctx, prompt) // outer, coarse retry\n        }\n    }\n    return err\n}","preventionTips":["Always inspect errors.Unwrap of the 'after N retries' wrapper — the cause dictates the fix","Set per-request timeouts and a MaxRetries whose backoff ladder exceeds rate-limit windows","Reduce parallel workers when the cause is 429; retrying harder makes throttling worse","Persist partial eval progress so an exhausted retry budget doesn't restart the run"],"tags":["go","retry","network","llm-client","backoff"],"backgroundTag":null,"analyzedSha":"49183d7e8daed0dba89ddbb6fcb60089401d9680","analyzedAt":"2026-08-15T21:55:41.315Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}