{"record":{"id":"5d8458f387ee6f60","repo":"sipeed/picoclaw","slug":"api-error-d-s","errorCode":null,"errorMessage":"API error %d: %s","messagePattern":"API error (.+?): (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"cmd/membench/llm_client.go","lineNumber":164,"sourceCode":"\t\t\tif c.APIKey != \"\" {\n\t\t\t\treq.Header.Set(\"Authorization\", \"Bearer \"+c.APIKey)\n\t\t\t}\n\t\t}\n\n\t\tvar resp *http.Response\n\t\tresp, lastErr = c.Client.Do(req)\n\t\tif lastErr != nil {\n\t\t\tcontinue // network/timeout error → retry\n\t\t}\n\n\t\trespBody, lastErr = io.ReadAll(resp.Body)\n\t\t_ = resp.Body.Close()\n\t\tif lastErr != nil {\n\t\t\tcontinue\n\t\t}\n\n\t\tif resp.StatusCode == 429 || resp.StatusCode >= 500 {\n\t\t\tlastErr = fmt.Errorf(\"API error %d: %s\", resp.StatusCode, string(respBody))\n\t\t\tcontinue // rate limit or server error → retry\n\t\t}\n\t\tif resp.StatusCode != 200 {\n\t\t\treturn \"\", fmt.Errorf(\"API error %d: %s\", resp.StatusCode, string(respBody))\n\t\t}\n\n\t\tlastErr = nil\n\t\tbreak\n\t}\n\tif lastErr != nil {\n\t\treturn \"\", fmt.Errorf(\"after %d retries: %w\", c.MaxRetries, lastErr)\n\t}\n\n\tvar chatResp chatResponse\n\tif err := json.Unmarshal(respBody, &chatResp); err != nil {\n\t\treturn \"\", fmt.Errorf(\"parse response: %w\", err)\n\t}\n\tif len(chatResp.Choices) == 0 {","sourceCodeStart":146,"sourceCodeEnd":182,"githubUrl":"https://github.com/sipeed/picoclaw/blob/49183d7e8daed0dba89ddbb6fcb60089401d9680/cmd/membench/llm_client.go#L146-L182","documentation":"Transient HTTP failure classification in membench's LLM client: any 429 or 5xx status is wrapped as 'API error : ' and assigned to lastErr, then the loop continues with exponential backoff (1s, 2s, 4s...). This error object is normally never returned to the caller — it only escapes wrapped by 'after N retries' if the budget is exhausted (llm_client.go:164-166).","triggerScenarios":"Provider rate limiting (429) under concurrent eval workers; OpenAI-compatible server 500/502/503 during load or model cold-start; Ollama returning 503 while the model is loading into memory; proxy or gateway 5xx.","commonSituations":"High-parallelism benchmark runs tripping tokens-per-minute limits; self-hosted inference server overloaded; maintenance windows on hosted APIs; retries firing against a 429 whose Retry-After exceeds the fixed backoff schedule.","solutions":["Reduce concurrency / add client-side throttling so 429s stop firing","Increase MaxRetries on the client so the backoff schedule can outlast the rate-limit window","Read the response body — provider JSON usually names the real cause (quota, model loading)","Honor Retry-After when present instead of the fixed 1s/2s/4s ladder"],"exampleFix":"// before\nbackoff := time.Duration(1<<(attempt-1)) * time.Second\n\n// after\nbackoff := time.Duration(1<<(attempt-1)) * time.Second\nif ra := resp.Header.Get(\"Retry-After\"); ra != \"\" {\n if secs, err := strconv.Atoi(ra); err == nil {\n backoff = time.Duration(secs) * time.Second\n }\n}","handlingStrategy":"retry","validationCode":"// probe the provider's rate posture before a benchmark run\nresp, err := http.Get(baseURL + \"/models\")\nif err == nil && resp.StatusCode == 429 {\n return fmt.Errorf(\"already rate-limited at %s; delay the run\", baseURL)\n}","typeGuard":null,"tryCatchPattern":"// already inside the client: keep 429/5xx on lastErr and continue\nif resp.StatusCode == 429 || resp.StatusCode >= 500 {\n lastErr = fmt.Errorf(\"API error %d: %s\", resp.StatusCode, string(respBody))\n continue\n}\n// caller side: this error only surfaces wrapped by 'after N retries' — handle it there","preventionTips":["Throttle concurrency below the provider's TPM/RPM limits","Honor Retry-After when present instead of the fixed 1s/2s/4s ladder","Size MaxRetries so total backoff exceeds the throttling window","Watch for 503 from Ollama while a model loads — warm it up with one request before benchmarking"],"tags":["go","http","rate-limit","retry","llm-client"],"backgroundTag":null,"analyzedSha":"49183d7e8daed0dba89ddbb6fcb60089401d9680","analyzedAt":"2026-08-15T21:55:41.315Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}