{"record":{"id":"5474c50dac781aec","repo":"thanos-io/thanos","slug":"expected-2xx-response-got-d-body-v","errorCode":null,"errorMessage":"expected 2xx response, got %d. Body: %v","messagePattern":"expected 2xx response, got (.+?)\\. Body: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/promclient/promclient.go","lineNumber":144,"sourceCode":"\tif c.userAgent != \"\" {\n\t\treq.Header.Set(\"User-Agent\", c.userAgent)\n\t}\n\tif method == http.MethodPost {\n\t\treq.Header.Set(\"Content-Type\", \"application/x-www-form-urlencoded\")\n\t}\n\n\tresp, err := c.Do(req.WithContext(ctx))\n\tif err != nil {\n\t\treturn nil, 0, errors.Wrapf(err, \"perform %s request against %s\", method, u.String())\n\t}\n\tdefer runutil.ExhaustCloseWithErrCapture(&err, resp.Body, \"%s: close body\", req.URL.String())\n\n\tbody, err := io.ReadAll(resp.Body)\n\tif err != nil {\n\t\treturn nil, resp.StatusCode, errors.Wrap(err, \"read body\")\n\t}\n\tif resp.StatusCode/100 != 2 {\n\t\treturn nil, resp.StatusCode, errors.Errorf(\"expected 2xx response, got %d. Body: %v\", resp.StatusCode, string(body))\n\t}\n\treturn body, resp.StatusCode, nil\n}\n\n// IsWALDirAccessible returns no error if WAL dir can be found. This helps to tell\n// if we have access to Prometheus TSDB directory.\nfunc IsWALDirAccessible(dir string) error {\n\tconst errMsg = \"WAL dir is not accessible. Is this dir a TSDB directory? If yes it is shared with TSDB?\"\n\n\tf, err := os.Stat(filepath.Join(dir, \"wal\"))\n\tif err != nil {\n\t\treturn errors.Wrap(err, errMsg)\n\t}\n\n\tif !f.IsDir() {\n\t\treturn errors.New(errMsg)\n\t}\n","sourceCodeStart":126,"sourceCodeEnd":162,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/pkg/promclient/promclient.go#L126-L162","documentation":"req2xx reads the full HTTP response body from a Prometheus API call and throws this error when the status code is not in the 2xx range. It is the generic non-2xx guard used by all client methods (ExternalLabels, QueryInstant, QueryRange, AlertmanagerAlerts, BuildVersion, get2xxResultWithGRPCErrors), carrying the actual status code and response body so the caller can diagnose the server-side failure.","triggerScenarios":"Any Client method that goes through req2xx when Prometheus/Alertmanager returns 4xx or 5xx: bad query (400), missing/invalid tenant or auth header (401/403), not-found endpoints (404), Prometheus restarting or overloaded (500/503), gateway timeouts from proxies.","commonSituations":"Querying a Prometheus that returns '422 unprocessable entity' for a bad PromQL expression; hitting Thanos Query with a wrong tenant header; scraping Alertmanager API with mismatched api version; sidecar behind a proxy that returns 502.","solutions":["Inspect the status code and body included in the error to identify the server-side cause","Verify the Prometheus/Alertmanager base URL and API path prefix are correct","Check authentication/tenancy headers required by the target (e.g. Thanos-Scope, auth bearer)","Validate PromQL query syntax before calling QueryInstant/QueryRange","Add retry with backoff for transient 5xx responses"],"exampleFix":"// before: error surfaced raw, hard to tell 400 from 503\nlabels, err := client.QueryInstant(ctx, rng, timeout, query)\n// after: distinguish non-2xx classes\nlabels, err := client.QueryInstant(ctx, rng, timeout, query)\nif err != nil {\n    var terr interface{ StatusCode() int }\n    if errors.As(err, &terr) && terr.StatusCode() >= 500 {\n        // retry with backoff\n    }\n    return err\n}","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"if err != nil {\n    if strings.Contains(err.Error(), \"expected 2xx response, got 5\") {\n        // transient server error: retry with backoff\n    } else if strings.Contains(err.Error(), \"got 4\") {\n        // client error: fail fast, fix request\n    }\n    return err\n}","preventionTips":["Validate queries and API paths before sending requests","Configure auth/tenant headers required by the target service","Add retry-with-backoff for 5xx, fail fast on 4xx","Monitor status codes of upstream Prometheus endpoints"],"tags":["http","prometheus","client","non-2xx"],"backgroundTag":"http-non-2xx-response","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}