{"record":{"id":"87b65d639a932f42","repo":"Tencent/WeKnora","slug":"http-d-for-s","errorCode":null,"errorMessage":"HTTP %d for %s","messagePattern":"HTTP (.+?) for (.+?)","errorType":"http","errorClass":null,"httpStatus":null,"severity":"error","filePath":"internal/utils/httputil.go","lineNumber":31,"sourceCode":"\tMaxRedirects: 10,\n})\n\n// DownloadBytes fetches the content at the given HTTP(S) URL and returns the\n// raw bytes. It reuses a package-level http.Client with a 60-second timeout.\nfunc DownloadBytes(url string) ([]byte, error) {\n\tif !strings.HasPrefix(url, \"http://\") && !strings.HasPrefix(url, \"https://\") {\n\t\treturn nil, fmt.Errorf(\"unsupported URL scheme: %s\", url)\n\t}\n\tif err := ValidateURLForSSRF(url); err != nil {\n\t\treturn nil, fmt.Errorf(\"URL rejected by SSRF policy: %w\", err)\n\t}\n\tresp, err := defaultHTTPClient.Get(url)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"HTTP GET: %w\", err)\n\t}\n\tdefer resp.Body.Close()\n\tif resp.StatusCode != http.StatusOK {\n\t\treturn nil, fmt.Errorf(\"HTTP %d for %s\", resp.StatusCode, url)\n\t}\n\tdata, err := io.ReadAll(resp.Body)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"read body: %w\", err)\n\t}\n\treturn data, nil\n}\n","sourceCodeStart":13,"sourceCodeEnd":39,"githubUrl":"https://github.com/Tencent/WeKnora/blob/988cbb03305e055d8ebb7d46d9ac6cc0803cd074/internal/utils/httputil.go#L13-L39","documentation":"DownloadBytes requires an HTTP 200 response; any other status (404, 403, 500, redirects not followed to a 200, etc.) produces \"HTTP <status> for <url>\". The function treats non-OK as failure and returns no body.","triggerScenarios":"Calling DownloadBytes against a URL that returns a non-200 status: missing resource, auth-required endpoint, rate limiting (429), server error, or a redirect chain not ending in 200 (the default client follows redirects, so this means the final response was non-200).","commonSituations":"Dead or moved download links, private assets requiring credentials, CDN rate limits, URLs behind login walls, or temporary upstream 5xx during deployments.","solutions":["Verify the URL is correct and publicly accessible (curl -I it)","Add required auth via a client that supports headers instead of DownloadBytes, or pre-sign the URL","Handle the status code in the error message — retry 429/5xx with backoff, fail fast on 4xx","Check for expired/rotated links (signed URLs past expiry return 403)"],"exampleFix":"// before\nDownloadBytes(\"https://example.com/old-file.pdf\") // 404\n// after\nresp, err := http.Head(\"https://example.com/file.pdf\")\nif err == nil && resp.StatusCode == http.StatusOK {\n    data, err := DownloadBytes(\"https://example.com/file.pdf\")\n}","handlingStrategy":"retry","validationCode":"resp, err := http.Head(url)\nif err != nil || resp.StatusCode != http.StatusOK {\n    return fmt.Errorf(\"URL not downloadable (status %v)\", resp)\n}","typeGuard":null,"tryCatchPattern":"data, err := DownloadBytes(url)\nif err != nil {\n    if strings.Contains(err.Error(), \"HTTP \") {\n        var code int\n        if _, scanErr := fmt.Sscanf(err.Error(), \"HTTP %d\", &code); scanErr == nil {\n            if code == 429 || code >= 500 { /* retry with backoff */ } else { /* permanent — fix URL/auth */ }\n        }\n    }\n}","preventionTips":["Pre-flight the URL with HEAD to catch 4xx before the full download","Retry 429/5xx with exponential backoff; never retry 4xx client errors","For authenticated assets, use signed URLs or a client that sends credentials","Alert on links whose signed URLs may expire before use"],"tags":["http","status-code","network","go"],"backgroundTag":"http-non-200","analyzedSha":"988cbb03305e055d8ebb7d46d9ac6cc0803cd074","analyzedAt":"2026-09-02T14:41:08.344Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}