{"record":{"id":"604e7f3d555b1a20","repo":"Tencent/WeKnora","slug":"http-get-w-604e7f","errorCode":null,"errorMessage":"HTTP GET: %w","messagePattern":"HTTP GET: %w","errorType":"http","errorClass":null,"httpStatus":null,"severity":"error","filePath":"internal/utils/httputil.go","lineNumber":27,"sourceCode":")\n\nvar defaultHTTPClient = NewSSRFSafeHTTPClient(SSRFSafeHTTPClientConfig{\n\tTimeout:      60 * time.Second,\n\tMaxRedirects: 10,\n})\n\n// DownloadBytes fetches the content at the given HTTP(S) URL and returns the\n// raw bytes. It reuses a package-level http.Client with a 60-second timeout.\nfunc DownloadBytes(url string) ([]byte, error) {\n\tif !strings.HasPrefix(url, \"http://\") && !strings.HasPrefix(url, \"https://\") {\n\t\treturn nil, fmt.Errorf(\"unsupported URL scheme: %s\", url)\n\t}\n\tif err := ValidateURLForSSRF(url); err != nil {\n\t\treturn nil, fmt.Errorf(\"URL rejected by SSRF policy: %w\", err)\n\t}\n\tresp, err := defaultHTTPClient.Get(url)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"HTTP GET: %w\", err)\n\t}\n\tdefer resp.Body.Close()\n\tif resp.StatusCode != http.StatusOK {\n\t\treturn nil, fmt.Errorf(\"HTTP %d for %s\", resp.StatusCode, url)\n\t}\n\tdata, err := io.ReadAll(resp.Body)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"read body: %w\", err)\n\t}\n\treturn data, nil\n}\n","sourceCodeStart":9,"sourceCodeEnd":39,"githubUrl":"https://github.com/Tencent/WeKnora/blob/988cbb03305e055d8ebb7d46d9ac6cc0803cd074/internal/utils/httputil.go#L9-L39","documentation":"DownloadBytes wraps any transport-level failure of the defaultHTTPClient.Get call (60s timeout client) as \"HTTP GET: <underlying error>\". It means the request could not be completed — DNS, connection, TLS, or timeout — not an HTTP error status.","triggerScenarios":"Calling DownloadBytes when the host doesn't resolve, the connection is refused/dropped, TLS handshake fails, or the request exceeds the client's 60-second timeout.","commonSituations":"Target host down or DNS misconfigured, firewall egress blocks, expired/invalid TLS certificates on the target, slow endpoints exceeding the 60s timeout, or no network from the running environment (containers, CI).","solutions":["Read the wrapped cause in the error to identify DNS vs connection vs TLS vs timeout","Verify the URL's host resolves and is reachable from the runtime environment (curl it)","Check egress/firewall/proxy settings in containers or CI","For large/slow downloads, use a client with a longer timeout than the default 60s","Add retry with backoff for transient network failures"],"exampleFix":"// before\nresp := DownloadBytes(url) // fails: HTTP GET: dial tcp: i/o timeout\n// after\nif err != nil {\n    var netErr net.Error\n    if errors.As(err, &netErr) && netErr.Timeout() {\n        err = retryWithBackoff(func() error { _, err = DownloadBytes(url); return err })\n    }\n}","handlingStrategy":"retry","validationCode":"if !isHTTPURL(raw) { return fmt.Errorf(\"bad url\") }\n// pre-check reachability where appropriate:\nconn, err := net.DialTimeout(\"tcp\", net.JoinHostPort(host, port), 5*time.Second)\nif err != nil { return fmt.Errorf(\"host unreachable: %w\", err) }\nconn.Close()","typeGuard":"func isTimeoutErr(err error) bool {\n    var ne net.Error\n    return errors.As(err, &ne) && ne.Timeout()\n}","tryCatchPattern":"data, err := DownloadBytes(url)\nif err != nil {\n    if strings.Contains(err.Error(), \"HTTP GET:\") {\n        var ne net.Error\n        if errors.As(err, &ne) && ne.Timeout() {\n            // retry with longer deadline\n        } else {\n            // DNS/connect/TLS problem — check host reachability\n        }\n    }\n}","preventionTips":["Monitor egress connectivity from your runtime environment (containers, CI)","Add retry with backoff for transient network errors","Increase the timeout for known-slow endpoints with a custom client","Verify TLS certificates on target hosts are valid and current"],"tags":["network","http","timeout","go"],"backgroundTag":"http-request-failed","analyzedSha":"988cbb03305e055d8ebb7d46d9ac6cc0803cd074","analyzedAt":"2026-09-02T14:41:08.344Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}