{"record":{"id":"675b24e56df10027","repo":"Tencent/WeKnora","slug":"http-get-w","errorCode":null,"errorMessage":"HTTP GET: %w","messagePattern":"HTTP GET: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"internal/infrastructure/docparser/image_resolver.go","lineNumber":1017,"sourceCode":"\n\timages = append(images, mdImages...)\n\timages = append(images, htmlImages...)\n\treturn markdown, images, nil\n}\n\n// downloadImage fetches an image from remoteURL using the provided SSRF-safe\n// client. It validates Content-Type and enforces maxRemoteImageSize.\nfunc downloadImage(ctx context.Context, client *http.Client, remoteURL string) (data []byte, mimeType string, err error) {\n\treq, err := http.NewRequestWithContext(ctx, http.MethodGet, remoteURL, nil)\n\tif err != nil {\n\t\treturn nil, \"\", fmt.Errorf(\"create request: %w\", err)\n\t}\n\t// Some CDNs require a browser-like User-Agent.\n\treq.Header.Set(\"User-Agent\", \"Mozilla/5.0 (compatible; WeKnora/1.0)\")\n\n\tresp, err := client.Do(req)\n\tif err != nil {\n\t\treturn nil, \"\", fmt.Errorf(\"HTTP GET: %w\", err)\n\t}\n\tdefer resp.Body.Close()\n\n\tif resp.StatusCode != http.StatusOK {\n\t\treturn nil, \"\", fmt.Errorf(\"unexpected status %d\", resp.StatusCode)\n\t}\n\n\t// Determine MIME type from Content-Type header.\n\tct := resp.Header.Get(\"Content-Type\")\n\tmimeType, _, _ = mime.ParseMediaType(ct)\n\tif mimeType == \"\" {\n\t\tmimeType = \"application/octet-stream\"\n\t}\n\n\t// Only allow image content types (or octet-stream which we sniff later).\n\tif !strings.HasPrefix(mimeType, \"image/\") && mimeType != \"application/octet-stream\" {\n\t\treturn nil, \"\", fmt.Errorf(\"non-image content type: %s\", mimeType)\n\t}","sourceCodeStart":999,"sourceCodeEnd":1035,"githubUrl":"https://github.com/Tencent/WeKnora/blob/988cbb03305e055d8ebb7d46d9ac6cc0803cd074/internal/infrastructure/docparser/image_resolver.go#L999-L1035","documentation":"This error wraps a transport-level failure of the HTTP GET performed during remote image download — the request could not be completed at all. The underlying error (DNS, dial, TLS, timeout, context cancellation) is preserved via %w.","triggerScenarios":"downloadImage's client.Do fails: DNS resolution failure, connection refused, TLS handshake error, request timeout, or the context was cancelled while fetching the image.","commonSituations":"Image host unreachable or DNS broken, egress firewall blocking outbound requests, slow hosts exceeding the SSRF-safe client's timeout, caller cancelled the context mid-download.","solutions":["Unwrap and classify the error (net.Error timeout, DNS, TLS) before retrying","Verify the image host resolves and is reachable from the server (curl the URL)","Check egress rules/proxy/timeout settings on the SSRF-safe HTTP client","Retry only transient errors (timeouts, connection reset) with backoff"],"exampleFix":"// before\nresp, err := client.Do(req)\nif err != nil { return nil, \"\", fmt.Errorf(\"HTTP GET: %w\", err) }\n// after\nresp, err := client.Do(req)\nif err != nil {\n    var ne net.Error\n    if errors.As(err, &ne) && ne.Timeout() {\n        return nil, \"\", fmt.Errorf(\"HTTP GET (timeout): %w\", err)\n    }\n    return nil, \"\", fmt.Errorf(\"HTTP GET: %w\", err)\n}","handlingStrategy":"retry","validationCode":"// resolve host before fetching\nif _, err := net.LookupHost(url.Host); err != nil {\n    return fmt.Errorf(\"host does not resolve: %w\", err)\n}","typeGuard":null,"tryCatchPattern":"data, mimeType, err := downloadImage(ctx, client, remoteURL)\nif err != nil {\n    var ne net.Error\n    if errors.As(err, &ne) && ne.Timeout() {\n        // retry once with longer timeout\n    }\n    if errors.Is(err, context.Canceled) {\n        return nil, \"\", err // do not retry cancellation\n    }\n    return nil, \"\", fmt.Errorf(\"HTTP GET: %w\", err)\n}","preventionTips":["Configure adequate timeouts on the SSRF-safe client","Distinguish transient (timeout, reset) from permanent (DNS NXDOMAIN) errors before retrying","Verify egress firewall/proxy rules allow outbound image fetches","Honor context cancellation instead of retrying"],"tags":["http","network","timeout","dns","image"],"backgroundTag":"http-request-failed","analyzedSha":"988cbb03305e055d8ebb7d46d9ac6cc0803cd074","analyzedAt":"2026-09-02T14:41:08.344Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}