{"record":{"id":"b8739fa5a3509f07","repo":"Tencent/WeKnora","slug":"fetch-failed-w","errorCode":null,"errorMessage":"fetch failed: %w","messagePattern":"fetch failed: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"internal/datasource/connector/rss/client.go","lineNumber":79,"sourceCode":"\t\treturn nil, err\n\t}\n\n\tif withAuthHeaders {\n\t\tfor k, v := range c.headers {\n\t\t\treq.Header.Set(k, v)\n\t\t}\n\t}\n\tif req.Header.Get(\"User-Agent\") == \"\" {\n\t\treq.Header.Set(\"User-Agent\", defaultUserAgent)\n\t}\n\tif req.Header.Get(\"Accept\") == \"\" {\n\t\treq.Header.Set(\"Accept\",\n\t\t\t\"application/rss+xml, application/atom+xml, application/xml, text/xml, application/json, text/html;q=0.9, */*;q=0.8\")\n\t}\n\n\tresp, err := c.httpClient.Do(req)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"fetch failed: %w\", err)\n\t}\n\tdefer resp.Body.Close()\n\n\tif resp.StatusCode < 200 || resp.StatusCode >= 300 {\n\t\treturn nil, fmt.Errorf(\"HTTP %d %s\", resp.StatusCode, resp.Status)\n\t}\n\n\tbody, err := io.ReadAll(io.LimitReader(resp.Body, maxSize))\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"read body failed: %w\", err)\n\t}\n\treturn body, nil\n}\n\n// fetchFeed retrieves the raw bytes of a feed document.\nfunc (c *client) fetchFeed(ctx context.Context, feedURL string) ([]byte, error) {\n\treturn c.fetch(ctx, feedURL, maxFeedSize, true)\n}","sourceCodeStart":61,"sourceCodeEnd":97,"githubUrl":"https://github.com/Tencent/WeKnora/blob/988cbb03305e055d8ebb7d46d9ac6cc0803cd074/internal/datasource/connector/rss/client.go#L61-L97","documentation":"Returned when c.httpClient.Do(req) fails, i.e. the HTTP transport itself failed before a response status was available. Because this client uses an SSRF-safe transport, the failure can be a DNS resolution error, TCP connect failure, TLS handshake failure, request timeout (20s context deadline), or the SSRF client refusing a resolved private/loopback address at connection time.","triggerScenarios":"Feed host is unreachable (DNS NXDOMAIN), server refuses connections, TLS certificate invalid or expired, the 20-second requestTimeout elapses on a slow server, or the SSRF-safe dialer blocks the connection after DNS resolves to a private/loopback IP.","commonSituations":"Feed server is down or rate-limiting; self-hosted feeds behind a firewall; internal hostnames that the SSRF guard intentionally blocks; corporate proxies stripping CONNECT; expired certificates on small self-hosted blogs; feeds that take >20s to respond.","solutions":["Check network reachability of the feed host (curl -v the URL from the server running the connector) and fix DNS/firewall issues.","If the error mentions a private/loopback address, the SSRF guard is blocking it — serve the feed on a public address or adjust the SSRF allowlist policy.","If the URL is correct but the server is slow or flaky, wrap fetchFeed in a retry with backoff; consider raising requestTimeout if 20s is consistently too short.","If certificates are expired, renew them on the feed server — do not disable TLS verification."],"exampleFix":"// before\ndata, err := cli.fetchFeed(ctx, feedURL)\nif err != nil {\n    return err\n}\n// after\nvar data []byte\nerr := retry.Do(3, time.Second, func() error {\n    var ferr error\n    data, ferr = cli.fetchFeed(ctx, feedURL)\n    return ferr\n})","handlingStrategy":"retry","validationCode":"func reachable(raw string) error {\n    u, _ := url.Parse(raw)\n    conn, err := net.DialTimeout(\"tcp\", net.JoinHostPort(u.Hostname(), portOr(u, \"443\")), 5*time.Second)\n    if err != nil {\n        return fmt.Errorf(\"host unreachable: %w\", err)\n    }\n    conn.Close()\n    return nil\n}","typeGuard":null,"tryCatchPattern":"data, err := cli.fetchFeed(ctx, feedURL)\nif err != nil {\n    var netErr net.Error\n    if errors.As(err, &netErr) && netErr.Timeout() {\n        // transient: retry with backoff\n        return retryFetch(ctx, feedURL, 3)\n    }\n    return fmt.Errorf(\"feed %s permanently unreachable: %w\", feedURL, err)\n}","preventionTips":["Add health-check probes for feed hosts before full syncs.","Set explicit, generous timeouts and honor context cancellation.","Retry transient errors (timeouts, connection reset) with exponential backoff and jitter.","Expect SSRF-guard rejections for private IPs and document the allowlist policy for users."],"tags":["network","http","go","timeout","dns"],"backgroundTag":"http-request-failed","analyzedSha":"988cbb03305e055d8ebb7d46d9ac6cc0803cd074","analyzedAt":"2026-09-02T14:41:08.344Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}