Tencent/WeKnora · error

fetch failed: %w

Error message

fetch failed: %w

What it means

Returned when c.httpClient.Do(req) fails, i.e. the HTTP transport itself failed before a response status was available. Because this client uses an SSRF-safe transport, the failure can be a DNS resolution error, TCP connect failure, TLS handshake failure, request timeout (20s context deadline), or the SSRF client refusing a resolved private/loopback address at connection time.

Source

Thrown at internal/datasource/connector/rss/client.go:79

		return nil, err
	}

	if withAuthHeaders {
		for k, v := range c.headers {
			req.Header.Set(k, v)
		}
	}
	if req.Header.Get("User-Agent") == "" {
		req.Header.Set("User-Agent", defaultUserAgent)
	}
	if req.Header.Get("Accept") == "" {
		req.Header.Set("Accept",
			"application/rss+xml, application/atom+xml, application/xml, text/xml, application/json, text/html;q=0.9, */*;q=0.8")
	}

	resp, err := c.httpClient.Do(req)
	if err != nil {
		return nil, fmt.Errorf("fetch failed: %w", err)
	}
	defer resp.Body.Close()

	if resp.StatusCode < 200 || resp.StatusCode >= 300 {
		return nil, fmt.Errorf("HTTP %d %s", resp.StatusCode, resp.Status)
	}

	body, err := io.ReadAll(io.LimitReader(resp.Body, maxSize))
	if err != nil {
		return nil, fmt.Errorf("read body failed: %w", err)
	}
	return body, nil
}

// fetchFeed retrieves the raw bytes of a feed document.
func (c *client) fetchFeed(ctx context.Context, feedURL string) ([]byte, error) {
	return c.fetch(ctx, feedURL, maxFeedSize, true)
}

View on GitHub (pinned to 988cbb0330)

Solutions

1. Check network reachability of the feed host (curl -v the URL from the server running the connector) and fix DNS/firewall issues.
2. If the error mentions a private/loopback address, the SSRF guard is blocking it — serve the feed on a public address or adjust the SSRF allowlist policy.
3. If the URL is correct but the server is slow or flaky, wrap fetchFeed in a retry with backoff; consider raising requestTimeout if 20s is consistently too short.
4. If certificates are expired, renew them on the feed server — do not disable TLS verification.

Example fix

// before
data, err := cli.fetchFeed(ctx, feedURL)
if err != nil {
    return err
}
// after
var data []byte
err := retry.Do(3, time.Second, func() error {
    var ferr error
    data, ferr = cli.fetchFeed(ctx, feedURL)
    return ferr
})

func reachable(raw string) error {
    u, _ := url.Parse(raw)
    conn, err := net.DialTimeout("tcp", net.JoinHostPort(u.Hostname(), portOr(u, "443")), 5*time.Second)
    if err != nil {
        return fmt.Errorf("host unreachable: %w", err)
    }
    conn.Close()
    return nil
}

data, err := cli.fetchFeed(ctx, feedURL)
if err != nil {
    var netErr net.Error
    if errors.As(err, &netErr) && netErr.Timeout() {
        // transient: retry with backoff
        return retryFetch(ctx, feedURL, 3)
    }
    return fmt.Errorf("feed %s permanently unreachable: %w", feedURL, err)
}

When it happens

Trigger: Feed host is unreachable (DNS NXDOMAIN), server refuses connections, TLS certificate invalid or expired, the 20-second requestTimeout elapses on a slow server, or the SSRF-safe dialer blocks the connection after DNS resolves to a private/loopback IP.

Common situations: Feed server is down or rate-limiting; self-hosted feeds behind a firewall; internal hostnames that the SSRF guard intentionally blocks; corporate proxies stripping CONNECT; expired certificates on small self-hosted blogs; feeds that take >20s to respond.

Understand the failure class

Background: 'Something went wrong' / 'Request failed (500)' / 'HTTP error! status: 404' — what failed HTTP requests actually mean and how to find the real cause — this error's family across 28 libraries.

Related errors

• HTTP GET: %w
• download zip: %w
• HTTP request: %w
• failed to execute Exa request: %w
• failed to execute request: %w

AI-assisted analysis of Tencent/WeKnora@988cbb0330 (2026-09-02). Data as JSON: /api/errors/b8739fa5a3509f07. Report an issue: GitHub.