Tencent/WeKnora · error

HTTP GET: %w

Error message

HTTP GET: %w

What it means

DownloadBytes wraps any transport-level failure of the defaultHTTPClient.Get call (60s timeout client) as "HTTP GET: <underlying error>". It means the request could not be completed — DNS, connection, TLS, or timeout — not an HTTP error status.

Source

Thrown at internal/utils/httputil.go:27

)

var defaultHTTPClient = NewSSRFSafeHTTPClient(SSRFSafeHTTPClientConfig{
	Timeout:      60 * time.Second,
	MaxRedirects: 10,
})

// DownloadBytes fetches the content at the given HTTP(S) URL and returns the
// raw bytes. It reuses a package-level http.Client with a 60-second timeout.
func DownloadBytes(url string) ([]byte, error) {
	if !strings.HasPrefix(url, "http://") && !strings.HasPrefix(url, "https://") {
		return nil, fmt.Errorf("unsupported URL scheme: %s", url)
	}
	if err := ValidateURLForSSRF(url); err != nil {
		return nil, fmt.Errorf("URL rejected by SSRF policy: %w", err)
	}
	resp, err := defaultHTTPClient.Get(url)
	if err != nil {
		return nil, fmt.Errorf("HTTP GET: %w", err)
	}
	defer resp.Body.Close()
	if resp.StatusCode != http.StatusOK {
		return nil, fmt.Errorf("HTTP %d for %s", resp.StatusCode, url)
	}
	data, err := io.ReadAll(resp.Body)
	if err != nil {
		return nil, fmt.Errorf("read body: %w", err)
	}
	return data, nil
}

View on GitHub (pinned to 988cbb0330)

Solutions

1. Read the wrapped cause in the error to identify DNS vs connection vs TLS vs timeout
2. Verify the URL's host resolves and is reachable from the runtime environment (curl it)
3. Check egress/firewall/proxy settings in containers or CI
4. For large/slow downloads, use a client with a longer timeout than the default 60s
5. Add retry with backoff for transient network failures

Example fix

// before
resp := DownloadBytes(url) // fails: HTTP GET: dial tcp: i/o timeout
// after
if err != nil {
    var netErr net.Error
    if errors.As(err, &netErr) && netErr.Timeout() {
        err = retryWithBackoff(func() error { _, err = DownloadBytes(url); return err })
    }
}

if !isHTTPURL(raw) { return fmt.Errorf("bad url") }
// pre-check reachability where appropriate:
conn, err := net.DialTimeout("tcp", net.JoinHostPort(host, port), 5*time.Second)
if err != nil { return fmt.Errorf("host unreachable: %w", err) }
conn.Close()

func isTimeoutErr(err error) bool {
    var ne net.Error
    return errors.As(err, &ne) && ne.Timeout()
}

data, err := DownloadBytes(url)
if err != nil {
    if strings.Contains(err.Error(), "HTTP GET:") {
        var ne net.Error
        if errors.As(err, &ne) && ne.Timeout() {
            // retry with longer deadline
        } else {
            // DNS/connect/TLS problem — check host reachability
        }
    }
}

When it happens

Trigger: Calling DownloadBytes when the host doesn't resolve, the connection is refused/dropped, TLS handshake fails, or the request exceeds the client's 60-second timeout.

Common situations: Target host down or DNS misconfigured, firewall egress blocks, expired/invalid TLS certificates on the target, slow endpoints exceeding the 60s timeout, or no network from the running environment (containers, CI).

Understand the failure class

Background: 'Something went wrong' / 'Request failed (500)' / 'HTTP error! status: 404' — what failed HTTP requests actually mean and how to find the real cause — this error's family across 28 libraries.

Related errors

• fetch failed: %w
• download zip: %w
• failed to execute Exa request: %w
• service unreachable: %w
• read body failed: %w

AI-assisted analysis of Tencent/WeKnora@988cbb0330 (2026-09-02). Data as JSON: /api/errors/604e7f3d555b1a20. Report an issue: GitHub.