charmbracelet/crush · error

failed to fetch URL: %w

Error message

failed to fetch URL: %w

What it means

The fetch tool wraps any error from the HTTP client's Do() call (network-level failures) with 'failed to fetch URL'. This happens before any HTTP response is received, so it covers DNS resolution failures, connection refusals/timeouts, TLS errors, and malformed URLs that escape request creation. The wrapped underlying error is preserved via %w.

Source

Thrown at internal/agent/tools/fetch.go:122

			if params.Timeout > 0 {
				if params.Timeout > maxFetchTimeoutSeconds {
					params.Timeout = maxFetchTimeoutSeconds
				}
				var cancel context.CancelFunc
				requestCtx, cancel = context.WithTimeout(ctx, time.Duration(params.Timeout)*time.Second)
				defer cancel()
			}

			req, err := http.NewRequestWithContext(requestCtx, "GET", params.URL, nil)
			if err != nil {
				return fantasy.ToolResponse{}, fmt.Errorf("failed to create request: %w", err)
			}

			req.Header.Set("User-Agent", "crush/1.0")

			resp, err := client.Do(req)
			if err != nil {
				return fantasy.ToolResponse{}, fmt.Errorf("failed to fetch URL: %w", err)
			}
			defer resp.Body.Close()

			if resp.StatusCode != http.StatusOK {
				return fantasy.NewTextErrorResponse(fmt.Sprintf("Request failed with status code: %d", resp.StatusCode)), nil
			}

			body, err := io.ReadAll(io.LimitReader(resp.Body, MaxFetchSize))
			if err != nil {
				return fantasy.NewTextErrorResponse("Failed to read response body: " + err.Error()), nil
			}

			content := string(body)

			validUTF8 := utf8.ValidString(content)
			if !validUTF8 {
				return fantasy.NewTextErrorResponse("Response content is not valid UTF-8"), nil
			}

View on GitHub (pinned to 7944b8e522)

Solutions

1. Verify the URL is reachable with `curl -v <url>` from the same machine
2. Check the wrapped cause (%w) to distinguish DNS vs connection vs TLS issues
3. If behind a proxy, check HTTP_PROXY/HTTPS_PROXY/NO_PROXY environment variables
4. For self-signed TLS, fix the certificate chain rather than disabling verification

Example fix

// before
resp, err := client.Do(req)
if err != nil {
    return fantasy.ToolResponse{}, fmt.Errorf("failed to fetch URL: %w", err)
}
// after
resp, err := client.Do(req)
if err != nil {
    var urlErr *url.Error
    if errors.As(err, &urlErr) {
        return fantasy.ToolResponse{}, fmt.Errorf("failed to fetch URL %s: %w", url, err)
    }
    return fantasy.ToolResponse{}, fmt.Errorf("failed to fetch URL: %w", err)
}

u, err := url.Parse(rawURL)
if err != nil || (u.Scheme != "http" && u.Scheme != "https") || u.Host == "" {
    return fmt.Errorf("invalid URL: %q", rawURL)
}

var dnsErr *net.DNSError
if errors.As(err, &dnsErr) {
    // host could not be resolved
}

resp, err := client.Do(req)
if err != nil {
    return fmt.Errorf("failed to fetch URL: %w", err) // inspect wrapped cause with errors.As/Is
}

When it happens

Trigger: client.Do(req) returns a non-nil error: DNS lookup failure for the hostname, connection refused or timed out, TLS handshake failure, or an unsupported/invalid URL scheme that passed http.NewRequestWithContext.

Common situations: Fetching a URL behind a firewall or proxy, offline environment, typo in hostname (e.g. missing scheme causing odd behavior or bad host), self-signed certificates, or an HTTP proxy misconfigured via HTTP_PROXY/HTTPS_PROXY env vars.

Understand the failure class

Background: 'Something went wrong' / 'Request failed (500)' / 'HTTP error! status: 404' — what failed HTTP requests actually mean and how to find the real cause — this error's family across 28 libraries.

Related errors

• failed to fetch URL: %w
• failed to fetch URL: %w
• failed to download from URL: %w
• failed to read response body: %w
• failed to make request: %w

AI-assisted analysis of charmbracelet/crush@7944b8e522 (2026-08-29). Data as JSON: /api/errors/4a30d54deab3407d. Report an issue: GitHub.