charmbracelet/crush · error

failed to read response body: %w

Error message

failed to read response body: %w

What it means

Reading the response body (bounded by a 5MB LimitReader) failed with an I/O error. This indicates the connection broke mid-transfer or the server terminated the stream early. The 5MB limit itself does not cause this error — only genuine read failures do.

Source

Thrown at internal/agent/tools/fetch_helpers.go:49

	// Use realistic browser headers for better compatibility.
	req.Header.Set("User-Agent", BrowserUserAgent)
	req.Header.Set("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8")
	req.Header.Set("Accept-Language", "en-US,en;q=0.5")

	resp, err := client.Do(req)
	if err != nil {
		return "", fmt.Errorf("failed to fetch URL: %w", err)
	}
	defer resp.Body.Close()

	if resp.StatusCode != http.StatusOK {
		return "", fmt.Errorf("request failed with status code: %d", resp.StatusCode)
	}

	maxSize := int64(5 * 1024 * 1024) // 5MB
	body, err := io.ReadAll(io.LimitReader(resp.Body, maxSize))
	if err != nil {
		return "", fmt.Errorf("failed to read response body: %w", err)
	}

	content := string(body)

	if !utf8.ValidString(content) {
		return "", errors.New("response content is not valid UTF-8")
	}

	contentType := resp.Header.Get("Content-Type")

	// Convert HTML to markdown for better AI processing.
	if strings.Contains(contentType, "text/html") {
		// Remove noisy elements before conversion.
		cleanedHTML := removeNoisyElements(content)
		markdown, err := ConvertHTMLToMarkdown(cleanedHTML)
		if err != nil {
			return "", fmt.Errorf("failed to convert HTML to markdown: %w", err)
		}

View on GitHub (pinned to 7944b8e522)

Solutions

1. Retry the fetch — transient connection resets are the most common cause
2. Verify with curl whether the URL serves a complete response consistently
3. Check for proxy/CDN body-size or time limits that truncate transfers
4. If the body may exceed 5MB, the LimitReader silently truncates — that is separate; genuine read errors need a retry

Example fix

// before
body, err := io.ReadAll(io.LimitReader(resp.Body, maxSize))
if err != nil {
    return "", fmt.Errorf("failed to read response body: %w", err)
}
// after
body, err := io.ReadAll(io.LimitReader(resp.Body, maxSize))
if err != nil {
    if errors.Is(err, io.ErrUnexpectedEOF) {
        return "", fmt.Errorf("connection closed mid-transfer, retry fetch: %w", err)
    }
    return "", fmt.Errorf("failed to read response body: %w", err)
}

null

if errors.Is(err, io.ErrUnexpectedEOF) || errors.Is(err, syscall.ECONNRESET) {
    // truncated transfer, retry
}

body, err := io.ReadAll(io.LimitReader(resp.Body, maxSize))
if err != nil {
    return "", fmt.Errorf("failed to read response body: %w", err)
}

When it happens

Trigger: io.ReadAll(io.LimitReader(resp.Body, 5MB)) returns an error: connection reset by peer, unexpected EOF, TLS record failure mid-body, or server closing the connection prematurely.

Common situations: Unstable networks or mobile connections, proxies that kill long transfers, very large responses on flaky CDNs, keep-alive connections dropped by load balancers mid-response.

Related errors

• failed to write file: %w
• failed to read response: %w
• failed to read response body: %w
• empty providers list from catwalk
• lost connection to the crush server

AI-assisted analysis of charmbracelet/crush@7944b8e522 (2026-08-29). Data as JSON: /api/errors/a1ddbbcdc9aa0321. Report an issue: GitHub.