{"record":{"id":"a1ddbbcdc9aa0321","repo":"charmbracelet/crush","slug":"failed-to-read-response-body-w","errorCode":null,"errorMessage":"failed to read response body: %w","messagePattern":"failed to read response body: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"internal/agent/tools/fetch_helpers.go","lineNumber":49,"sourceCode":"\t// Use realistic browser headers for better compatibility.\n\treq.Header.Set(\"User-Agent\", BrowserUserAgent)\n\treq.Header.Set(\"Accept\", \"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\")\n\treq.Header.Set(\"Accept-Language\", \"en-US,en;q=0.5\")\n\n\tresp, err := client.Do(req)\n\tif err != nil {\n\t\treturn \"\", fmt.Errorf(\"failed to fetch URL: %w\", err)\n\t}\n\tdefer resp.Body.Close()\n\n\tif resp.StatusCode != http.StatusOK {\n\t\treturn \"\", fmt.Errorf(\"request failed with status code: %d\", resp.StatusCode)\n\t}\n\n\tmaxSize := int64(5 * 1024 * 1024) // 5MB\n\tbody, err := io.ReadAll(io.LimitReader(resp.Body, maxSize))\n\tif err != nil {\n\t\treturn \"\", fmt.Errorf(\"failed to read response body: %w\", err)\n\t}\n\n\tcontent := string(body)\n\n\tif !utf8.ValidString(content) {\n\t\treturn \"\", errors.New(\"response content is not valid UTF-8\")\n\t}\n\n\tcontentType := resp.Header.Get(\"Content-Type\")\n\n\t// Convert HTML to markdown for better AI processing.\n\tif strings.Contains(contentType, \"text/html\") {\n\t\t// Remove noisy elements before conversion.\n\t\tcleanedHTML := removeNoisyElements(content)\n\t\tmarkdown, err := ConvertHTMLToMarkdown(cleanedHTML)\n\t\tif err != nil {\n\t\t\treturn \"\", fmt.Errorf(\"failed to convert HTML to markdown: %w\", err)\n\t\t}","sourceCodeStart":31,"sourceCodeEnd":67,"githubUrl":"https://github.com/charmbracelet/crush/blob/7944b8e52225d8805e31eacbf7ef24856b0dfb7a/internal/agent/tools/fetch_helpers.go#L31-L67","documentation":"Reading the response body (bounded by a 5MB LimitReader) failed with an I/O error. This indicates the connection broke mid-transfer or the server terminated the stream early. The 5MB limit itself does not cause this error — only genuine read failures do.","triggerScenarios":"io.ReadAll(io.LimitReader(resp.Body, 5MB)) returns an error: connection reset by peer, unexpected EOF, TLS record failure mid-body, or server closing the connection prematurely.","commonSituations":"Unstable networks or mobile connections, proxies that kill long transfers, very large responses on flaky CDNs, keep-alive connections dropped by load balancers mid-response.","solutions":["Retry the fetch — transient connection resets are the most common cause","Verify with curl whether the URL serves a complete response consistently","Check for proxy/CDN body-size or time limits that truncate transfers","If the body may exceed 5MB, the LimitReader silently truncates — that is separate; genuine read errors need a retry"],"exampleFix":"// before\nbody, err := io.ReadAll(io.LimitReader(resp.Body, maxSize))\nif err != nil {\n    return \"\", fmt.Errorf(\"failed to read response body: %w\", err)\n}\n// after\nbody, err := io.ReadAll(io.LimitReader(resp.Body, maxSize))\nif err != nil {\n    if errors.Is(err, io.ErrUnexpectedEOF) {\n        return \"\", fmt.Errorf(\"connection closed mid-transfer, retry fetch: %w\", err)\n    }\n    return \"\", fmt.Errorf(\"failed to read response body: %w\", err)\n}","handlingStrategy":"retry","validationCode":"null","typeGuard":"if errors.Is(err, io.ErrUnexpectedEOF) || errors.Is(err, syscall.ECONNRESET) {\n    // truncated transfer, retry\n}","tryCatchPattern":"body, err := io.ReadAll(io.LimitReader(resp.Body, maxSize))\nif err != nil {\n    return \"\", fmt.Errorf(\"failed to read response body: %w\", err)\n}","preventionTips":["Retry transient body-read failures with backoff","Keep the LimitReader cap to avoid OOM on huge responses","Check network stability/proxy timeouts for recurring truncation","Validate content (e.g. UTF-8) after read, as the code already does"],"tags":["io","network","http-body"],"backgroundTag":"connection-reset-during-transfer","analyzedSha":"7944b8e52225d8805e31eacbf7ef24856b0dfb7a","analyzedAt":"2026-08-29T12:48:59.079Z","schemaVersion":2},"datasetVersion":"2026-08-29T17:17:51.833Z"}