{"record":{"id":"e95d57afaee050dc","repo":"charmbracelet/crush","slug":"request-failed-with-status-code-d","errorCode":null,"errorMessage":"request failed with status code: %d","messagePattern":"request failed with status code: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"internal/agent/tools/fetch_helpers.go","lineNumber":43,"sourceCode":"func FetchURLAndConvert(ctx context.Context, client *http.Client, url string) (string, error) {\n\treq, err := http.NewRequestWithContext(ctx, \"GET\", url, nil)\n\tif err != nil {\n\t\treturn \"\", fmt.Errorf(\"failed to create request: %w\", err)\n\t}\n\n\t// Use realistic browser headers for better compatibility.\n\treq.Header.Set(\"User-Agent\", BrowserUserAgent)\n\treq.Header.Set(\"Accept\", \"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\")\n\treq.Header.Set(\"Accept-Language\", \"en-US,en;q=0.5\")\n\n\tresp, err := client.Do(req)\n\tif err != nil {\n\t\treturn \"\", fmt.Errorf(\"failed to fetch URL: %w\", err)\n\t}\n\tdefer resp.Body.Close()\n\n\tif resp.StatusCode != http.StatusOK {\n\t\treturn \"\", fmt.Errorf(\"request failed with status code: %d\", resp.StatusCode)\n\t}\n\n\tmaxSize := int64(5 * 1024 * 1024) // 5MB\n\tbody, err := io.ReadAll(io.LimitReader(resp.Body, maxSize))\n\tif err != nil {\n\t\treturn \"\", fmt.Errorf(\"failed to read response body: %w\", err)\n\t}\n\n\tcontent := string(body)\n\n\tif !utf8.ValidString(content) {\n\t\treturn \"\", errors.New(\"response content is not valid UTF-8\")\n\t}\n\n\tcontentType := resp.Header.Get(\"Content-Type\")\n\n\t// Convert HTML to markdown for better AI processing.\n\tif strings.Contains(contentType, \"text/html\") {","sourceCodeStart":25,"sourceCodeEnd":61,"githubUrl":"https://github.com/charmbracelet/crush/blob/7944b8e52225d8805e31eacbf7ef24856b0dfb7a/internal/agent/tools/fetch_helpers.go#L25-L61","documentation":"The server responded, but with a non-200 status code. FetchURLAndConvert treats anything other than HTTP 200 as an error and reports the numeric status. Unlike error 130/132 the request itself succeeded — the remote server rejected or redirected it without following to a 200.","triggerScenarios":"The fetched URL returns 404 (missing page), 403 (blocked/bot-detected), 500 (server error), 301/302 loop, or 429 rate-limit — any status != http.StatusOK after redirects are followed.","commonSituations":"URLs that require authentication, sites blocking the client User-Agent, moved/removed pages, API endpoints requiring tokens, or Cloudflare bot protection returning 403.","solutions":["Verify the URL works in a browser or with `curl -I <url>` to see the real status","Use a current, valid URL (404 means the page moved or never existed)","Check whether the site requires auth or API keys and use the appropriate authenticated endpoint","If bot-blocked (403 from CDN), the browser-like User-Agent already set may need cookies or a different fetch strategy"],"exampleFix":"// before\nif resp.StatusCode != http.StatusOK {\n    return \"\", fmt.Errorf(\"request failed with status code: %d\", resp.StatusCode)\n}\n// after\nif resp.StatusCode != http.StatusOK {\n    return \"\", fmt.Errorf(\"request failed with status %d for %s\", resp.StatusCode, url)\n}","handlingStrategy":"type-guard","validationCode":"resp, err := http.Head(url) // or curl -I\nif err == nil && resp.StatusCode != http.StatusOK {\n    return fmt.Errorf(\"URL returns status %d; not fetchable\", resp.StatusCode)\n}","typeGuard":"if resp.StatusCode != http.StatusOK {\n    // handle 4xx/5xx explicitly before reading body\n}","tryCatchPattern":"if resp.StatusCode != http.StatusOK {\n    return \"\", fmt.Errorf(\"request failed with status code: %d\", resp.StatusCode)\n}","preventionTips":["Check status before parsing the body","Handle redirects and 404/410 by verifying the URL is current","Use authenticated endpoints when the resource requires auth","Distinguish 429 rate-limits (retry later) from 403 (do not retry)"],"tags":["http","status-code","server-error"],"backgroundTag":"http-non-200-response","analyzedSha":"7944b8e52225d8805e31eacbf7ef24856b0dfb7a","analyzedAt":"2026-08-29T12:48:59.079Z","schemaVersion":2},"datasetVersion":"2026-08-29T17:17:51.833Z"}