What causes "failed to unmarshal response: %w" in charmbracelet/crush?

Log the first bytes of body on failure to see whether it is HTML, empty, or truncated JSON.

failed to unmarshal response

Error message

failed to unmarshal response: %w

What it means

Wraps a json.Unmarshal error when decoding the Sourcegraph GraphQL JSON response into map[string]any (internal/agent/tools/sourcegraph.go:143-145). The HTTP response was 200 but the body is not valid JSON — e.g. an HTML error page, an empty body, or a truncated payload. A 200 with non-JSON content usually indicates an intermediary (proxy, auth wall) intercepting the response.

Solutions

Log the first bytes of body on failure to see whether it is HTML, empty, or truncated JSON.
Check for a proxy/auth wall intercepting sourcegraph.com and bypass or authenticate it.
Retry the request; a single truncation is usually transient.
Consider validating Content-Type is application/json before unmarshalling to produce a clearer error.

Example fix

// before
var result map[string]any
if err = json.Unmarshal(body, &result); err != nil {
    return fantasy.ToolResponse{}, fmt.Errorf("failed to unmarshal response: %w", err)
}

// after
var result map[string]any
if err = json.Unmarshal(body, &result); err != nil {
    return fantasy.ToolResponse{}, fmt.Errorf("failed to unmarshal response: %w (body: %.200q)", err, body)
}

Defensive patterns

Strategy: validation

Validate before calling

if ct := resp.Header.Get("Content-Type"); !strings.Contains(ct, "application/json") {
    return fmt.Errorf("expected JSON response, got Content-Type: %q", ct)
}
if len(body) == 0 {
    return fmt.Errorf("empty response body with status 200")
}

Try / catch

var result map[string]any
if err := json.Unmarshal(body, &result); err != nil {
    log.Printf("invalid JSON from sourcegraph, head: %.200q", body)
    return fmt.Errorf("failed to unmarshal response: %w", err)
}

Prevention

Check Content-Type and body prefix before unmarshalling to detect HTML/WAF pages.
Watch for proxies or auth walls that answer 200 with non-JSON bodies.
Retry once on truncation; it is usually transient.
Log body heads on failure — never the whole body — for fast diagnosis.

When it happens

Trigger: json.Unmarshal(body, &result) fails: response body is empty, truncated mid-stream (still yields invalid JSON), HTML from a proxy/captive portal served with status 200, or a WAF challenge page.

Common situations: Corporate proxies returning 200 HTML login pages; a load balancer serving an empty 200 during Sourcegraph incidents; connection truncation producing partial JSON; misconfigured local proxies (mitmproxy, Charles) mangling the body.

Understand the failure class

Background: "Invalid JSON response" and "Failed to parse response" errors: when an API answers 200 but the body isn't the JSON your library expected — this error's family across 28 libraries.

Related errors

AI-assisted analysis of charmbracelet/crush@7944b8e522 (2026-08-29). Data as JSON: /api/errors/b4f100ff558a1a65. Report an issue: GitHub.

Appendix: source

Thrown at internal/agent/tools/sourcegraph.go:144

			}
			defer resp.Body.Close()

			if resp.StatusCode != http.StatusOK {
				body, _ := io.ReadAll(resp.Body)
				if len(body) > 0 {
					return fantasy.NewTextErrorResponse(fmt.Sprintf("Request failed with status code: %d, response: %s", resp.StatusCode, string(body))), nil
				}

				return fantasy.NewTextErrorResponse(fmt.Sprintf("Request failed with status code: %d", resp.StatusCode)), nil
			}
			body, err := io.ReadAll(resp.Body)
			if err != nil {
				return fantasy.ToolResponse{}, fmt.Errorf("failed to read response body: %w", err)
			}

			var result map[string]any
			if err = json.Unmarshal(body, &result); err != nil {
				return fantasy.ToolResponse{}, fmt.Errorf("failed to unmarshal response: %w", err)
			}

			formattedResults, err := formatSourcegraphResults(result, params.ContextWindow, params.Count)
			if err != nil {
				return fantasy.NewTextErrorResponse("Failed to format results: " + err.Error()), nil
			}

			return fantasy.NewTextResponse(formattedResults), nil
		},
	)
}

func formatSourcegraphResults(result map[string]any, contextWindow, maxResults int) (string, error) {
	var buffer strings.Builder

	if writeSourcegraphErrors(&buffer, result) {
		return buffer.String(), nil
	}

View on GitHub (pinned to 7944b8e522)