What causes "failed to unmarshal response: %w" in charmbracelet/crush?

Log the first bytes of body on failure to see whether it is HTML, empty, or truncated JSON.

failed to unmarshal response: %w

Error message

failed to unmarshal response: %w

What it means

Wraps a json.Unmarshal error when decoding the Sourcegraph GraphQL JSON response into map[string]any (internal/agent/tools/sourcegraph.go:143-145). The HTTP response was 200 but the body is not valid JSON — e.g. an HTML error page, an empty body, or a truncated payload. A 200 with non-JSON content usually indicates an intermediary (proxy, auth wall) intercepting the response.

Source

Thrown at internal/agent/tools/sourcegraph.go:144

			}
			defer resp.Body.Close()

			if resp.StatusCode != http.StatusOK {
				body, _ := io.ReadAll(resp.Body)
				if len(body) > 0 {
					return fantasy.NewTextErrorResponse(fmt.Sprintf("Request failed with status code: %d, response: %s", resp.StatusCode, string(body))), nil
				}

				return fantasy.NewTextErrorResponse(fmt.Sprintf("Request failed with status code: %d", resp.StatusCode)), nil
			}
			body, err := io.ReadAll(resp.Body)
			if err != nil {
				return fantasy.ToolResponse{}, fmt.Errorf("failed to read response body: %w", err)
			}

			var result map[string]any
			if err = json.Unmarshal(body, &result); err != nil {
				return fantasy.ToolResponse{}, fmt.Errorf("failed to unmarshal response: %w", err)
			}

			formattedResults, err := formatSourcegraphResults(result, params.ContextWindow, params.Count)
			if err != nil {
				return fantasy.NewTextErrorResponse("Failed to format results: " + err.Error()), nil
			}

			return fantasy.NewTextResponse(formattedResults), nil
		},
	)
}

func formatSourcegraphResults(result map[string]any, contextWindow, maxResults int) (string, error) {
	var buffer strings.Builder

	if writeSourcegraphErrors(&buffer, result) {
		return buffer.String(), nil
	}

View on GitHub (pinned to 7944b8e522)

Solutions

Log the first bytes of body on failure to see whether it is HTML, empty, or truncated JSON.
Check for a proxy/auth wall intercepting sourcegraph.com and bypass or authenticate it.
Retry the request; a single truncation is usually transient.
Consider validating Content-Type is application/json before unmarshalling to produce a clearer error.

Example fix

// before
var result map[string]any
if err = json.Unmarshal(body, &result); err != nil {
    return fantasy.ToolResponse{}, fmt.Errorf("failed to unmarshal response: %w", err)
}

// after
var result map[string]any
if err = json.Unmarshal(body, &result); err != nil {
    return fantasy.ToolResponse{}, fmt.Errorf("failed to unmarshal response: %w (body: %.200q)", err, body)
}

Defensive patterns

Strategy: validation

Validate before calling

if ct := resp.Header.Get("Content-Type"); !strings.Contains(ct, "application/json") {
    return fmt.Errorf("expected JSON response, got Content-Type: %q", ct)
}
if len(body) == 0 {
    return fmt.Errorf("empty response body with status 200")
}

Try / catch

var result map[string]any
if err := json.Unmarshal(body, &result); err != nil {
    log.Printf("invalid JSON from sourcegraph, head: %.200q", body)
    return fmt.Errorf("failed to unmarshal response: %w", err)
}

Prevention

Check Content-Type and body prefix before unmarshalling to detect HTML/WAF pages.
Watch for proxies or auth walls that answer 200 with non-JSON bodies.
Retry once on truncation; it is usually transient.
Log body heads on failure — never the whole body — for fast diagnosis.

When it happens

Trigger: json.Unmarshal(body, &result) fails: response body is empty, truncated mid-stream (still yields invalid JSON), HTML from a proxy/captive portal served with status 200, or a WAF challenge page.

Common situations: Corporate proxies returning 200 HTML login pages; a load balancer serving an empty 200 during Sourcegraph incidents; connection truncation producing partial JSON; misconfigured local proxies (mitmproxy, Charles) mangling the body.

Understand the failure class

Background: "Invalid JSON response" and "Failed to parse response" errors: when an API answers 200 but the body isn't the JSON your library expected — this error's family across 28 libraries.

Related errors

AI-assisted analysis of charmbracelet/crush@7944b8e522 (2026-08-29). Data as JSON: /api/errors/b4f100ff558a1a65. Report an issue: GitHub.