vxcontrol/pentagi · error · Fatal

failed to decode response body: %v

Error message

failed to decode response body: %v

What it means

Thrown in tavily.parseHTTPResponse when Tavily answered HTTP 200 but the body could not be JSON-decoded into tavilySearchResult (json.NewDecoder(resp.Body).Decode failed). It is classified Fatal, so no retry or engine fallback from the error alone — the response is considered unusable. Usually means the body is HTML (error page/proxy block page), empty, or truncated.

Source

Thrown at backend/pkg/tools/searchers/tavily.go:146

	req = req.WithContext(ctx)
	req.Header.Set("Content-Type", "application/json")

	resp, err := client.Do(req)
	if err != nil {
		return "", Retryable(fmt.Errorf("failed to do request: %v", err), 0)
	}
	defer resp.Body.Close()

	return t.parseHTTPResponse(ctx, resp)
}

func (t *tavily) parseHTTPResponse(ctx context.Context, resp *http.Response) (string, error) {
	switch resp.StatusCode {
	case http.StatusOK:
		var respBody tavilySearchResult
		if err := json.NewDecoder(resp.Body).Decode(&respBody); err != nil {
			return "", Fatal(fmt.Errorf("failed to decode response body: %v", err))
		}
		return t.buildTavilyResult(ctx, &respBody), nil
	case http.StatusBadRequest:
		return "", Fatal(fmt.Errorf("request is invalid"))
	case http.StatusUnauthorized:
		return "", Fatal(fmt.Errorf("API key is wrong"))
	case http.StatusForbidden:
		return "", Fatal(fmt.Errorf("the endpoint requested is hidden for administrators only"))
	case http.StatusNotFound:
		return "", Fatal(fmt.Errorf("the specified endpoint could not be found"))
	case http.StatusMethodNotAllowed:
		return "", Fatal(fmt.Errorf("there need to try to access an endpoint with an invalid method"))
	case http.StatusTooManyRequests:
		return "", Retryable(fmt.Errorf("there are requesting too many results"), 0)
	case http.StatusInternalServerError:
		return "", Retryable(fmt.Errorf("there had a problem with our server. try again later"), 0)
	case http.StatusBadGateway:
		return "", Retryable(fmt.Errorf("there was a problem with the server. Please try again later"), 0)

View on GitHub (pinned to ea665308ba)

Solutions

1. Log/inspect the raw response body on decode failure (read a bounded prefix with io.LimitReader) to see what actually arrived.
2. Check proxy/MITM appliances in the path — an HTML login or block page with status 200 is the classic cause.
3. Re-run the same query with curl against api.tavily.com/search to compare the real body shape.
4. Check Tavily API changelog for schema changes to the /search response and update the tavilySearchResult/tavilyResult structs.
5. Consider reclassifying as Retryable for truncated-body cases (io.ErrUnexpectedEOF) so the orchestrator can fall back to another engine.

Example fix

// before
if err := json.NewDecoder(resp.Body).Decode(&respBody); err != nil {
    return "", Fatal(fmt.Errorf("failed to decode response body: %v", err))
}
// after
raw, _ := io.ReadAll(io.LimitReader(resp.Body, 4096))
if err := json.Unmarshal(raw, &respBody); err != nil {
    return "", Fatal(fmt.Errorf("failed to decode response body: %v (body prefix: %q)", err, string(raw[:min(len(raw), 256)])))
}

// verify the body looks like JSON before decoding
body, _ := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
if !json.Valid(body) {
    return "", Fatal(fmt.Errorf("non-JSON response body"))
}

func isValidTavilyResult(v any) bool {
    r, ok := v.(*tavilySearchResult)
    return ok && r != nil // schema fields are all optional per struct; presence of a parseable object is the gate
}

if err := json.NewDecoder(resp.Body).Decode(&respBody); err != nil {
    var serr *json.SyntaxError
    if errors.As(err, &serr) {
        // log body prefix + offset for diagnosis; fall back to another engine
    }
    return "", Fatal(err)
}

When it happens

Trigger: Tavily returns 200 with a non-JSON body: an intercepting proxy or captive portal HTML page, an empty body, a truncated response (connection cut mid-body), or an unexpected schema change causing json.UnmarshalTypeError.

Common situations: Corporate proxy injecting an HTML error/auth page with status 200; gzip/encoding issues when a custom transport strips Content-Encoding handling; Tavily API response schema drift after an API update; container memory/conn limits truncating the body.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• request is invalid
• failed to marshal provider config: %w
• failed to unmarshal search arguments: %w
• failed to marshal request body: %v
• failed to decode response body: %v

AI-assisted analysis of vxcontrol/pentagi@ea665308ba (2026-09-01). Data as JSON: /api/errors/27076ff79a1ef489. Report an issue: GitHub.