vxcontrol/pentagi · error · Fatal

failed to decode Sploitus response: %w

Error message

failed to decode Sploitus response: %w

What it means

Decoding the Sploitus JSON response into sploitusResponse failed — the body was not valid JSON or did not match the expected shape; the searcher returns Fatal since retrying the identical request will likely produce the same body. Cloudflare HTML challenge pages or empty bodies commonly cause this.

Source

Thrown at backend/pkg/tools/searchers/sploitus.go:175

	defer resp.Body.Close()

	// Sploitus API returns 499 (and sometimes 422) when its rate limit is temporarily
	// exceeded — a transient condition that may clear on retry.
	if resp.StatusCode == 499 || resp.StatusCode == 422 {
		return "", Retryable(fmt.Errorf("Sploitus API rate limit exceeded (HTTP %d), please try again later", resp.StatusCode), 0)
	}

	if resp.StatusCode != http.StatusOK {
		err := fmt.Errorf("Sploitus API returned HTTP %d", resp.StatusCode)
		if resp.StatusCode == http.StatusTooManyRequests || resp.StatusCode >= 500 {
			return "", Retryable(err, 0)
		}
		return "", Fatal(err)
	}

	var apiResp sploitusResponse
	if err := json.NewDecoder(resp.Body).Decode(&apiResp); err != nil {
		return "", Fatal(fmt.Errorf("failed to decode Sploitus response: %w", err))
	}

	return formatSploitusResults(query, exploitType, limit, apiResp), nil
}

// IsAvailable returns true if the Sploitus tool is enabled and configured
func (s *sploitus) IsAvailable() bool {
	return s.enabled()
}

func (s *sploitus) enabled() bool {
	return s.cfg != nil && s.cfg.SploitusEnabled
}

// sploitusRequest is the JSON body sent to the Sploitus search API
type sploitusRequest struct {
	Query  string `json:"query"`
	Type   string `json:"type"`

View on GitHub (pinned to ea665308ba)

Solutions

1. Capture and inspect the raw response body to see whether it is HTML, empty, or JSON
2. If it is a Cloudflare challenge, switch egress IP/proxy — headers alone may no longer suffice
3. Compare the body against sploitusResponse fields and update the struct to the current API schema
4. Read the body into memory first and log a snippet before decoding for easier diagnosis

Example fix

// before
var apiResp sploitusResponse
json.NewDecoder(resp.Body).Decode(&apiResp)
// after
raw, _ := io.ReadAll(resp.Body)
log.Printf("sploitus raw body: %.200s", raw)
var apiResp sploitusResponse
json.Unmarshal(raw, &apiResp)

raw, err := io.ReadAll(resp.Body)
if err != nil || len(bytes.TrimSpace(raw)) == 0 {
    return fmt.Errorf("empty or unreadable Sploitus body")
}
if !bytes.HasPrefix(bytes.TrimSpace(raw), []byte("{")) {
    return fmt.Errorf("non-JSON body (likely Cloudflare page): %.100s", raw)
}

var apiResp sploitusResponse
if err := json.NewDecoder(resp.Body).Decode(&apiResp); err != nil {
    return "", Fatal(fmt.Errorf("failed to decode Sploitus response: %w", err))
}

When it happens

Trigger: Handle() calls json.NewDecoder(resp.Body).Decode(&apiResp) after a 200 response whose body is HTML (Cloudflare interstitial), empty, truncated, or JSON with an incompatible schema.

Common situations: Cloudflare returns 200 with a challenge page, a proxy/SSL appliance rewrites the body, network truncation mid-body, or Sploitus changes its response schema (field type changes like string vs number).

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• Sploitus API returned HTTP %d
• knowledge: marshal cmetadata: %w
• failed to unmarshal primary agent msg chain %d: %w
• failed to marshal provider config: %w
• failed to parse get_flow_status args: %w

AI-assisted analysis of vxcontrol/pentagi@ea665308ba (2026-09-01). Data as JSON: /api/errors/fd1f29f5f1ea0044. Report an issue: GitHub.