vxcontrol/pentagi · error

unexpected status code: %d

Error message

unexpected status code: %d

What it means

DuckDuckGo returned a non-200 status on every retry attempt; after the final attempt the status code is reported. The code intentionally retries once per second (transient 5xx or soft blocks) but gives up after duckduckgoMaxRetries. Handle wraps it into a Fatal error so the orchestrator tries the next engine.

Source

Thrown at backend/pkg/tools/searchers/duckduckgo.go:175

		req.Header.Set("Accept-Language", "en-US,en;q=0.9")

		resp, err := client.Do(req)
		if err != nil {
			if attempt == duckduckgoMaxRetries-1 {
				return "", fmt.Errorf("failed to execute search after %d attempts: %w", duckduckgoMaxRetries, err)
			}
			select {
			case <-ctx.Done():
				return "", ctx.Err()
			case <-time.After(time.Second):
			}
			continue
		}

		if resp.StatusCode != http.StatusOK {
			resp.Body.Close()
			if attempt == duckduckgoMaxRetries-1 {
				return "", fmt.Errorf("unexpected status code: %d", resp.StatusCode)
			}
			select {
			case <-ctx.Done():
				return "", ctx.Err()
			case <-time.After(time.Second):
			}
			continue
		}

		body, err := io.ReadAll(resp.Body)
		resp.Body.Close()
		if err != nil {
			return "", fmt.Errorf("failed to read response body: %w", err)
		}

		response, err = d.parseHTMLResponse(body)
		if err != nil {
			return "", fmt.Errorf("failed to parse search response: %w", err)

View on GitHub (pinned to ea665308ba)

Solutions

1. Log the exact status code: 403/202 means bot mitigation — reduce query rate, rotate egress IP, or add proper cookies/session handling.
2. 429 means rate limiting — back off and lower concurrency of web_search calls.
3. 5xx are transient server issues — retry later or rely on the next engine in the fallback chain.
4. Check the response body (capture it before Close) — DuckDuckGo often explains blocks (e.g. anomaly page) in the HTML.

Example fix

// before
if attempt == duckduckgoMaxRetries-1 {
    return "", fmt.Errorf("unexpected status code: %d", resp.StatusCode)
}
// after
if attempt == duckduckgoMaxRetries-1 {
    return "", fmt.Errorf("unexpected status code: %d", resp.StatusCode) // consider logging a body snippet for diagnosis
}

// pre-flight: detect soft blocks before search
resp, err := client.PostForm(duckduckgoSearchURL, url.Values{"q": {"test"}})
if err == nil && (resp.StatusCode == 403 || resp.StatusCode == 202 || resp.StatusCode == 429) {
    // engine is bot-blocked or rate-limited; prefer fallback engine
}

type statusErr interface{ error; StatusCode() int }
var se statusErr
if errors.As(err, &se) && se.StatusCode() == 429 {
    // rate-limited: back off significantly before using this engine again
}

if err != nil {
    var fatal searchers.FatalError
    if errors.As(err, &fatal) && strings.Contains(err.Error(), "unexpected status code: 403") {
        log.Warn("duckduckgo bot-blocked; disabling engine for this run")
    }
    return err
}

When it happens

Trigger: Server responds 403 (bot detection / missing cookies), 202 (DuckDuckGo anomaly mitigation), 429 (rate limit), 5xx (server-side issue), or a captive-portal 302 that http.Client does not follow to a 200.

Common situations: Running from datacenter/VPS IPs that DuckDuckGo throttles; very high query volume without delays; user-agent blocked after DuckDuckGo HTML changes; corporate proxy injecting its own responses.

Related errors

• unexpected status code: %d
• duckduckgo search failed: %w
• %s (HTTP 429)
• unexpected status code: %d
• bearer scheme must be used

AI-assisted analysis of vxcontrol/pentagi@ea665308ba (2026-09-01). Data as JSON: /api/errors/e2774e1abebd4471. Report an issue: GitHub.