{"record":{"id":"e2774e1abebd4471","repo":"vxcontrol/pentagi","slug":"unexpected-status-code-d","errorCode":null,"errorMessage":"unexpected status code: %d","messagePattern":"unexpected status code: (.+?)","errorType":"http","errorClass":null,"httpStatus":null,"severity":"error","filePath":"backend/pkg/tools/searchers/duckduckgo.go","lineNumber":175,"sourceCode":"\t\treq.Header.Set(\"Accept-Language\", \"en-US,en;q=0.9\")\n\n\t\tresp, err := client.Do(req)\n\t\tif err != nil {\n\t\t\tif attempt == duckduckgoMaxRetries-1 {\n\t\t\t\treturn \"\", fmt.Errorf(\"failed to execute search after %d attempts: %w\", duckduckgoMaxRetries, err)\n\t\t\t}\n\t\t\tselect {\n\t\t\tcase <-ctx.Done():\n\t\t\t\treturn \"\", ctx.Err()\n\t\t\tcase <-time.After(time.Second):\n\t\t\t}\n\t\t\tcontinue\n\t\t}\n\n\t\tif resp.StatusCode != http.StatusOK {\n\t\t\tresp.Body.Close()\n\t\t\tif attempt == duckduckgoMaxRetries-1 {\n\t\t\t\treturn \"\", fmt.Errorf(\"unexpected status code: %d\", resp.StatusCode)\n\t\t\t}\n\t\t\tselect {\n\t\t\tcase <-ctx.Done():\n\t\t\t\treturn \"\", ctx.Err()\n\t\t\tcase <-time.After(time.Second):\n\t\t\t}\n\t\t\tcontinue\n\t\t}\n\n\t\tbody, err := io.ReadAll(resp.Body)\n\t\tresp.Body.Close()\n\t\tif err != nil {\n\t\t\treturn \"\", fmt.Errorf(\"failed to read response body: %w\", err)\n\t\t}\n\n\t\tresponse, err = d.parseHTMLResponse(body)\n\t\tif err != nil {\n\t\t\treturn \"\", fmt.Errorf(\"failed to parse search response: %w\", err)","sourceCodeStart":157,"sourceCodeEnd":193,"githubUrl":"https://github.com/vxcontrol/pentagi/blob/ea665308baaff015b226f308438a68d929d0f29b/backend/pkg/tools/searchers/duckduckgo.go#L157-L193","documentation":"DuckDuckGo returned a non-200 status on every retry attempt; after the final attempt the status code is reported. The code intentionally retries once per second (transient 5xx or soft blocks) but gives up after duckduckgoMaxRetries. Handle wraps it into a Fatal error so the orchestrator tries the next engine.","triggerScenarios":"Server responds 403 (bot detection / missing cookies), 202 (DuckDuckGo anomaly mitigation), 429 (rate limit), 5xx (server-side issue), or a captive-portal 302 that http.Client does not follow to a 200.","commonSituations":"Running from datacenter/VPS IPs that DuckDuckGo throttles; very high query volume without delays; user-agent blocked after DuckDuckGo HTML changes; corporate proxy injecting its own responses.","solutions":["Log the exact status code: 403/202 means bot mitigation — reduce query rate, rotate egress IP, or add proper cookies/session handling.","429 means rate limiting — back off and lower concurrency of web_search calls.","5xx are transient server issues — retry later or rely on the next engine in the fallback chain.","Check the response body (capture it before Close) — DuckDuckGo often explains blocks (e.g. anomaly page) in the HTML."],"exampleFix":"// before\nif attempt == duckduckgoMaxRetries-1 {\n    return \"\", fmt.Errorf(\"unexpected status code: %d\", resp.StatusCode)\n}\n// after\nif attempt == duckduckgoMaxRetries-1 {\n    return \"\", fmt.Errorf(\"unexpected status code: %d\", resp.StatusCode) // consider logging a body snippet for diagnosis\n}","handlingStrategy":"fallback","validationCode":"// pre-flight: detect soft blocks before search\nresp, err := client.PostForm(duckduckgoSearchURL, url.Values{\"q\": {\"test\"}})\nif err == nil && (resp.StatusCode == 403 || resp.StatusCode == 202 || resp.StatusCode == 429) {\n    // engine is bot-blocked or rate-limited; prefer fallback engine\n}","typeGuard":"type statusErr interface{ error; StatusCode() int }\nvar se statusErr\nif errors.As(err, &se) && se.StatusCode() == 429 {\n    // rate-limited: back off significantly before using this engine again\n}","tryCatchPattern":"if err != nil {\n    var fatal searchers.FatalError\n    if errors.As(err, &fatal) && strings.Contains(err.Error(), \"unexpected status code: 403\") {\n        log.Warn(\"duckduckgo bot-blocked; disabling engine for this run\")\n    }\n    return err\n}","preventionTips":["Throttle and jitter requests to stay under bot-detection thresholds","Avoid datacenter IPs known to be throttled; rotate egress if needed","Capture the response body on non-200 to detect challenge pages","Keep alternate engines configured in the fallbackStrategy chain"],"tags":["http","status-code","rate-limit","duckduckgo"],"backgroundTag":"http-403-forbidden","analyzedSha":"ea665308baaff015b226f308438a68d929d0f29b","analyzedAt":"2026-09-01T14:16:31.421Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}