{"record":{"id":"5c2a030f600cf0a2","repo":"can1357/oh-my-pi","slug":"duckduckgo-html-error-page-status","errorCode":null,"errorMessage":"DuckDuckGo HTML error (${page.status})","messagePattern":"DuckDuckGo HTML error \\((.+?)\\)","errorType":"exception","errorClass":"SearchProviderError","httpStatus":null,"severity":"error","filePath":"packages/coding-agent/src/web/search/providers/duckduckgo.ts","lineNumber":321,"sourceCode":"\nasync function callDuckDuckGoHtml(params: SearchParams, form: URLSearchParams, signal: AbortSignal): Promise<string> {\n\tconst page = await browserFetch(DUCKDUCKGO_HTML_URL, {\n\t\tfetch: params.fetch ?? fetch,\n\t\tsignal,\n\t\ttimeoutMs: params.timeoutMs,\n\t\treferer: \"https://html.duckduckgo.com/\",\n\t\tinit: {\n\t\t\tmethod: \"POST\",\n\t\t\tbody: form.toString(),\n\t\t},\n\t\theaders: { \"Content-Type\": \"application/x-www-form-urlencoded\" },\n\t});\n\n\tconst body = page.html;\n\tif (page.status < 200 || page.status >= 300) {\n\t\tconst classified = classifyProviderHttpError(\"duckduckgo\", page.status, body);\n\t\tif (classified) throw classified;\n\t\tthrow new SearchProviderError(\"duckduckgo\", `DuckDuckGo HTML error (${page.status})`, page.status);\n\t}\n\n\tif (isAnomalyResponse(body)) {\n\t\tthrow new SearchProviderError(\n\t\t\t\"duckduckgo\",\n\t\t\t\"DuckDuckGo blocked the request with a bot-detection challenge. DuckDuckGo throttles automated HTML searches from datacenter/shared-egress IPs; configure a credentialed provider such as Brave, Tavily, Exa, or Kagi for reliable web search.\",\n\t\t\t429,\n\t\t);\n\t}\n\n\treturn body;\n}\n\n/** Execute a DuckDuckGo web search via the no-JS HTML frontend. */\nexport async function searchDuckDuckGo(params: SearchParams): Promise<SearchResponse> {\n\tconst numResults = clampNumResults(params.numSearchResults ?? params.limit, DEFAULT_NUM_RESULTS, MAX_NUM_RESULTS);\n\tconst signal = withHardTimeout(params.signal, params.timeoutMs);\n\tconst sources: SearchSource[] = [];","sourceCodeStart":303,"sourceCodeEnd":339,"githubUrl":"https://github.com/can1357/oh-my-pi/blob/969062200754ea02cfac922e5ebb8c608c079e15/packages/coding-agent/src/web/search/providers/duckduckgo.ts#L303-L339","documentation":"callDuckDuckGoHtml POSTs the query to DuckDuckGo's html.duckduckgo.com frontend via browserFetch. Any non-2xx status that classifyProviderHttpError cannot map to a more specific error becomes a SearchProviderError carrying the raw HTTP status, so callers can react to rate limits, outages, or gateway errors from the HTML endpoint.","triggerScenarios":"browserFetch returns a page with status outside 200–299 and classifyProviderHttpError(\"duckduckgo\", status, body) returns undefined — e.g. unusual statuses (408, 500-series) or bodies not matching known block patterns.","commonSituations":"DuckDuckGo temporary outages or 5xx errors; datacenter IP rate limiting with a status not matching the anomaly page; corporate proxies returning gateway errors; DDG changing response codes for automated traffic.","solutions":["Retry with backoff — many 5xx statuses are transient.","Check the numeric status in the message: 429-style throttling means switch egress IP or use a credentialed provider.","Fall back to another provider (Brave, Tavily, Exa, Kagi) via the provider chain.","Verify network/proxy is not intercepting html.duckduckgo.com with an error page."],"exampleFix":"// before\nconst res = await searchDuckDuckGo(params); // hard failure on 503\n// after\ntry {\n  return await searchDuckDuckGo(params);\n} catch (e) {\n  if (e instanceof SearchProviderError && (e.status ?? 0) >= 500) return retryWithBackoff(() => searchDuckDuckGo(params));\n  throw e;\n}","handlingStrategy":"retry","validationCode":null,"typeGuard":"function isDuckDuckGoHttpError(e: unknown): e is SearchProviderError {\n  return e instanceof SearchProviderError && e.provider === \"duckduckgo\" && /^DuckDuckGo HTML error \\(\\d+\\)$/.test(e.message);\n}","tryCatchPattern":"try {\n  return await searchDuckDuckGo(params);\n} catch (e) {\n  if (isDuckDuckGoHttpError(e)) {\n    const status = e.status ?? 0;\n    if (status >= 500 || status === 408) return retryWithBackoff(() => searchDuckDuckGo(params), 3);\n    if (status === 429) return fallbackProvider(params);\n  }\n  throw e;\n}","preventionTips":["Prefer API-backed providers for production; treat DDG HTML scraping as best-effort.","Add exponential backoff for 5xx statuses and immediate fallback for 429s.","Monitor provider status codes to detect DDG endpoint changes early."],"tags":["network","http-status","duckduckgo","rate-limit","search-provider"],"backgroundTag":"http-error-response","analyzedSha":"969062200754ea02cfac922e5ebb8c608c079e15","analyzedAt":"2026-08-31T10:29:35.737Z","schemaVersion":2},"datasetVersion":"2026-08-31T14:17:45.589Z"}