can1357/oh-my-pi · error · SearchProviderError

DuckDuckGo blocked the request with a bot-detection challeng

Error message

DuckDuckGo blocked the request with a bot-detection challenge. DuckDuckGo throttles automated HTML searches from datacenter/shared-egress IPs; configure a credentialed provider such as Brave, Tavily, Exa, or Kagi for reliable web search.

What it means

DuckDuckGo's HTML endpoint sometimes returns an HTTP 200 page containing a bot-detection anomaly page instead of results. isAnomalyResponse detects this body, and callDuckDuckGoHtml throws a SearchProviderError with status 429 explaining that DuckDuckGo throttles automated searches from datacenter/shared-egress IPs and suggesting credentialed providers.

Source

Thrown at packages/coding-agent/src/web/search/providers/duckduckgo.ts:325

		signal,
		timeoutMs: params.timeoutMs,
		referer: "https://html.duckduckgo.com/",
		init: {
			method: "POST",
			body: form.toString(),
		},
		headers: { "Content-Type": "application/x-www-form-urlencoded" },
	});

	const body = page.html;
	if (page.status < 200 || page.status >= 300) {
		const classified = classifyProviderHttpError("duckduckgo", page.status, body);
		if (classified) throw classified;
		throw new SearchProviderError("duckduckgo", `DuckDuckGo HTML error (${page.status})`, page.status);
	}

	if (isAnomalyResponse(body)) {
		throw new SearchProviderError(
			"duckduckgo",
			"DuckDuckGo blocked the request with a bot-detection challenge. DuckDuckGo throttles automated HTML searches from datacenter/shared-egress IPs; configure a credentialed provider such as Brave, Tavily, Exa, or Kagi for reliable web search.",
			429,
		);
	}

	return body;
}

/** Execute a DuckDuckGo web search via the no-JS HTML frontend. */
export async function searchDuckDuckGo(params: SearchParams): Promise<SearchResponse> {
	const numResults = clampNumResults(params.numSearchResults ?? params.limit, DEFAULT_NUM_RESULTS, MAX_NUM_RESULTS);
	const signal = withHardTimeout(params.signal, params.timeoutMs);
	const sources: SearchSource[] = [];
	const seen = new Set<string>();
	let form: URLSearchParams | undefined = createDuckDuckGoForm(params);

	while (form && sources.length < numResults) {

View on GitHub (pinned to 9690622007)

Solutions

1. Switch to a credentialed provider (Brave, Tavily, Exa, Kagi) — DuckDuckGo offers no official API escape from this throttle.
2. Change egress IP (residential proxy, different network) or wait for the throttle to expire.
3. Reduce request rate / add backoff between automated searches.
4. Catch this error and advance the provider fallback chain.

Example fix

// before
providers = ["duckduckgo"]; // only DDG, blocked in CI
// after
providers = ["brave", "tavily", "duckduckgo"]; // credentialed providers first

function isBotChallenge(e: unknown): e is SearchProviderError {
  return e instanceof SearchProviderError && (e.status === 429 || e.message.includes("bot-detection challenge"));
}

try {
  return await searchDuckDuckGo(params);
} catch (e) {
  if (isBotChallenge(e)) return searchTavily(params); // credentialed provider
  throw e;
}

When it happens

Trigger: The fetched HTML body passes isAnomalyResponse (challenge/anomaly markers present) even though page.status is 2xx — typical after repeated automated POSTs to html.duckduckgo.com from flagged IPs.

Common situations: Running web search from cloud VMs/CI runners with shared egress; high query volume in a short window; VPN/datacenter proxies; containers reusing a blocked IP.

Related errors

• DuckDuckGo HTML error (${page.status})
• Ecosia blocked the request with a Cloudflare bot challenge.
• Ecosia search timed out.
• Google blocked the browser search with an automated-traffic
• Mojeek blocked the request with its automated-queries wall.

AI-assisted analysis of can1357/oh-my-pi@9690622007 (2026-08-31). Data as JSON: /api/errors/ac5b60ed5434e1cf. Report an issue: GitHub.