Tencent/WeKnora · error

duckduckgo HTML search failed: %w

Error message

duckduckgo HTML search failed: %w

What it means

DuckDuckGo's Search first tries the HTML endpoint; if that fails AND the Instant Answer API fallback also fails or returns no results, the original HTML error is wrapped as "duckduckgo HTML search failed". It indicates both retrieval paths failed, and the HTML error is reported as the root cause.

Source

Thrown at internal/infrastructure/web_search/duckduckgo.go:61

	query string,
	maxResults int,
	includeDate bool,
) ([]*types.WebSearchResult, error) {
	if maxResults <= 0 {
		maxResults = 5
	}
	// Try HTML scraping first (more reliable for general results)
	htmlResults, err := p.searchHTML(ctx, query, maxResults)
	if err == nil && len(htmlResults) > 0 {
		return htmlResults, nil
	}
	// Fallback to Instant Answer API
	apiResults, apiErr := p.searchAPI(ctx, query, maxResults)
	if apiErr == nil && len(apiResults) > 0 {
		return apiResults, nil
	}
	if err != nil {
		return nil, fmt.Errorf("duckduckgo HTML search failed: %w", err)
	}
	return nil, fmt.Errorf("duckduckgo API search failed: %w", apiErr)
}

// searchHTML performs a web search using DuckDuckGo HTML endpoint
func (p *DuckDuckGoProvider) searchHTML(
	ctx context.Context,
	query string,
	maxResults int,
) ([]*types.WebSearchResult, error) {
	baseURL := "https://html.duckduckgo.com/html/"
	params := url.Values{}
	params.Set("q", query)
	params.Set("kl", "cn-zh")

	reqURL := baseURL + "?" + params.Encode()
	req, err := http.NewRequestWithContext(ctx, "GET", reqURL, nil)
	if err != nil {

View on GitHub (pinned to 988cbb0330)

Solutions

1. Check the wrapped root cause (network error vs status code vs parse error) and address that first.
2. If blocked, add/refresh realistic headers or route through a proxy; DuckDuckGo aggressively blocks datacenter IPs.
3. Add delays/backoff between searches to avoid anomaly detection.
4. Consider a different search provider if DuckDuckGo scraping reliability is insufficient.

func isDDGHTMLFailure(err error) bool {
    return err != nil && strings.Contains(err.Error(), "duckduckgo HTML search failed")
}

results, err := ddg.Search(ctx, query, 10, false)
if isDDGHTMLFailure(err) {
    // both HTML and API paths failed; route to a different provider
    results, err = bing.Search(ctx, query, 10, false)
}

When it happens

Trigger: searchHTML failed (non-200 status, network error, parse error) and then searchAPI returned an error or an empty result list, so the code falls back to reporting the original HTML failure.

Common situations: DuckDuckGo rate-limiting or bot-blocking the client (anomaly/captcha pages); no network egress; queries with no API answers; DKLM region param mismatch.

Related errors

• duckduckgo API search failed: %w
• failed to perform request: %w
• duckduckgo HTML returned status %d
• failed to parse HTML: %w
• rerank call failed: %w

AI-assisted analysis of Tencent/WeKnora@988cbb0330 (2026-09-02). Data as JSON: /api/errors/4402b3a7d9138808. Report an issue: GitHub.