What causes "[%s] 解析搜索结果HTML失败: %w" in fish2018/pansou?

Unwrap the error to see the underlying body-read failure.

fish2018/pansou · error

[ ] 解析搜索结果HTML失败

Error message

[%s] 解析搜索结果HTML失败: %w

What it means

After a successful page fetch, executeSearch parses the response body with goquery.NewDocumentFromReader. If constructing the document fails (I/O error while reading the body, malformed stream), the error is wrapped with this message and the plugin name. This indicates the search response could not be turned into a parseable HTML document.

Solutions

Unwrap the error to see the underlying body-read failure.
Check Content-Encoding handling — ensure the scraper/client negotiates and decompresses encodings correctly.
Retry the request; truncated bodies are usually transient.
Capture the raw response (status, length) when it happens to diagnose whether the server sent a valid body.

Example fix

// before
doc, err := goquery.NewDocumentFromReader(resp.Body)
if err != nil { return err }
// after
doc, err := goquery.NewDocumentFromReader(resp.Body)
if err != nil {
    return fmt.Errorf("bad search response (status=%d): %w", resp.StatusCode, err)
}

Defensive patterns

Strategy: retry

Validate before calling

if resp.ContentLength == 0 {
    return errors.New("empty search response body, skipping parse")
}

Try / catch

doc, err := goquery.NewDocumentFromReader(resp.Body)
if err != nil {
    return fmt.Errorf("unreadable response (status=%d): %w", resp.StatusCode, err) // caller retries
}

Prevention

Retry truncated-body failures; they are usually transient.
Verify Content-Encoding negotiation matches the client's decompressor.
Log status and content length whenever parsing fails.

When it happens

Trigger: goquery.NewDocumentFromReader(resp.Body) returns an error — typically a read failure on the response body stream (connection closed mid-body, decompression error) rather than invalid HTML, since goquery tolerates most markup.

Common situations: Server closes connection while streaming the body; gzip/brotli decompression mismatch because Accept-Encoding and actual encoding diverge; an interrupted proxy connection delivering a truncated response.

Understand the failure class

Background: "Invalid JSON response" and "Failed to parse response" errors: when an API answers 200 but the body isn't the JSON your library expected — this error's family across 28 libraries.

Related errors

AI-assisted analysis of fish2018/pansou@beaa561337 (2026-09-07). Data as JSON: /api/errors/f4648ac0a18cb34e. Report an issue: GitHub.

Appendix: source

Thrown at plugin/diduan/diduan.go:181

// executeSearch 执行搜索请求
func (p *DiduanPlugin) executeSearch(keyword string) ([]model.SearchResult, error) {
	// 构建搜索URL
	searchURL := fmt.Sprintf("%s%s", BaseURL, fmt.Sprintf(SearchPath, url.QueryEscape(keyword)))

	resp, err := p.getPage(searchURL)
	if err != nil {
		return nil, fmt.Errorf("[%s] 搜索请求失败: %w", p.Name(), err)
	}
	defer resp.Body.Close()

	if resp.StatusCode != 200 {
		return nil, p.httpStatusError("搜索", resp)
	}

	// 解析HTML提取搜索结果
	doc, err := goquery.NewDocumentFromReader(resp.Body)
	if err != nil {
		return nil, fmt.Errorf("[%s] 解析搜索结果HTML失败: %w", p.Name(), err)
	}

	return p.parseSearchResults(doc)
}

// getPage 串行化 cloudscraper 调用，避免其 stealth 计数器并发竞争。
func (p *DiduanPlugin) getPage(rawURL string) (*http.Response, error) {
	p.scraperMu.Lock()
	defer p.scraperMu.Unlock()
	return p.scraper.Get(rawURL)
}

func (p *DiduanPlugin) httpStatusError(action string, resp *http.Response) error {
	if strings.EqualFold(resp.Header.Get("cf-mitigated"), "challenge") {
		return fmt.Errorf("[%s] %s触发 Cloudflare Managed Challenge (HTTP %d)", p.Name(), action, resp.StatusCode)
	}
	return fmt.Errorf("[%s] %sHTTP状态错误: %d", p.Name(), action, resp.StatusCode)
}

View on GitHub (pinned to beaa561337)