fish2018/pansou · error

[ ] HTML解析失败

Error message

[%s] HTML解析失败: %w

What it means

ClmaoPlugin.searchPage fails while parsing the decoded HTML with goquery.NewDocumentFromReader. goquery only errors when the underlying reader fails (its HTML parser is error-tolerant), so this usually indicates the decoded payload reader failed or an empty stream was handed in. Note: most real parse problems (no results found) do NOT produce this error — they surface as empty results from extractSearchResults.

Solutions

1. Log the first N bytes of decodedHTML to confirm it is actually search-results HTML and not a CAPTCHA/error page.
2. Check decodeModernPayload for edge cases that could corrupt or empty the payload (e.g. wrong base64/encoding assumptions after a site update).
3. Goquery itself rarely errors — verify the error comes from the reader, and treat empty decodedHTML as an empty-result case instead.
4. If the site changed markup so modern/legacy parsing both fail, update the selectors in parseModernSearchResults/extractSearchResults.

Example fix

// before
doc, err := goquery.NewDocumentFromReader(strings.NewReader(decodedHTML))
if err != nil {
    return nil, fmt.Errorf("[%s] HTML解析失败: %w", p.Name(), err)
}
// after
if strings.TrimSpace(decodedHTML) == "" {
    return nil, fmt.Errorf("[%s] 空响应，可能被反爬拦截", p.Name())
}
doc, err := goquery.NewDocumentFromReader(strings.NewReader(decodedHTML))
if err != nil {
    return nil, fmt.Errorf("[%s] HTML解析失败: %w", p.Name(), err)
}

func looksLikeSearchResults(html string) bool {
    return strings.Contains(html, "result") || strings.Contains(html, "search")
}

results, err := p.searchPage(client, keyword, page)
if err != nil {
    if strings.Contains(err.Error(), "HTML解析失败") {
        // decoded payload was not parseable HTML — likely CAPTCHA or changed page
        log.Printf("clmao returned non-HTML payload; possible anti-bot page")
        return nil, err
    }
    return nil, err
}

When it happens

Trigger: After decodeModernPayload produced decodedHTML, strings.NewReader(decodedHTML) is passed to goquery.NewDocumentFromReader and returns a non-nil err (reader/IO failure), so searchPage returns "[clmao] HTML解析失败: %w".

Common situations: decodeModernPayload returned corrupted/empty output on an unexpected page (e.g. a CAPTCHA or error page fed through the decoder); a bug in the decode step produced an invalid reader state; extremely large decoded payloads hitting memory limits.

Related errors

• [ ] 解析HTML失败
• [ ] 解析搜索结果HTML失败
• 解析HTML失败
• [ ] 解析搜索结果失败
• [ ] 解析详情页失败

AI-assisted analysis of fish2018/pansou@beaa561337 (2026-09-07). Data as JSON: /api/errors/74664f790df90180. Report an issue: GitHub.

Appendix: source

Thrown at plugin/clmao/clmao.go:213

	}

	// 读取响应体内容
	body, err := io.ReadAll(resp.Body)
	if err != nil {
		return nil, fmt.Errorf("[%s] 读取响应失败: %w", p.Name(), err)
	}

	decodedHTML := decodeModernPayload(string(body))
	if decodedHTML != string(body) {
		if modernResults := p.parseModernSearchResults(client, decodedHTML); len(modernResults) > 0 {
			return modernResults, nil
		}
	}

	// 兼容旧模板
	doc, err := goquery.NewDocumentFromReader(strings.NewReader(decodedHTML))
	if err != nil {
		return nil, fmt.Errorf("[%s] HTML解析失败: %w", p.Name(), err)
	}

	// 提取搜索结果
	return p.extractSearchResults(doc), nil
}

func decodeModernPayload(raw string) string {
	match := modernPayloadRegex.FindStringSubmatch(raw)
	if len(match) < 2 {
		return raw
	}
	decoded, err := base64.StdEncoding.DecodeString(match[1])
	if err != nil {
		return raw
	}
	text, err := url.PathUnescape(string(decoded))
	if err != nil {
		return string(decoded)

View on GitHub (pinned to beaa561337)