What causes "[%s] 第%d页HTML解析失败: %w" in fish2018/pansou?

Log/save htmlContent (debug mode already can dump it) and inspect what the server actually returned

fish2018/pansou · error

[ ] 第页HTML解析失败

Error message

[%s] 第%d页HTML解析失败: %w

What it means

This error is thrown by Dy4kPlugin.searchPage when goquery.NewDocumentFromReader cannot parse the fetched HTML. It means the response body, although readable, is not valid/parseable HTML (e.g., a truncated document, an anti-bot challenge page, or binary/encoded content instead of HTML). The underlying goquery/xml parse error is wrapped with the plugin name and page number.

Solutions

Log/save htmlContent (debug mode already can dump it) and inspect what the server actually returned
Check for anti-bot interstitials and rotate User-Agent / add cookies
Retry the request — truncation is often transient
Verify the response Content-Encoding is being handled (disable manual gzip handling conflicts in the transport)

Example fix

// before
doc, err := goquery.NewDocumentFromReader(strings.NewReader(htmlContent))
if err != nil {
    return nil, 0, fmt.Errorf("[%s] 第%d页HTML解析失败: %w", p.Name(), page, err)
}
// after
doc, err := goquery.NewDocumentFromReader(strings.NewReader(htmlContent))
if err != nil {
    return nil, 0, fmt.Errorf("[%s] 第%d页HTML解析失败 (长度=%d, 前100字节=%.100q): %w", p.Name(), page, len(htmlContent), htmlContent, err)
}

Defensive patterns

Strategy: validation

Validate before calling

if len(htmlContent) == 0 { return errors.New("empty HTML response") }
if !strings.Contains(htmlContent, "<html") && !strings.Contains(htmlContent, "<!DOCTYPE") {
    return fmt.Errorf("non-HTML response: %.200q", htmlContent)
}

Try / catch

doc, err := goquery.NewDocumentFromReader(strings.NewReader(htmlContent))
if err != nil {
    return fmt.Errorf("parse failed: %w (head=%.200q)", err, htmlContent) // inspect before retrying
}

Prevention

Verify response Content-Type is text/html before parsing
Detect anti-bot/captcha interstitials by signature strings and rotate UA/cookies
Log a body preview on parse failure to diagnose what the server returned
Handle Content-Encoding correctly so decompressed text is parsed

When it happens

Trigger: Calling Search when the HTML fetched for a search page fails goquery parsing — typically a truncated or malformed response, a WAF/captcha interstitial, or non-HTML content returned with 200.

Common situations: Site serves a Cloudflare/security challenge page; response truncated by an unstable connection; site changed markup or returned compressed content without correct Content-Encoding handling.

Related errors

AI-assisted analysis of fish2018/pansou@beaa561337 (2026-09-07). Data as JSON: /api/errors/f8de89eea779e818. Report an issue: GitHub.

Appendix: source

Thrown at plugin/dy4k/dy4k.go:446

	if DebugMode {
		htmlDir := "./html"
		os.MkdirAll(htmlDir, 0755)

		filename := fmt.Sprintf("dy4k_page_%d_%s.html", page, strings.ReplaceAll(encodedKeyword, "%", "_"))
		filepath := filepath.Join(htmlDir, filename)

		err = os.WriteFile(filepath, htmlBytes, 0644)
		if err != nil {
			debugPrintf("❌ [Dy4k DEBUG] 保存HTML文件失败: %v\n", err)
		} else {
			debugPrintf("✅ [Dy4k DEBUG] HTML已保存到: %s\n", filepath)
		}
	}

	// 解析HTML响应
	doc, err := goquery.NewDocumentFromReader(strings.NewReader(htmlContent))
	if err != nil {
		return nil, 0, fmt.Errorf("[%s] 第%d页HTML解析失败: %w", p.Name(), page, err)
	}

	// 8. 解析分页信息
	totalPages := p.parseTotalPages(doc)

	// 9. 提取搜索结果。旧版使用 .hl-list-item，新版 4KDY 使用 /4K-detail/ 链接。
	results := make([]model.SearchResult, 0)
	if legacyItems := doc.Find(".hl-list-item"); legacyItems.Length() > 0 {
		legacyItems.Each(func(_ int, s *goquery.Selection) {
			if result := p.parseSearchResultItem(s); result != nil {
				results = append(results, *result)
			}
		})
	} else {
		p.extractNewSearchResults(doc, &results)
	}

	return results, totalPages, nil

View on GitHub (pinned to beaa561337)

[ ] 第 页HTML解析失败