fish2018/pansou · error

[ ] 解析详情页失败

Error message

[%s] 解析详情页失败: %w

What it means

Raised in fetchDetailData when goquery.NewDocumentFromReader fails to parse the HTTP response body as an HTML document. This means the body was malformed, truncated, or not HTML at all (e.g. a JSON error payload, gzip issue, or anti-bot interstitial page).

Solutions

1. Check that resp.Body is not nil and has not been read before passing to NewDocumentFromReader
2. Verify the request disables or correctly handles compression (set Accept-Encoding explicitly or let http.Client auto-decompress)
3. Log a snippet of the raw body on failure to see what the server actually returned
4. Ensure the response is fully downloaded (no early ctx cancellation) before parsing
5. Fall back to a plain io.ReadAll check to confirm the content is HTML before goquery parsing

Example fix

// before
doc, err := goquery.NewDocumentFromReader(resp.Body)
if err != nil {
    return detailData{}, fmt.Errorf("[%s] 解析详情页失败: %w", p.Name(), err)
}
// after
bodyBytes, rerr := io.ReadAll(resp.Body)
if rerr != nil {
    return detailData{}, fmt.Errorf("[%s] 读取详情页失败: %w", p.Name(), rerr)
}
doc, err := goquery.NewDocumentFromReader(bytes.NewReader(bodyBytes))
if err != nil {
    return detailData{}, fmt.Errorf("[%s] 解析详情页失败: %w (body prefix: %.200s)", p.Name(), err, string(bodyBytes))
}

body, err := io.ReadAll(resp.Body)
if err != nil { return err }
ct := resp.Header.Get("Content-Type")
if !strings.Contains(ct, "text/html") || len(bytes.TrimSpace(body)) == 0 {
    return fmt.Errorf("unexpected response: content-type=%s len=%d", ct, len(body))
}
doc, err := goquery.NewDocumentFromReader(bytes.NewReader(body))

doc, err := goquery.NewDocumentFromReader(resp.Body)
if err != nil {
    log.Printf("html parse failed: %v", err)
    return detailData{}, errParse
}

When it happens

Trigger: Triggered when the detail page response body cannot be tokenized into an HTML document: empty body, invalid encoding, corrupted/compressed content read incorrectly, or the server returned an error page goquery cannot parse.

Common situations: Server returns a compressed body without the client handling Content-Encoding; response body already consumed before parsing; CDN serves a challenge/JS page; network interruption truncated the body mid-transfer.

Understand the failure class

Background: "Invalid JSON response" and "Failed to parse response" errors: when an API answers 200 but the body isn't the JSON your library expected — this error's family across 28 libraries.

Related errors

• [ ] 解析HTML失败
• [ ] 解析搜索结果HTML失败
• 解析HTML失败
• [ ] 解析搜索结果失败
• [ ] 解析搜索页面失败

AI-assisted analysis of fish2018/pansou@beaa561337 (2026-09-07). Data as JSON: /api/errors/ee23bc2e09c3bace. Report an issue: GitHub.

Appendix: source

Thrown at plugin/mizixing/mizixing.go:259

	req, err := http.NewRequestWithContext(ctx, http.MethodGet, detailURL, nil)
	if err != nil {
		return detailData{}, fmt.Errorf("[%s] 创建详情页请求失败: %w", p.Name(), err)
	}
	setHTMLHeaders(req, detailURL)

	resp, err := p.doRequestWithRetry(req, client, maxRequestRetries)
	if err != nil {
		return detailData{}, err
	}
	defer resp.Body.Close()

	if resp.StatusCode != http.StatusOK {
		return detailData{}, fmt.Errorf("[%s] 详情页返回状态码: %d", p.Name(), resp.StatusCode)
	}

	doc, err := goquery.NewDocumentFromReader(resp.Body)
	if err != nil {
		return detailData{}, fmt.Errorf("[%s] 解析详情页失败: %w", p.Name(), err)
	}

	content := doc.Find("article.article-content")
	if content.Length() == 0 {
		content = doc.Find(".article-content")
	}
	if content.Length() == 0 {
		content = doc.Find(".entry-content")
	}
	if content.Length() == 0 {
		content = doc.Selection
	}

	content.Find("script, style, .bdsharebuttonbox, #respond, .post-views, .share, .relates").Remove()

	links := extractLinksFromSelection(content)

	description := strings.TrimSpace(doc.Find("meta[name='description']").AttrOr("content", ""))

View on GitHub (pinned to beaa561337)