fish2018/pansou · error

[ ] 解析详情页失败

Error message

[%s] 解析详情页失败: %w

What it means

goquery.NewDocumentFromReader parses the detail response body as HTML; this error wraps any parser failure. goquery's underlying golang.org/x/net/html parser is lenient, so this rarely triggers on valid text — it fires when the body is not HTML at all (binary, gzip-encoded-but-undeclared, or a truncated stream) or the body was already consumed.

Solutions

1. Log the wrapped %w error and the first bytes of resp.Body to see what was actually returned
2. Verify no custom Transport is interfering with gzip handling (let Go handle Content-Encoding automatically)
3. Check Content-Type of the response; skip non-HTML responses before parsing
4. Increase retry robustness: a transient truncated body usually succeeds on a later retry

Example fix

// before
doc, err := goquery.NewDocumentFromReader(resp.Body)
if err != nil {
    return detailResult{}, fmt.Errorf("[%s] 解析详情页失败: %w", p.Name(), err)
}
// after
if ct := resp.Header.Get("Content-Type"); ct != "" && !strings.Contains(ct, "html") {
    return detailResult{}, fmt.Errorf("[%s] 非HTML响应: %s", p.Name(), ct)
}
doc, err := goquery.NewDocumentFromReader(resp.Body)
if err != nil {
    return detailResult{}, fmt.Errorf("[%s] 解析详情页失败: %w", p.Name(), err)
}

ct := resp.Header.Get("Content-Type")
if ct != "" && !strings.Contains(ct, "text/html") {
    return skip // not an HTML page, parsing will fail
}

if err != nil {
    log.Printf("html parse failed: %v", err)
    // fall back to returning an empty result for this detail page
    return detailResult{}, nil
}

When it happens

Trigger: Response body is not readable HTML: compressed content with wrong Content-Encoding, a binary error page, or an empty/corrupt stream that makes net/html parsing return an error.

Common situations: Server returns a challenge page or garbage bytes instead of the forum HTML; a proxy mangles content-encoding; double decompression when a custom Transport sets Accept-Encoding and also has DisableCompression=false.

Understand the failure class

Background: "Invalid JSON response" and "Failed to parse response" errors: when an API answers 200 but the body isn't the JSON your library expected — this error's family across 28 libraries.

Related errors

• [ ] 解析搜索页面失败
• [ ] HTML解析失败
• [ ] 解析搜索结果HTML失败
• [ ] 解析搜索页面失败
• [ ] 解析搜索结果失败

AI-assisted analysis of fish2018/pansou@beaa561337 (2026-09-07). Data as JSON: /api/errors/5309951a8fd7a1f1. Report an issue: GitHub.

Appendix: source

Thrown at plugin/lou1/lou1.go:279

	req, err := http.NewRequestWithContext(ctx, http.MethodGet, detailURL, nil)
	if err != nil {
		return detailResult{}, fmt.Errorf("[%s] 创建详情页请求失败: %w", p.Name(), err)
	}
	setHTMLHeaders(req, baseURL)

	resp, err := p.doRequestWithRetry(req, client, maxRequestRetries)
	if err != nil {
		return detailResult{}, err
	}
	defer resp.Body.Close()

	if resp.StatusCode != http.StatusOK {
		return detailResult{}, fmt.Errorf("[%s] 详情页返回状态码: %d", p.Name(), resp.StatusCode)
	}

	doc, err := goquery.NewDocumentFromReader(resp.Body)
	if err != nil {
		return detailResult{}, fmt.Errorf("[%s] 解析详情页失败: %w", p.Name(), err)
	}

	content := doc.Find("div.message[isfirst='1']")
	if content.Length() == 0 {
		content = doc.Find(".message")
	}
	if content.Length() == 0 {
		content = doc.Selection
	}

	content.Find("script, style").Remove()

	links := extractLinksFromSelection(content)
	links = filterQuarkLinks(links)
	description := strings.TrimSpace(doc.Find("meta[name='description']").AttrOr("content", ""))
	if description == "" {
		description = truncateString(strings.TrimSpace(content.Text()), 200)
	}

View on GitHub (pinned to beaa561337)