fish2018/pansou · error
[ ] 解析搜索结果HTML失败
Error message
[%s] 解析搜索结果HTML失败: %w
What it means
dyyj.executeSearchHTML could not parse the fetched search page HTML into a goquery document (goquery.NewDocumentFromReader failed). Indicates the body was not valid parseable HTML — empty, binary, or severely malformed content.
Solutions
- Log bodyString length and first bytes to see what was actually received
- Ensure the transport handles Content-Encoding (EnableFullDuplex/default Go transport handles gzip; brotli needs manual handling)
- Guard against empty bodyString before parsing
- Verify the site still returns HTML and not a JSON API or challenge page
Example fix
// before
doc, err := goquery.NewDocumentFromReader(strings.NewReader(bodyString))
if err != nil {
return nil, fmt.Errorf("[%s] 解析搜索结果HTML失败: %w", p.Name(), err)
}
// after
if len(bodyString) == 0 {
return nil, fmt.Errorf("[%s] 响应体为空", p.Name())
}
doc, err := goquery.NewDocumentFromReader(strings.NewReader(bodyString))
if err != nil {
return nil, fmt.Errorf("[%s] 解析搜索结果HTML失败: %w (head=%q)", p.Name(), err, bodyString[:min(64, len(bodyString))])
} Defensive patterns
Strategy: validation
Validate before calling
if len(bodyString) == 0 { return errors.New("empty search response") }
if !strings.Contains(strings.ToLower(resp.Header.Get("Content-Type")), "html") { return fmt.Errorf("unexpected content-type: %s", resp.Header.Get("Content-Type")) } Type guard
func looksLikeHTML(s string) bool {
s = strings.TrimSpace(s)
return strings.HasPrefix(s, "<") || strings.HasPrefix(s, "\xef\xbb\xbf<") || strings.Contains(s[:min(512, len(s))], "<html")
} Try / catch
doc, err := goquery.NewDocumentFromReader(strings.NewReader(bodyString))
if err != nil {
log.Printf("html parse failed: %v, head=%q", err, bodyString[:min(64, len(bodyString))])
return nil, err
} Prevention
- Verify Content-Encoding handling (gzip/brotli) in the transport
- Reject empty bodies before parsing
- Check for WAF challenge pages before parsing
- Snapshot a body prefix on every parse failure
When it happens
Trigger: goquery.NewDocumentFromReader(strings.NewReader(bodyString)) returned err — typically an empty body, gzip/brotli bytes not decompressed (Content-Encoding mishandled), or binary garbage from a WAF.
Common situations: Response was compressed but the client lacks automatic decompression; site returned an empty 200; anti-bot served raw challenge bytes the parser chokes on.
Related errors
AI-assisted analysis of fish2018/pansou@beaa561337 (2026-09-07).
Data as JSON: /api/errors/9d1144de8887db9c.
Report an issue: GitHub.
Appendix: source
Thrown at plugin/dyyj/dyyj.go:364
}
// 查找所有包含/d/的链接（使用预编译的正则）
matches := linkHrefRegex.FindAllStringSubmatch(bodyString, -1)
log.Printf("[DYYJ] 使用正则表达式找到 %d 个包含'/d/'的链接", len(matches))
for i, match := range matches {
if i < 10 {
log.Printf("[DYYJ] 链接 %d: %s", i+1, match[1])
}
}
}
// 解析HTML提取搜索结果
doc, err := goquery.NewDocumentFromReader(strings.NewReader(bodyString))
if err != nil {
if p.debugMode {
log.Printf("[DYYJ] 解析搜索结果HTML失败: %v", err)
}
return nil, fmt.Errorf("[%s] 解析搜索结果HTML失败: %w", p.Name(), err)
}
results, err := p.parseSearchResults(doc, bodyString)
if p.debugMode {
log.Printf("[DYYJ] 解析搜索结果完成，获取到 %d 个结果", len(results))
}
return results, err
}
type dyyjAPIResponse struct {
Data []dyyjDiscussion `json:"data"`
Included []dyyjIncluded `json:"included"`
}
type dyyjDiscussion struct {
ID string `json:"id"`
Attributes struct {View on GitHub (pinned to beaa561337)