fish2018/pansou · error

[ ] 解析搜索结果HTML失败

Error message

[%s] 解析搜索结果HTML失败: %w

What it means

executeSearch wraps the error from goquery.NewDocumentFromReader as 解析搜索结果HTML失败. NewDocumentFromReader only fails if the response body cannot be read from the stream (I/O error), since html.Parse is lenient with malformed markup.

Solutions

1. Check that resp.Body is not read or closed anywhere before goquery parses it
2. Retry the request if the body transfer was interrupted (connection reset)
3. Limit-Read the body into memory first (io.ReadAll) so read errors surface clearly
4. Capture the server's actual content; a challenge/compressed body may need special handling

Example fix

// before
doc, err := goquery.NewDocumentFromReader(resp.Body)
if err != nil {
    return nil, fmt.Errorf("[%s] 解析搜索结果HTML失败: %w", p.Name(), err)
}
// after
bodyBytes, err := io.ReadAll(resp.Body)
if err != nil {
    return nil, fmt.Errorf("[%s] 读取响应体失败: %w", p.Name(), err)
}
doc, err := goquery.NewDocumentFromReader(bytes.NewReader(bodyBytes))
if err != nil {
    return nil, fmt.Errorf("[%s] 解析搜索结果HTML失败: %w", p.Name(), err)
}

// Go: read the body into memory first so I/O errors surface clearly
bodyBytes, err := io.ReadAll(io.LimitReader(resp.Body, maxBodySize))
if err != nil {
    return fmt.Errorf("failed to read response body: %w", err)
}

bodyBytes, rerr := io.ReadAll(resp.Body)
if rerr != nil {
    return nil, fmt.Errorf("body read failed (connection reset?): %w", rerr)
}
doc, err := goquery.NewDocumentFromReader(bytes.NewReader(bodyBytes))

When it happens

Trigger: The response body stream errors while being read: connection reset mid-response, server closed the connection early, or the body was already consumed/closed before parsing.

Common situations: Server (or a middlebox) truncating the response; timeouts during body transfer; accidentally closing resp.Body earlier or reading it in another step (e.g. for logging) before goquery.

Understand the failure class

Background: "Invalid JSON response" and "Failed to parse response" errors: when an API answers 200 but the body isn't the JSON your library expected — this error's family across 28 libraries.

Related errors

• [ ] 解析搜索页面失败
• [ ] 解析搜索结果HTML失败
• [ ] 第 页HTML解析失败
• [ ] 网页搜索 HTML 解析失败
• [ ] HTML解析失败

AI-assisted analysis of fish2018/pansou@beaa561337 (2026-09-07). Data as JSON: /api/errors/0630f32d1918315d. Report an issue: GitHub.

Appendix: source

Thrown at plugin/hdmoli/hdmoli.go:146

	req.Header.Set("Connection", "keep-alive")
	req.Header.Set("Upgrade-Insecure-Requests", "1")
	req.Header.Set("Cache-Control", "max-age=0")
	req.Header.Set("Referer", BaseURL+"/") // HDmoli需要设置referer

	resp, err := p.doRequestWithRetry(req, client)
	if err != nil {
		return nil, fmt.Errorf("[%s] 搜索请求失败: %w", p.Name(), err)
	}
	defer resp.Body.Close()

	if resp.StatusCode != 200 {
		return nil, fmt.Errorf("[%s] 搜索请求HTTP状态错误: %d", p.Name(), resp.StatusCode)
	}

	// 解析HTML提取搜索结果
	doc, err := goquery.NewDocumentFromReader(resp.Body)
	if err != nil {
		return nil, fmt.Errorf("[%s] 解析搜索结果HTML失败: %w", p.Name(), err)
	}

	return p.parseSearchResults(doc)
}

// doRequestWithRetry 带重试机制的HTTP请求
func (p *HdmoliPlugin) doRequestWithRetry(req *http.Request, client *http.Client) (*http.Response, error) {
	maxRetries := 3
	var lastErr error

	for i := 0; i < maxRetries; i++ {
		if i > 0 {
			// 指数退避重试
			backoff := time.Duration(1<<uint(i-1)) * 200 * time.Millisecond
			time.Sleep(backoff)
		}

		// 克隆请求避免并发问题

View on GitHub (pinned to beaa561337)