fish2018/pansou · error

[ ] 解析搜索页面失败

Error message

[%s] 解析搜索页面失败: %w

What it means

searchImpl wraps a goquery.NewDocumentFromReader failure as "[%s] 解析搜索页面失败". goquery parsing only fails when reading the underlying response body fails, so this means the HTML document could not be fully read/decoded — typically a truncated or reset connection during body transfer.

Solutions

1. Retry the search; mid-body resets are usually transient.
2. Read the body manually with io.ReadAll first, then parse with goquery.NewDocumentFromReader to separate I/O errors from parse issues and enable logging of partial content.
3. Check proxy/VPN interference and ensure the client handles Content-Encoding (gzip) consistently with the Accept-Encoding header sent in setCommonHeaders.
4. Increase retry/backoff so doRequestWithRetry absorbs these resets.

Example fix

// before
doc, err := goquery.NewDocumentFromReader(resp.Body)
if err != nil {
    return nil, fmt.Errorf("[%s] 解析搜索页面失败: %w", p.Name(), err)
}
// after
bodyBytes, err := io.ReadAll(resp.Body)
if err != nil {
    return nil, fmt.Errorf("[%s] 读取搜索页面失败: %w", p.Name(), err)
}
doc, err := goquery.NewDocumentFromReader(bytes.NewReader(bodyBytes))
if err != nil {
    return nil, fmt.Errorf("[%s] 解析搜索页面失败: %w", p.Name(), err)
}

results, err := plugin.Search(ctx, keyword)
if err != nil && strings.Contains(err.Error(), "解析搜索页面失败") {
    // body transfer failed; retry once after a short delay
    time.Sleep(time.Second)
    results, err = plugin.Search(ctx, keyword)
}

When it happens

Trigger: goquery.NewDocumentFromReader(resp.Body) returns an error: the connection was closed mid-body (unexpected EOF, connection reset) while streaming the search results HTML.

Common situations: Server/LB cutting large search-result pages, flaky proxy or VPN connections, gzip/deflate body mismatch when the client does not handle the advertised encoding, or the site throttling by dropping connections.

Understand the failure class

Background: "Invalid JSON response" and "Failed to parse response" errors: when an API answers 200 but the body isn't the JSON your library expected — this error's family across 28 libraries.

Related errors

• 解析HTML失败
• [ ] HTML解析失败
• [ ] 解析搜索结果失败
• [ ] 搜索请求返回 HTTP
• [ ] 读取搜索结果失败

AI-assisted analysis of fish2018/pansou@beaa561337 (2026-09-07). Data as JSON: /api/errors/188a5150e577cfb6. Report an issue: GitHub.

Appendix: source

Thrown at plugin/daishudj/daishudj.go:147

	req, err := http.NewRequestWithContext(ctx, http.MethodGet, searchURL, nil)
	if err != nil {
		return nil, fmt.Errorf("[%s] 创建搜索请求失败: %w", p.Name(), err)
	}
	setCommonHeaders(req, "https://www.daishuduanju.com/")

	resp, err := p.doRequestWithRetry(req, client)
	if err != nil {
		return nil, fmt.Errorf("[%s] 搜索请求失败: %w", p.Name(), err)
	}
	defer resp.Body.Close()

	if resp.StatusCode != http.StatusOK {
		return nil, fmt.Errorf("[%s] 搜索返回状态码: %d", p.Name(), resp.StatusCode)
	}

	doc, err := goquery.NewDocumentFromReader(resp.Body)
	if err != nil {
		return nil, fmt.Errorf("[%s] 解析搜索页面失败: %w", p.Name(), err)
	}

	var (
		results []model.SearchResult
		wg      sync.WaitGroup
		mu      sync.Mutex
		sem     = make(chan struct{}, maxConcurrency)
	)

	doc.Find(".item-jx.item-blog").Each(func(_ int, item *goquery.Selection) {
		titleSel := item.Find(".subtitle h5 a")
		title := strings.TrimSpace(titleSel.Text())
		detailURL, ok := titleSel.Attr("href")
		if !ok || title == "" || detailURL == "" {
			return
		}

		postID := extractPostID(detailURL)

View on GitHub (pinned to beaa561337)