fish2018/pansou · error

[ ] 解析搜索页面失败

Error message

[%s] 解析搜索页面失败: %w

What it means

Wrapped parse error in KkMaoPlugin.searchImpl (plugin/kkmao/kkmao.go:134): the 200 response body could not be turned into a goquery document — usually an HTML challenge or non-HTML error page instead of search results.

Solutions

1. Read and size-check the body before parsing; log the first bytes on failure
2. Ensure the transport handles gzip/deflate (http.DefaultTransport does; custom ones may not)
3. Confirm no earlier code consumed resp.Body
4. Handle encoding indicated by the Content-Type charset

Example fix

// before
doc, err := goquery.NewDocumentFromReader(resp.Body)
if err != nil { return nil, fmt.Errorf("[%s] 解析搜索页面失败: %w", p.Name(), err) }
// after
body, rerr := io.ReadAll(resp.Body)
if rerr != nil || len(bytes.TrimSpace(body)) == 0 {
    return nil, fmt.Errorf("[%s] empty response body (status %d)", p.Name(), resp.StatusCode)
}
doc, err := goquery.NewDocumentFromReader(bytes.NewReader(body))

body, _ := io.ReadAll(io.LimitReader(resp.Body, 256))
if len(bytes.TrimSpace(body)) == 0 || !utf8.Valid(body) {
    return errors.New("response body empty or invalid; skip parsing")
}

results, err := plugin.Search(keyword)
if err != nil {
    if strings.Contains(err.Error(), "解析搜索页面失败") {
        return useFallbackSource(keyword)
    }
    return err
}

When it happens

Trigger: goquery.NewDocumentFromReader(resp.Body) errors: empty body from a blocked request, truncated transfer, charset/compression mismatch, or a JSON/challenge page instead of HTML.

Common situations: 站点改版返回 JSON 或错误页。

Understand the failure class

Background: "Invalid JSON response" and "Failed to parse response" errors: when an API answers 200 but the body isn't the JSON your library expected — this error's family across 28 libraries.

Related errors

• [ ] 解析搜索页面失败
• [ ] 解析搜索结果HTML失败
• [ ] 第 页HTML解析失败
• [ ] HTML解析失败
• [ ] HTML解析失败

AI-assisted analysis of fish2018/pansou@beaa561337 (2026-09-07). Data as JSON: /api/errors/7714d748e6e74ea4. Report an issue: GitHub.

Appendix: source

Thrown at plugin/kkmao/kkmao.go:134

	if err != nil {
		return nil, fmt.Errorf("[%s] 创建请求失败: %w", p.Name(), err)
	}

	setCommonHeaders(req, "https://www.kuakemao.com/")

	resp, err := p.doRequestWithRetry(req, client, searchMaxRetries, retryBaseDelay)
	if err != nil {
		return nil, fmt.Errorf("[%s] 搜索请求失败: %w", p.Name(), err)
	}
	defer resp.Body.Close()

	if resp.StatusCode != http.StatusOK {
		return nil, fmt.Errorf("[%s] 搜索返回状态码: %d", p.Name(), resp.StatusCode)
	}

	doc, err := goquery.NewDocumentFromReader(resp.Body)
	if err != nil {
		return nil, fmt.Errorf("[%s] 解析搜索页面失败: %w", p.Name(), err)
	}

	var (
		results []model.SearchResult
		wg      sync.WaitGroup
		mu      sync.Mutex
		sem     = make(chan struct{}, maxConcurrency)
	)

	doc.Find("article.excerpt").Each(func(_ int, item *goquery.Selection) {
		titleSel := item.Find("header h2 a")
		title := strings.TrimSpace(titleSel.Text())
		detailURL, ok := titleSel.Attr("href")
		if !ok || title == "" || detailURL == "" {
			return
		}

		articleID := extractArticleID(detailURL)

View on GitHub (pinned to beaa561337)