fish2018/pansou · error

[ ] 解析搜索页面失败

Error message

[%s] 解析搜索页面失败: %w

What it means

goquery.NewDocumentFromReader failed to parse the fetched search page as HTML in the djgou plugin. This wraps the goquery/html parse error and is thrown only when the response body cannot be tokenized into a document (e.g. empty or severely malformed content).

Solutions

  1. Inspect the raw body bytes on failure (hexdump/log prefix) to see what was actually returned
  2. Ensure transparent decompression: check if you set Accept-Encoding manually without disabling Decompress in the Transport
  3. Retry — if the body is empty it is often a transient server issue
  4. If persistent, update parsing to handle the new content type

Example fix

// before
transport := &http.Transport{}
// after
transport := &http.Transport{ DisableCompression: false } // or don't set Accept-Encoding manually
Defensive patterns

Strategy: fallback

Validate before calling

body, _ := io.ReadAll(resp.Body)
if len(bytes.TrimSpace(body)) == 0 {
    return fmt.Errorf("empty body, skip parse")
}

Try / catch

doc, err := goquery.NewDocumentFromReader(strings.NewReader(string(body)))
if err != nil {
    log.Printf("unparseable body (%d bytes): %q", len(body), body[:min(200,len(body))])
    return fallbackResult()
}

Prevention

When it happens

Trigger: goquery.NewDocumentFromReader(strings.NewReader(string(body))) returns err in searchImpl — typically empty body or binary/garbage content (gzip not decompressed, wrong charset stream).

Common situations: Server returns a compressed body without Content-Encoding handling; empty response after a 200; CDN serving error pages in unexpected formats.

Understand the failure class

Background: "invalid response format", "malformed payload", "missing data field": when an API returns 200 but the response shape is wrong — this error's family across 23 libraries.

Related errors

AI-assisted analysis of fish2018/pansou@beaa561337 (2026-09-07). Data as JSON: /api/errors/e6a07c85a076eaa4. Report an issue: GitHub.

Appendix: source

Thrown at plugin/djgou/djgou.go:156

	resp, err := p.doRequestWithRetry(req, client)
	if err != nil {
		return nil, fmt.Errorf("[%s] 搜索请求失败: %w", p.Name(), err)
	}

	if resp.StatusCode != 200 {
		resp.Body.Close()
		return nil, fmt.Errorf("[%s] 搜索请求返回状态码: %d", p.Name(), resp.StatusCode)
	}

	// 6. 读取并解析搜索结果页面。部分节点先返回 BTWAF JS 跳转页。
	body, err := io.ReadAll(resp.Body)
	resp.Body.Close()
	if err != nil {
		return nil, fmt.Errorf("[%s] 读取搜索页面失败: %w", p.Name(), err)
	}
	doc, err := goquery.NewDocumentFromReader(strings.NewReader(string(body)))
	if err != nil {
		return nil, fmt.Errorf("[%s] 解析搜索页面失败: %w", p.Name(), err)
	}
	if doc.Find("article.post-item-row").Length() == 0 {
		if match := btwafURLRegex.FindStringSubmatch(string(body)); len(match) > 1 {
			challengeURL := match[1]
			if strings.HasPrefix(challengeURL, "/") {
				challengeURL = SiteURL + challengeURL
			}
			challengeReq, reqErr := http.NewRequestWithContext(ctx, http.MethodGet, challengeURL, nil)
			if reqErr == nil {
				challengeReq.Header = req.Header.Clone()
				challengeResp, doErr := p.doRequestWithRetry(challengeReq, client)
				if doErr == nil {
					challengeBody, readErr := io.ReadAll(challengeResp.Body)
					challengeResp.Body.Close()
					if readErr == nil {
						doc, _ = goquery.NewDocumentFromReader(strings.NewReader(string(challengeBody)))
					}
				}

View on GitHub (pinned to beaa561337)