fish2018/pansou · error

[ ] 解析搜索结果HTML失败

Error message

[%s] 解析搜索结果HTML失败: %w

What it means

executeSearchWithRateLimit returns '[javdb] 解析搜索结果HTML失败: %w' (failed to parse search results HTML) when goquery.NewDocumentFromReader fails to parse the fetched body as HTML. With goquery this is rare and almost always means the reader itself errored, but it signals the response could not be turned into a parseable document.

Solutions

1. Log the first bytes/Content-Type of bodyBytes when this fires to see what was actually returned
2. Verify automatic gzip/deflate decompression (check Content-Encoding handling in the transport)
3. Check for soft-block pages (challenge/JS) that are not real search HTML
4. Guard against empty bodies before parsing and return a clearer 'empty response' error
5. Update selectors/endpoint if the site changed its HTML contract

Example fix

// before
doc, err := goquery.NewDocumentFromReader(strings.NewReader(string(bodyBytes)))
// after
if len(bodyBytes) == 0 {
    return nil, fmt.Errorf("[%s] empty response body", p.Name()), false
}
doc, err := goquery.NewDocumentFromReader(strings.NewReader(string(bodyBytes)))

// Go: sanity-check the body before HTML parsing
ct := resp.Header.Get("Content-Type")
if !strings.Contains(ct, "text/html") {
    return fmt.Errorf("unexpected content-type %q", ct)
}
if len(bodyBytes) == 0 {
    return errors.New("empty response body")
}

func looksLikeHTML(b []byte) bool {
    s := strings.TrimSpace(string(b))
    return len(s) > 0 && (strings.HasPrefix(s, "<") || strings.Contains(s[:min(200, len(s))], "<html"))
}

doc, err := goquery.NewDocumentFromReader(bytes.NewReader(bodyBytes))
if err != nil {
    log.Printf("javdb html parse failed, body head=%q", bodyBytes[:min(200, len(bodyBytes))])
    return nil, err, false
}

When it happens

Trigger: goquery.NewDocumentFromReader returns an error on the bodyBytes reader — typically because the body is empty/corrupt or the underlying reader fails, after a successful (but wrong) response was fetched.

Common situations: Site returns a non-HTML body (JSON error, compressed/garbled bytes when Content-Encoding handling is broken, empty body from a soft block) that the HTML parser chokes on.

Related errors

• [ ] 解析HTML失败
• 解析HTML失败
• [ ] 解析搜索结果失败
• [ ] 解析详情页失败
• [ ] 解析搜索页面失败

AI-assisted analysis of fish2018/pansou@beaa561337 (2026-09-07). Data as JSON: /api/errors/9332d0c2610e59b4. Report an issue: GitHub.

Appendix: source

Thrown at plugin/javdb/javdb.go:237

	if err != nil {
		return nil, fmt.Errorf("[%s] 读取搜索结果失败: %w", p.Name(), err), false
	}

	if p.debugMode {
		bodyStr := string(bodyBytes)
		log.Printf("[JAVDB] 响应体长度: %d", len(bodyStr))
		// 输出前500个字符用于调试
		if len(bodyStr) > 500 {
			log.Printf("[JAVDB] 响应体前500字符: %s", bodyStr[:500])
		} else {
			log.Printf("[JAVDB] 完整响应体: %s", bodyStr)
		}
	}

	// 解析HTML提取搜索结果
	doc, err := goquery.NewDocumentFromReader(strings.NewReader(string(bodyBytes)))
	if err != nil {
		return nil, fmt.Errorf("[%s] 解析搜索结果HTML失败: %w", p.Name(), err), false
	}

	results, err := p.parseSearchResults(doc)
	return results, err, false
}


// doRequestWithRetry 带重试机制的HTTP请求
func (p *JavdbPlugin) doRequestWithRetry(req *http.Request, client *http.Client) (*http.Response, error) {
	maxRetries := 3
	var lastErr error
	
	for i := 0; i < maxRetries; i++ {
		if i > 0 {
			// 指数退避重试
			backoff := time.Duration(1<<uint(i-1)) * 200 * time.Millisecond
			time.Sleep(backoff)
		}

View on GitHub (pinned to beaa561337)