fish2018/pansou · error

[ ] 解析搜索页面失败

Error message

[%s] 解析搜索页面失败: %w

What it means

nyaa's searchImpl fails while goquery parses the (supposedly HTML) response body into a document. This indicates the response body is not the expected search-results HTML — parsing itself rarely fails on well-formed responses, so the usual cause is an empty, truncated, or compressed body (e.g. gzip/brotli not decompressed because Accept-Encoding was set manually without handling it).

Solutions

1. Don't set Accept-Encoding manually — let net/http's transparent gzip handling work, or decompress explicitly with gzip.NewReader.
2. Dump the first bytes of resp.Body (via io.LimitReader + TeeReader) to see what was actually returned.
3. Check the Content-Type/Content-Encoding response headers before parsing.
4. Handle charset explicitly with golang.org/x/net/html/charset if the page is non-UTF-8.

Example fix

// before
req.Header.Set("Accept-Encoding", "gzip, deflate")
doc, err := goquery.NewDocumentFromReader(resp.Body)
// after
// omit manual Accept-Encoding; net/http decompresses gzip transparently
doc, err := goquery.NewDocumentFromReader(resp.Body)

ct := resp.Header.Get("Content-Type")
if !strings.Contains(ct, "text/html") {
    return fmt.Errorf("expected HTML, got %s", ct)
}

null

doc, err := goquery.NewDocumentFromReader(resp.Body)
if err != nil {
    // log Content-Type/Content-Encoding and a body prefix, then fail fast
    return nil, fmt.Errorf("parse failed (ct=%s): %w", resp.Header.Get("Content-Type"), err)
}

When it happens

Trigger: goquery.NewDocumentFromReader(resp.Body) returns an error — the reader yields malformed/undecodable content: the site returned a charset goquery can't detect, the body is compressed but the client didn't send Accept-Encoding so no auto-decompression happened, or the body was already consumed/truncated.

Common situations: Manually set headers (User-Agent etc.) on a default client plus explicit Accept-Encoding: gzip without decompression; site returns a binary challenge page; TLS interception corrupts the stream; empty body from an interrupted connection.

Understand the failure class

Background: "Invalid JSON response" and "Failed to parse response" errors: when an API answers 200 but the body isn't the JSON your library expected — this error's family across 28 libraries.

Related errors

• [ ] 解析HTML失败
• [ ] HTML解析失败
• [ ] 解析搜索结果HTML失败
• 解析HTML失败
• [ ] 解析搜索结果失败

AI-assisted analysis of fish2018/pansou@beaa561337 (2026-09-07). Data as JSON: /api/errors/89f827f7a1c3aa56. Report an issue: GitHub.

Appendix: source

Thrown at plugin/nyaa/nyaa.go:133

	req.Header.Set("Accept-Language", "en-US,en;q=0.9,zh-CN;q=0.8,zh;q=0.7")
	req.Header.Set("Connection", "keep-alive")
	req.Header.Set("Referer", SiteURL)
	
	// 5. 发送请求（带重试机制）
	resp, err := p.doRequestWithRetry(req, client)
	if err != nil {
		return nil, fmt.Errorf("[%s] 搜索请求失败: %w", p.Name(), err)
	}
	defer resp.Body.Close()
	
	if resp.StatusCode != 200 {
		return nil, fmt.Errorf("[%s] 搜索请求返回状态码: %d", p.Name(), resp.StatusCode)
	}
	
	// 6. 解析搜索结果页面
	doc, err := goquery.NewDocumentFromReader(resp.Body)
	if err != nil {
		return nil, fmt.Errorf("[%s] 解析搜索页面失败: %w", p.Name(), err)
	}
	
	// 7. 提取搜索结果
	var results []model.SearchResult
	
	// 查找种子列表表格
	table := doc.Find("table.torrent-list tbody")
	if table.Length() == 0 {
		return []model.SearchResult{}, nil // 没有搜索结果
	}
	
	// 8. 解析每个搜索结果行
	table.Find("tr").Each(func(i int, s *goquery.Selection) {
		result := p.parseSearchRow(s)
		if result.UniqueID != "" {
			results = append(results, result)
		}
	})

View on GitHub (pinned to beaa561337)