fish2018/pansou · error

[ ] 请求返回状态码

Error message

[%s] 请求返回状态码: %d

What it means

jutoushe searchImpl returns this when the HTTP response status code is anything other than 200 after a successful request. It indicates the site responded but rejected the scrape attempt (or the page moved). No body inspection is done before failing.

Solutions

  1. Log the actual status code and response body snippet to identify the cause
  2. Refresh the User-Agent and other headers to a current browser value
  3. Add backoff/rate-limiting between scrape requests to avoid 429/403
  4. Handle redirects (check if 3xx and follow to the new URL structure)

Example fix

// before
if resp.StatusCode != 200 {
    return nil, fmt.Errorf("[%s] 请求返回状态码: %d", p.Name(), resp.StatusCode)
}
// after
if resp.StatusCode != 200 {
    body, _ := io.ReadAll(io.LimitReader(resp.Body, 512))
    return nil, fmt.Errorf("[%s] 请求返回状态码: %d, body: %s", p.Name(), resp.StatusCode, body)
}
Defensive patterns

Strategy: retry

Validate before calling

resp, err := http.Head(baseURL + "/")
if err != nil || resp.StatusCode == http.StatusForbidden {
    return errors.New("target site is blocking requests; adjust headers/proxy first")
}

Try / catch

results, err := plugin.Search(keyword)
if err != nil {
    var statusErr interface{ Error() string }
    if strings.Contains(err.Error(), "请求返回状态码") {
        log.Printf("non-200 from source, backing off: %v", err)
        return backoffAndRetry(err)
    }
    return err
}

Prevention

When it happens

Trigger: Any non-200 status from the search endpoint: 403 (bot blocking), 404 (page moved), 429 (rate limit), 5xx (server error).

Common situations: Site added WAF/anti-bot protection rejecting the fixed Chrome/91 User-Agent; search path changed; request throttled after frequent scraping.

Related errors

AI-assisted analysis of fish2018/pansou@beaa561337 (2026-09-07). Data as JSON: /api/errors/7c3d3eb2866c1e56. Report an issue: GitHub.

Appendix: source

Thrown at plugin/jutoushe/jutoushe.go:76

	}

	// 4. 设置请求头，避免反爬虫检测
	req.Header.Set("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36")
	req.Header.Set("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8")
	req.Header.Set("Accept-Language", "zh-CN,zh;q=0.9,en;q=0.8")
	req.Header.Set("Connection", "keep-alive")
	req.Header.Set("Referer", baseURL+"/")

	// 5. 发送HTTP请求（带重试机制）
	resp, err := p.doRequestWithRetry(req, client)
	if err != nil {
		return nil, fmt.Errorf("[%s] 搜索请求失败: %w", p.Name(), err)
	}
	defer resp.Body.Close()

	// 6. 检查状态码
	if resp.StatusCode != 200 {
		return nil, fmt.Errorf("[%s] 请求返回状态码: %d", p.Name(), resp.StatusCode)
	}

	// 7. 解析搜索结果页面
	doc, err := goquery.NewDocumentFromReader(resp.Body)
	if err != nil {
		return nil, fmt.Errorf("[%s] HTML解析失败: %w", p.Name(), err)
	}

	// 8. 提取搜索结果
	var results []model.SearchResult
	doc.Find("ul.erx-list li.item").Each(func(i int, s *goquery.Selection) {
		// 提取标题和链接
		linkElem := s.Find(".a a.main")
		title := strings.TrimSpace(linkElem.Text())
		detailPath, exists := linkElem.Attr("href")
		
		if !exists || title == "" {
			return // 跳过无效项

View on GitHub (pinned to beaa561337)