fish2018/pansou · error

[ ] 第 页请求返回状态码

Error message

[%s] 第%d页请求返回状态码: %d

What it means

searchPage rejects the response for page N because resp.StatusCode != 200, treating it as a failed page fetch. The dy4k site returned an error/challenge status instead of the expected HTML listing. This is upstream behavior (blocking, rate limiting, or site errors), not a caller bug.

Solutions

  1. Log the status code (already done via debugPrintf) and capture a body snippet to confirm whether it's a WAF challenge.
  2. Stop paginating on 404 and treat it as end-of-results rather than an error.
  3. Add delays between page requests and back off on 429 to stay under rate limits.
  4. Use realistic browser headers (full UA, Accept, Accept-Language, Referer) or rotate proxies if 403 persists.

Example fix

// before
if resp.StatusCode != 200 {
    return nil, 0, fmt.Errorf("[%s] 第%d页请求返回状态码: %d", p.Name(), page, resp.StatusCode)
}
// after
if resp.StatusCode == 404 {
    return nil, 0, nil // end of results
}
if resp.StatusCode != 200 {
    return nil, 0, fmt.Errorf("[%s] 第%d页请求返回状态码: %d", p.Name(), page, resp.StatusCode)
}
Defensive patterns

Strategy: retry

Try / catch

if resp.StatusCode != 200 {
    switch {
    case resp.StatusCode == 429:
        time.Sleep(backoff) // then retry
    case resp.StatusCode == 404:
        return nil, 0, nil // end of pagination
    default:
        return nil, 0, fmt.Errorf("page %d: status %d", page, resp.StatusCode)
    }
}

Prevention

When it happens

Trigger: The GET for page N completes but returns 403 (anti-bot/WAF), 429 (rate limit from paginating too fast), 404 (page beyond available results), or 5xx (server error).

Common situations: Rapid multi-page scraping triggers rate limiting; random UA/IP tricks fail against Cloudflare; requesting pages past the last result page returns 404; site under maintenance returns 5xx.

Related errors

AI-assisted analysis of fish2018/pansou@beaa561337 (2026-09-07). Data as JSON: /api/errors/eb413eb72b68098b. Report an issue: GitHub.

Appendix: source

Thrown at plugin/dy4k/dy4k.go:415

			fmt.Printf("    URL错误: %v\n", netErr.Err)
			if netErr.Timeout() {
				fmt.Printf("    -> 这是超时错误\n")
			}
			if netErr.Temporary() {
				fmt.Printf("    -> 这是临时错误\n")
			}
		}
		return nil, 0, fmt.Errorf("[%s] 第%d页搜索请求失败: %w", p.Name(), page, err)
	}
	defer resp.Body.Close()

	debugPrintf("✅ [Dy4k DEBUG] HTTP请求成功 (耗时: %v)\n", requestDuration)

	// 6. 检查状态码
	debugPrintf("🔧 [Dy4k DEBUG] HTTP响应状态码: %d\n", resp.StatusCode)
	if resp.StatusCode != 200 {
		debugPrintf("❌ [Dy4k DEBUG] 状态码异常: %d\n", resp.StatusCode)
		return nil, 0, fmt.Errorf("[%s] 第%d页请求返回状态码: %d", p.Name(), page, resp.StatusCode)
	}

	// 7. 读取并打印HTML响应
	htmlBytes, err := io.ReadAll(resp.Body)
	if err != nil {
		return nil, 0, fmt.Errorf("[%s] 第%d页读取响应失败: %w", p.Name(), page, err)
	}

	htmlContent := string(htmlBytes)
	debugPrintf("🔧 [Dy4k DEBUG] 第%d页 HTML长度: %d bytes\n", page, len(htmlContent))

	// 保存HTML到文件（仅在调试模式下）
	if DebugMode {
		htmlDir := "./html"
		os.MkdirAll(htmlDir, 0755)

		filename := fmt.Sprintf("dy4k_page_%d_%s.html", page, strings.ReplaceAll(encodedKeyword, "%", "_"))
		filepath := filepath.Join(htmlDir, filename)

View on GitHub (pinned to beaa561337)