fish2018/pansou · error
HTTP 状态码
Error message
HTTP 状态码 %d
What it means
This error is created inside doRequestWithRetry when client.Do succeeded (err == nil) but the response status was not 200 after every retry. The last non-200 status code is recorded as lastErr and ultimately returned wrapped as '重试 3 次后失败: HTTP 状态码 %d'. It represents the upstream server answering, but with an unacceptable status (403, 429, 5xx, etc.).
Solutions
- Read the status code from the message and react: 403 → rotate User-Agent/Referer or reduce request rate; 429 → increase retryBaseDelay and reduce maxDetailWorkers; 404 → drop the dead URL; 5xx → retry later.
- Note a bug in the source: when err != nil, resp is nil and the code accesses resp.StatusCode only when err == nil, but on transport errors lastErr may be a wrapped 'client.Do' error — ensure you surface the wrapped error via errors.Unwrap to find the true cause.
- Slow down concurrency (maxDetailWorkers=8 parallel detail fetches) if you see 429s.
- Use the same headers a browser would send (setHTMLHeaders/setAPIHeaders already do this) and verify with curl whether the block is IP-based; consider a proxy if your host IP is blocked.
- Increase maxRequestRetries or use jittered exponential backoff for transient 5xx.
Example fix
// before
lastErr = err
if err == nil {
lastErr = fmt.Errorf("HTTP 状态码 %d", resp.StatusCode)
}
// after
lastErr = err
if err == nil {
lastErr = fmt.Errorf("HTTP 状态码 %d", resp.StatusCode)
}
if resp != nil && (resp.StatusCode == http.StatusTooManyRequests || resp.StatusCode >= 500) {
// retryable — increase backoff for these statuses
time.Sleep(retryBaseDelay * time.Duration(1<<attempt))
} Defensive patterns
Strategy: retry
Validate before calling
// pre-check the URL responds 200 before heavy parallel fetching
resp, err := client.Head(detailURL)
if err == nil && resp.StatusCode == http.StatusNotFound {
log.Printf("skipping dead detail page: %s", detailURL)
return
} Try / catch
resp, err := p.doRequestWithRetry(req, client, maxRequestRetries)
if err != nil {
if strings.Contains(err.Error(), "状态码 429") || strings.Contains(err.Error(), "状态码 403") {
// blocked/rate-limited — wait longer before the next attempt
time.Sleep(30 * time.Second)
}
return nil, err
} Prevention
- Throttle concurrency (maxDetailWorkers) and add jittered backoff to avoid 429 rate limiting.
- Send full browser-like headers; if a datacenter IP is blocked (403), route through a proxy.
- Skip URLs that repeatedly return 404 instead of retrying them.
- Distinguish retryable (429, 5xx) from non-retryable (401, 403, 404) statuses before spending retry budget.
When it happens
Trigger: All retry attempts receive a response with StatusCode != http.StatusOK from the target (homepage, posts API, or a detail page). Callers fetchDataKey, fetchPosts, and fetchDetailLinks all route through this; e.g. a detail page returns 404 or the API returns 403 due to bot blocking.
Common situations: Cloudflare/bot protection blocking datacenter IPs with 403; detail post URLs returning 404 after posts were deleted; rate limiting (429) from aggressive parallel detail fetches (8 workers × retries); temporary 502/503 during site maintenance.
Understand the failure class
Background: "API error: {status}" and "HTTP 401/403/404/429/5xx" errors: non-2xx HTTP responses explained — this error's family across 27 libraries.
Related errors
AI-assisted analysis of fish2018/pansou@beaa561337 (2026-09-07).
Data as JSON: /api/errors/5b672174fe52e2ab.
Report an issue: GitHub.
Appendix: source
Thrown at plugin/jsnoteclub/jsnoteclub.go:509
}
}
return matched
}
func (p *JsNoteClubPlugin) doRequestWithRetry(req *http.Request, client *http.Client, maxRetries int) (*http.Response, error) {
var lastErr error
for attempt := 0; attempt < maxRetries; attempt++ {
resp, err := client.Do(req.Clone(req.Context()))
if err == nil && resp.StatusCode == http.StatusOK {
return resp, nil
}
if resp != nil {
resp.Body.Close()
}
lastErr = err
if err == nil {
lastErr = fmt.Errorf("HTTP 状态码 %d", resp.StatusCode)
}
if attempt < maxRetries-1 {
time.Sleep(retryBaseDelay * time.Duration(1<<attempt))
}
}
return nil, fmt.Errorf("重试 %d 次后失败: %w", maxRetries, lastErr)
}
func newHTTPClient() *http.Client {
return &http.Client{
Timeout: requestTimeout,
Transport: &http.Transport{
MaxIdleConns: httpMaxIdleConns,
MaxIdleConnsPerHost: httpMaxIdlePerHost,
MaxConnsPerHost: httpMaxConnsPerHost,
IdleConnTimeout: 90 * time.Second,
TLSHandshakeTimeout: 10 * time.Second,View on GitHub (pinned to beaa561337)