fish2018/pansou · error
document returned status
Error message
document returned status %d
What it means
fetchPansoDocument scrapes a panso document page via HTTP; when the response status is not 200 OK it aborts and returns this error instead of trying to parse the body. It signals the remote page was not served successfully, so goquery parsing would operate on an error page or empty body.
Solutions
- Log resp.StatusCode and retry the request with backoff for transient 429/5xx statuses
- Send browser-like headers (User-Agent, Referer, Accept-Language) on the request to pass anti-bot checks
- Handle 404 distinctly as 'resource no longer exists' and skip the item instead of surfacing an error
- If 403 persists, check whether the site requires cookies/JS challenge and consider updating the scraper
Example fix
// before
if resp.StatusCode != http.StatusOK {
return model.SearchResult{}, fmt.Errorf("document returned status %d", resp.StatusCode)
}
// after
if resp.StatusCode == http.StatusNotFound {
return model.SearchResult{}, ErrResourceGone // skip item
}
if resp.StatusCode != http.StatusOK {
return model.SearchResult{}, fmt.Errorf("document returned status %d (waf/rate-limit possible)", resp.StatusCode)
} Defensive patterns
Strategy: retry
Validate before calling
resp, err := client.Do(req)
if err == nil && resp.StatusCode != http.StatusOK {
return fmt.Errorf("unexpected status %d", resp.StatusCode)
} Type guard
func isTransientStatus(code int) bool { return code == 429 || code >= 500 } Try / catch
resp, err := fetchPansoDocument(...)
if err != nil {
var httpErr interface{ HTTPStatus() int }
if errors.As(err, &httpErr) && isTransientStatus(...) { /* retry */ }
return model.SearchResult{}, err
} Prevention
- Send complete browser-like headers to avoid 403 anti-bot blocks
- Throttle request rate to avoid 429
- Distinguish 404 as a skippable 'gone' resource
- Check sousou/panso reachability from the host before blaming the code
When it happens
Trigger: The GET to the panso document URL returns any non-200 status (403 anti-bot block, 404 removed resource, 429 rate limit, 5xx server error) and resp.StatusCode != http.StatusOK.
Common situations: Site adds WAF/anti-scraping that returns 403 without a browser fingerprint; the shared resource was deleted (404); too many rapid requests trigger 429; transient upstream 502/503 during scraping.
Understand the failure class
Background: "API error: {status}" and "HTTP 401/403/404/429/5xx" errors: non-2xx HTTP responses explained — this error's family across 27 libraries.
Related errors
AI-assisted analysis of fish2018/pansou@beaa561337 (2026-09-07).
Data as JSON: /api/errors/bb028b223d9683da.
Report an issue: GitHub.
Appendix: source
Thrown at plugin/sousou/sousou.go:257
}
return results, nil
}
func (p *SousouAsyncPlugin) fetchPansoDocument(client *http.Client, item pansoSearchItem) (model.SearchResult, error) {
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
defer cancel()
req, err := http.NewRequestWithContext(ctx, http.MethodGet, item.DocURL, nil)
if err != nil {
return model.SearchResult{}, err
}
setSousouWebHeaders(req, SousouWebURL+"?q=")
resp, err := client.Do(req)
if err != nil {
return model.SearchResult{}, err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return model.SearchResult{}, fmt.Errorf("document returned status %d", resp.StatusCode)
}
doc, err := goquery.NewDocumentFromReader(resp.Body)
if err != nil {
return model.SearchResult{}, err
}
linkURL := strings.TrimSpace(doc.Find("a.jump-link[href]").First().AttrOr("href", ""))
if linkURL == "" {
return model.SearchResult{}, fmt.Errorf("document has no share link")
}
linkType := util.GetLinkType(linkURL)
if linkType == "others" || linkType == "" {
return model.SearchResult{}, fmt.Errorf("unsupported share link: %s", linkURL)
}
title := cleanPansoTitle(doc.Find(".resource-box h1").First().Text())
if title == "" {
title = cleanPansoTitle(item.Title)
}
datetime := item.DatetimeView on GitHub (pinned to beaa561337)