{"record":{"id":"77c110843b267106","repo":"fish2018/pansou","slug":"429-too-many-requests","errorCode":null,"errorMessage":"429 Too Many Requests","messagePattern":"429 Too Many Requests","errorType":"exception","errorClass":null,"httpStatus":429,"severity":"error","filePath":"plugin/javdb/javdb.go","lineNumber":325,"sourceCode":"\t\tif p.debugMode {\n\t\t\tlog.Printf(\"[JAVDB] 遇到429限流，尝试 %d/%d\", attempt+1, MaxRetryOnRateLimit+1)\n\t\t}\n\t\t\n\t\t// 如果不允许重试或已达到最大重试次数\n\t\tif MaxRetryOnRateLimit == 0 || attempt >= MaxRetryOnRateLimit {\n\t\t\tatomic.StoreInt32(&p.rateLimited, 1)\n\t\t\tresp.Body.Close()\n\t\t\treturn nil, fmt.Errorf(\"[%s] 429限流，%s\", p.Name(), \n\t\t\t\tfunc() string {\n\t\t\t\t\tif MaxRetryOnRateLimit == 0 {\n\t\t\t\t\t\treturn \"不重试\"\n\t\t\t\t\t}\n\t\t\t\t\treturn fmt.Sprintf(\"重试%d次后仍然限流\", MaxRetryOnRateLimit)\n\t\t\t\t}())\n\t\t}\n\t\t\n\t\tresp.Body.Close()\n\t\tlastErr = fmt.Errorf(\"429 Too Many Requests\")\n\t}\n\t\n\treturn nil, lastErr\n}\n\n// parseSearchResults 解析搜索结果HTML\nfunc (p *JavdbPlugin) parseSearchResults(doc *goquery.Document) ([]model.SearchResult, error) {\n\tvar results []model.SearchResult\n\n\tif p.debugMode {\n\t\t// 检查是否找到了.movie-list元素\n\t\tmovieListEl := doc.Find(\".movie-list\")\n\t\tlog.Printf(\"[JAVDB] 找到.movie-list元素数量: %d\", movieListEl.Length())\n\t\t\n\t\t// 检查是否找到了.item元素\n\t\titemEls := doc.Find(\".movie-list .item\")\n\t\tlog.Printf(\"[JAVDB] 找到.movie-list .item元素数量: %d\", itemEls.Length())\n\t\t","sourceCodeStart":307,"sourceCodeEnd":343,"githubUrl":"https://github.com/fish2018/pansou/blob/beaa56133755a548ebc51b090b3816e2ae044aa6/plugin/javdb/javdb.go#L307-L343","documentation":"doRequestWithRateLimitRetry retries a JAVDB HTTP request when the server answers HTTP 429 (rate limited). On the final retry attempt, instead of returning the 429-specific wrapped error, the loop sets lastErr = \"429 Too Many Requests\" and falls through to `return nil, lastErr` — so the caller receives a bare 429 error after exhausting MaxRetryOnRateLimit retries with random backoff delays.","triggerScenarios":"fetchDetailPageMagnetLinks requests a JAVDB detail page and every attempt (initial + MaxRetryOnRateLimit retries) receives HTTP 429 from the JAVDB server, exhausting the retry loop.","commonSituations":"Scraping too many detail pages too fast; many plugin goroutines querying JAVDB concurrently from one IP; JAVDB tightening anti-bot rate limits; long-running crawls without cooldown triggering IP-based throttling.","solutions":["Increase delays between JAVDB requests or add global rate limiting across goroutines","Reduce concurrent search/detail workers or add a shared cooldown when p.rateLimited is set","Honor the existing random backoff (MinRetryDelay/MaxRetryDelay) and raise MaxRetryOnRateLimit only if the site tolerates it","Check for proxy/ban status — a persistent 429 may be IP-level throttling; rotate proxies or wait longer","Propagate the richer 429-limited error (line 315 branch) by bounding attempts correctly instead of leaking the bare lastErr"],"exampleFix":"// before\nlastErr = fmt.Errorf(\"429 Too Many Requests\")\n}\nreturn nil, lastErr\n\n// after\nlastErr = fmt.Errorf(\"[%s] 429 Too Many Requests after %d retries\", p.Name(), MaxRetryOnRateLimit)\n}\nreturn nil, lastErr","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"// Go: caller of fetchDetailPageMagnetLinks\nresults, err := p.fetchDetailPageMagnetLinks(url, client)\nif err != nil {\n\tif strings.Contains(err.Error(), \"429\") {\n\t\tselect {\n\t\tcase <-time.After(cooldown):\n\t\tcase <-ctx.Done():\n\t\t\treturn ctx.Err()\n\t\t}\n\t\tresults, err = p.fetchDetailPageMagnetLinks(url, client)\n\t}\n\tif err != nil {\n\t\treturn nil, err\n\t}\n}","preventionTips":["Throttle requests to JAVDB globally (e.g. golang.org/x/time/rate limiter)","Cap concurrent goroutines touching the site","Back off exponentially when any 429 is observed","Monitor p.rateLimited and pause the whole plugin when set"],"tags":["http","rate-limit","retry","scraping"],"backgroundTag":"rate-limit-exceeded","analyzedSha":"beaa56133755a548ebc51b090b3816e2ae044aa6","analyzedAt":"2026-09-07T00:31:18.025Z","contentChangedAt":"2026-09-07T00:31:18.025Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}