{"record":{"id":"eb413eb72b68098b","repo":"fish2018/pansou","slug":"s-d-d","errorCode":null,"errorMessage":"[%s] 第%d页请求返回状态码: %d","messagePattern":"\\[(.+?)\\] 第(.+?)页请求返回状态码: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"plugin/dy4k/dy4k.go","lineNumber":415,"sourceCode":"\t\t\tfmt.Printf(\"    URL错误: %v\\n\", netErr.Err)\n\t\t\tif netErr.Timeout() {\n\t\t\t\tfmt.Printf(\"    -> 这是超时错误\\n\")\n\t\t\t}\n\t\t\tif netErr.Temporary() {\n\t\t\t\tfmt.Printf(\"    -> 这是临时错误\\n\")\n\t\t\t}\n\t\t}\n\t\treturn nil, 0, fmt.Errorf(\"[%s] 第%d页搜索请求失败: %w\", p.Name(), page, err)\n\t}\n\tdefer resp.Body.Close()\n\n\tdebugPrintf(\"✅ [Dy4k DEBUG] HTTP请求成功 (耗时: %v)\\n\", requestDuration)\n\n\t// 6. 检查状态码\n\tdebugPrintf(\"🔧 [Dy4k DEBUG] HTTP响应状态码: %d\\n\", resp.StatusCode)\n\tif resp.StatusCode != 200 {\n\t\tdebugPrintf(\"❌ [Dy4k DEBUG] 状态码异常: %d\\n\", resp.StatusCode)\n\t\treturn nil, 0, fmt.Errorf(\"[%s] 第%d页请求返回状态码: %d\", p.Name(), page, resp.StatusCode)\n\t}\n\n\t// 7. 读取并打印HTML响应\n\thtmlBytes, err := io.ReadAll(resp.Body)\n\tif err != nil {\n\t\treturn nil, 0, fmt.Errorf(\"[%s] 第%d页读取响应失败: %w\", p.Name(), page, err)\n\t}\n\n\thtmlContent := string(htmlBytes)\n\tdebugPrintf(\"🔧 [Dy4k DEBUG] 第%d页 HTML长度: %d bytes\\n\", page, len(htmlContent))\n\n\t// 保存HTML到文件（仅在调试模式下）\n\tif DebugMode {\n\t\thtmlDir := \"./html\"\n\t\tos.MkdirAll(htmlDir, 0755)\n\n\t\tfilename := fmt.Sprintf(\"dy4k_page_%d_%s.html\", page, strings.ReplaceAll(encodedKeyword, \"%\", \"_\"))\n\t\tfilepath := filepath.Join(htmlDir, filename)","sourceCodeStart":397,"sourceCodeEnd":433,"githubUrl":"https://github.com/fish2018/pansou/blob/beaa56133755a548ebc51b090b3816e2ae044aa6/plugin/dy4k/dy4k.go#L397-L433","documentation":"searchPage rejects the response for page N because resp.StatusCode != 200, treating it as a failed page fetch. The dy4k site returned an error/challenge status instead of the expected HTML listing. This is upstream behavior (blocking, rate limiting, or site errors), not a caller bug.","triggerScenarios":"The GET for page N completes but returns 403 (anti-bot/WAF), 429 (rate limit from paginating too fast), 404 (page beyond available results), or 5xx (server error).","commonSituations":"Rapid multi-page scraping triggers rate limiting; random UA/IP tricks fail against Cloudflare; requesting pages past the last result page returns 404; site under maintenance returns 5xx.","solutions":["Log the status code (already done via debugPrintf) and capture a body snippet to confirm whether it's a WAF challenge.","Stop paginating on 404 and treat it as end-of-results rather than an error.","Add delays between page requests and back off on 429 to stay under rate limits.","Use realistic browser headers (full UA, Accept, Accept-Language, Referer) or rotate proxies if 403 persists."],"exampleFix":"// before\nif resp.StatusCode != 200 {\n    return nil, 0, fmt.Errorf(\"[%s] 第%d页请求返回状态码: %d\", p.Name(), page, resp.StatusCode)\n}\n// after\nif resp.StatusCode == 404 {\n    return nil, 0, nil // end of results\n}\nif resp.StatusCode != 200 {\n    return nil, 0, fmt.Errorf(\"[%s] 第%d页请求返回状态码: %d\", p.Name(), page, resp.StatusCode)\n}","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"if resp.StatusCode != 200 {\n    switch {\n    case resp.StatusCode == 429:\n        time.Sleep(backoff) // then retry\n    case resp.StatusCode == 404:\n        return nil, 0, nil // end of pagination\n    default:\n        return nil, 0, fmt.Errorf(\"page %d: status %d\", page, resp.StatusCode)\n    }\n}","preventionTips":["Throttle pagination requests to avoid 429.","Treat 404 as end-of-results, not failure.","Use full realistic browser headers to pass WAFs.","Rotate UA/proxy if 403s recur during scraping."],"tags":["http","status-code","rate-limit","scraping"],"backgroundTag":"http-non-200-response","analyzedSha":"beaa56133755a548ebc51b090b3816e2ae044aa6","analyzedAt":"2026-09-07T00:31:18.025Z","contentChangedAt":"2026-09-07T00:31:18.025Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}