{"record":{"id":"533123f6598bc272","repo":"fish2018/pansou","slug":"s-parse-json-failed-on-page-d-w","errorCode":null,"errorMessage":"[%s] parse json failed on page %d: %w","messagePattern":"\\[(.+?)\\] parse json failed on page (.+?): %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"plugin/discourse/discourse.go","lineNumber":253,"sourceCode":"\t\t// 读取响应体\n\t\tbody, err := io.ReadAll(resp.Body)\n\t\tresp.Body.Close()\n\t\tif err != nil {\n\t\t\tif len(allResults) > 0 {\n\t\t\t\tfmt.Printf(\"[%s] Warning: failed to read page %d: %v\\n\", p.Name(), currentPage, err)\n\t\t\t\tbreak\n\t\t\t}\n\t\t\treturn nil, fmt.Errorf(\"[%s] read response failed on page %d: %w\", p.Name(), currentPage, err)\n\t\t}\n\n\t\t// 解析JSON响应\n\t\tvar searchResp SearchResponse\n\t\tif err := json.Unmarshal(body, &searchResp); err != nil {\n\t\t\tif len(allResults) > 0 {\n\t\t\t\tfmt.Printf(\"[%s] Warning: failed to parse page %d: %v\\n\", p.Name(), currentPage, err)\n\t\t\t\tbreak\n\t\t\t}\n\t\t\treturn nil, fmt.Errorf(\"[%s] parse json failed on page %d: %w\", p.Name(), currentPage, err)\n\t\t}\n\n\t\t// 如果没有帖子了，停止获取\n\t\tif len(searchResp.Posts) == 0 {\n\t\t\tbreak\n\t\t}\n\t\t\n\t\t// 转换为SearchResult并去重\n\t\tpageResults := p.convertToSearchResults(searchResp)\n\t\t\n\t\t// 添加结果（去重）\n\t\tfor _, result := range pageResults {\n\t\t\t// 从 UniqueID 中提取帖子ID\n\t\t\tvar postID int\n\t\t\tfmt.Sscanf(result.UniqueID, \"discourse-%d\", &postID)\n\t\t\t\n\t\t\tif !seenPostIDs[postID] {\n\t\t\t\tseenPostIDs[postID] = true","sourceCodeStart":235,"sourceCodeEnd":271,"githubUrl":"https://github.com/fish2018/pansou/blob/beaa56133755a548ebc51b090b3816e2ae044aa6/plugin/discourse/discourse.go#L235-L271","documentation":"After reading a Discourse search page, searchImpl unmarshals the body into SearchResponse. If json.Unmarshal fails on the first page (no accumulated results), it returns this error wrapping the JSON error with the page number; otherwise it warns and stops.","triggerScenarios":"A Discourse search page body is not valid JSON matching SearchResponse — e.g. an HTML Cloudflare challenge/login page or an error page was returned with status 200.","commonSituations":"Cloudflare or anti-bot interstitial served with 200 status, forum returning HTML error pages, API response shape changed across Discourse versions.","solutions":["Inspect the raw response body to confirm whether it is HTML (bot challenge) instead of JSON","Ensure cloudscraper is properly initialized so anti-bot challenges are solved before the request","Verify the site's search API endpoint still returns the expected JSON shape","Log body[:200] on parse failure to diagnose the actual content"],"exampleFix":"// before\nif err := json.Unmarshal(body, &searchResp); err != nil {\n    return nil, fmt.Errorf(\"[%s] parse json failed on page %d: %w\", p.Name(), currentPage, err)\n}\n// after\nif err := json.Unmarshal(body, &searchResp); err != nil {\n    return nil, fmt.Errorf(\"[%s] parse json failed on page %d (body head: %q): %w\", p.Name(), currentPage, body[:min(200, len(body))], err)\n}","handlingStrategy":"try-catch","validationCode":"// pre-flight: confirm the search endpoint returns JSON\nresp, _ := http.Get(baseURL + \"/search.json?q=test\")\nct, _ := io.ReadAll(io.LimitReader(resp.Body, 64))\nif !strings.HasPrefix(strings.TrimSpace(string(ct)), \"{\") {\n    return errors.New(\"endpoint not returning JSON; anti-bot likely\")\n}","typeGuard":null,"tryCatchPattern":"links, err := plugin.Search(query)\nif err != nil && strings.Contains(err.Error(), \"parse json failed\") {\n    log.Printf(\"non-JSON response from discourse (bot challenge?); refreshing scraper session\")\n    return nil, ErrBotChallenge\n}","preventionTips":["Keep cloudscraper initialized and cookies fresh","Check for 200-with-HTML responses when sites update their anti-bot","Pin/verify Discourse API schema when upgrading a forum"],"tags":["json","parsing","http","discourse"],"backgroundTag":"json-unmarshal-failed","analyzedSha":"beaa56133755a548ebc51b090b3816e2ae044aa6","analyzedAt":"2026-09-07T00:31:18.025Z","contentChangedAt":"2026-09-07T00:31:18.025Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}