fish2018/pansou · error

[ ] parse json failed on page

Error message

[%s] parse json failed on page %d: %w

What it means

After reading a Discourse search page, searchImpl unmarshals the body into SearchResponse. If json.Unmarshal fails on the first page (no accumulated results), it returns this error wrapping the JSON error with the page number; otherwise it warns and stops.

Solutions

1. Inspect the raw response body to confirm whether it is HTML (bot challenge) instead of JSON
2. Ensure cloudscraper is properly initialized so anti-bot challenges are solved before the request
3. Verify the site's search API endpoint still returns the expected JSON shape
4. Log body[:200] on parse failure to diagnose the actual content

Example fix

// before
if err := json.Unmarshal(body, &searchResp); err != nil {
    return nil, fmt.Errorf("[%s] parse json failed on page %d: %w", p.Name(), currentPage, err)
}
// after
if err := json.Unmarshal(body, &searchResp); err != nil {
    return nil, fmt.Errorf("[%s] parse json failed on page %d (body head: %q): %w", p.Name(), currentPage, body[:min(200, len(body))], err)
}

// pre-flight: confirm the search endpoint returns JSON
resp, _ := http.Get(baseURL + "/search.json?q=test")
ct, _ := io.ReadAll(io.LimitReader(resp.Body, 64))
if !strings.HasPrefix(strings.TrimSpace(string(ct)), "{") {
    return errors.New("endpoint not returning JSON; anti-bot likely")
}

links, err := plugin.Search(query)
if err != nil && strings.Contains(err.Error(), "parse json failed") {
    log.Printf("non-JSON response from discourse (bot challenge?); refreshing scraper session")
    return nil, ErrBotChallenge
}

When it happens

Trigger: A Discourse search page body is not valid JSON matching SearchResponse — e.g. an HTML Cloudflare challenge/login page or an error page was returned with status 200.

Common situations: Cloudflare or anti-bot interstitial served with 200 status, forum returning HTML error pages, API response shape changed across Discourse versions.

Understand the failure class

Background: "failed to unmarshal" / json.Unmarshal errors: why parsing a response into a Go struct fails and how to fix it — this error's family across 23 libraries.

Related errors

• parse json failed
• [ ] 网盘第 页JSON解析失败
• [ ] JSON解析失败
• JSON解析失败
• 解析响应失败

AI-assisted analysis of fish2018/pansou@beaa561337 (2026-09-07). Data as JSON: /api/errors/533123f6598bc272. Report an issue: GitHub.

Appendix: source

Thrown at plugin/discourse/discourse.go:253

		// 读取响应体
		body, err := io.ReadAll(resp.Body)
		resp.Body.Close()
		if err != nil {
			if len(allResults) > 0 {
				fmt.Printf("[%s] Warning: failed to read page %d: %v\n", p.Name(), currentPage, err)
				break
			}
			return nil, fmt.Errorf("[%s] read response failed on page %d: %w", p.Name(), currentPage, err)
		}

		// 解析JSON响应
		var searchResp SearchResponse
		if err := json.Unmarshal(body, &searchResp); err != nil {
			if len(allResults) > 0 {
				fmt.Printf("[%s] Warning: failed to parse page %d: %v\n", p.Name(), currentPage, err)
				break
			}
			return nil, fmt.Errorf("[%s] parse json failed on page %d: %w", p.Name(), currentPage, err)
		}

		// 如果没有帖子了，停止获取
		if len(searchResp.Posts) == 0 {
			break
		}
		
		// 转换为SearchResult并去重
		pageResults := p.convertToSearchResults(searchResp)
		
		// 添加结果（去重）
		for _, result := range pageResults {
			// 从 UniqueID 中提取帖子ID
			var postID int
			fmt.Sscanf(result.UniqueID, "discourse-%d", &postID)
			
			if !seenPostIDs[postID] {
				seenPostIDs[postID] = true

View on GitHub (pinned to beaa561337)