fish2018/pansou · error

[ ] 解析搜索结果失败

Error message

[%s] 解析搜索结果失败: %w

What it means

Returned by searchImpl when parseEmbeddedList fails to extract or decode the search results embedded in the page HTML. parseEmbeddedList locates a JS snippet ('const list = JSON.parse(' or 'let listItems = JSON.parse('), scans the single-quoted string, decodes JS escapes, and unmarshals JSON; any failure in that pipeline is surfaced here wrapped with %w. It almost always means the site's page structure/JavaScript changed and the scraper's markers no longer match.

Solutions

1. errors.Unwrap the cause to distinguish marker-missing vs scan/decode vs JSON unmarshal failures
2. Fetch the page with curl and diff against the expected markers (const list = JSON.parse(' / let listItems = JSON.parse(') — update listMarkers to match the new markup
3. If the site moved to an API, rewrite parseEmbeddedList to call the JSON API directly
4. Check whether an anti-bot page was served (look for Cloudflare markers in the body) and improve headers

Example fix

// before
var listMarkers = [][]byte{
    []byte("const list = JSON.parse('"),
    []byte("let listItems = JSON.parse('"),
}
// after
var listMarkers = [][]byte{
    []byte("const list = JSON.parse('"),
    []byte("let listItems = JSON.parse('"),
    []byte("const searchResults = JSON.parse('"), // new marker observed on site
}

results, err := plugin.Search(kw, nil)
if err != nil {
    // unwrap to see which parsing stage failed
    log.Warnf("haitunsou parse failed: %v", err)
    // degrade gracefully: continue with other plugins' results
}

When it happens

Trigger: The page HTML no longer contains either list marker, the embedded string is malformed/truncated, or the embedded JSON no longer matches the []searchItem schema (e.g. items became an object).

Common situations: Upstream site redesign changes variable names or moves results into an API endpoint; CDN serves a challenge page instead of search results; partial/truncated HTML delivery.

Understand the failure class

Background: "invalid response format", "malformed payload", "missing data field": when an API returns 200 but the response shape is wrong — this error's family across 23 libraries.

Related errors

• [ ] 解析HTML失败
• [ ] HTML解析失败
• 未找到页面内嵌结果
• 无法提取帖子ID: index=
• [ ] 解析搜索结果HTML失败

AI-assisted analysis of fish2018/pansou@beaa561337 (2026-09-07). Data as JSON: /api/errors/061cbc14b616e3b1. Report an issue: GitHub.

Appendix: source

Thrown at plugin/haitunsou/haitunsou.go:113

	resp, err := client.Do(req)
	if err != nil {
		return nil, fmt.Errorf("[%s] 搜索请求失败: %w", p.Name(), err)
	}
	defer resp.Body.Close()
	if resp.StatusCode != http.StatusOK {
		return nil, fmt.Errorf("[%s] 搜索请求返回 HTTP %d", p.Name(), resp.StatusCode)
	}
	body, err := io.ReadAll(io.LimitReader(resp.Body, maxResponseSize+1))
	if err != nil {
		return nil, fmt.Errorf("[%s] 读取搜索响应失败: %w", p.Name(), err)
	}
	if len(body) > maxResponseSize {
		return nil, fmt.Errorf("[%s] 搜索响应超过 %d 字节", p.Name(), maxResponseSize)
	}

	items, err := parseEmbeddedList(body)
	if err != nil {
		return nil, fmt.Errorf("[%s] 解析搜索结果失败: %w", p.Name(), err)
	}
	results := make([]model.SearchResult, 0, len(items))
	seen := make(map[string]struct{}, len(items))
	for _, item := range items {
		result, ok := convertItem(item)
		if !ok {
			continue
		}
		key := result.Links[0].URL + "\x00" + result.Links[0].Password
		if _, exists := seen[key]; exists {
			continue
		}
		seen[key] = struct{}{}
		results = append(results, result)
	}
	return plugin.FilterResultsByKeyword(results, keyword), nil
}

View on GitHub (pinned to beaa561337)