fish2018/pansou · error

解析搜索数据失败

Error message

解析搜索数据失败: %w

What it means

Wraps a json.Unmarshal failure while parsing the extracted _obj.search JSON (matches[1]) into the SearchData struct. The regex found candidate JSON but its content is not valid JSON or does not fit SearchData's expected shape (e.g. type mismatch on Q/N fields).

Solutions

  1. Inspect the wrapped %w error and the raw JSON printed under DebugLog to see the exact mismatch.
  2. Update the SearchData struct to match the site's current field names/types (use json.RawMessage or flexible types).
  3. Tighten or update searchDataPattern so it captures only the complete JSON object.
  4. Check the site for a frontend/API update and adjust the parser accordingly.

Example fix

// before
type SearchData struct {
    N string `json:"n"`
}
// after — tolerate number-or-string count
type SearchData struct {
    N json.Number `json:"n"`
}
Defensive patterns

Strategy: try-catch

Validate before calling

if len(matches) < 2 || !json.Valid(matches[1]) {
    return errors.New("extracted search JSON is not valid")
}

Try / catch

var searchData SearchData
if err := json.Unmarshal(matches[1], &searchData); err != nil {
    var ute *json.UnmarshalTypeError
    if errors.As(err, &ute) {
        // field type changed upstream: log field and raw payload for triage
        log.Printf("field %s type mismatch, raw: %s", ute.Field, matches[1])
    }
    return err
}

Prevention

When it happens

Trigger: searchWithScraper extracts matches[1] from the page and json.Unmarshal returns an error: malformed/truncated JSON in the page, JSON field types changed (e.g. N became a number instead of string), or the regex captured the wrong span of text.

Common situations: Site update changed the embedded JSON structure or field types; greedy/lazy regex captured trailing HTML; page truncated mid-JSON due to response size limits or proxy issues.

Understand the failure class

Background: "failed to unmarshal" / json.Unmarshal errors: why parsing a response into a Go struct fails and how to fix it — this error's family across 23 libraries.

Related errors

AI-assisted analysis of fish2018/pansou@beaa561337 (2026-09-07). Data as JSON: /api/errors/501e521f35b1f067. Report an issue: GitHub.

Appendix: source

Thrown at plugin/gying/gying.go:2523

		}
		return nil, fmt.Errorf("未找到搜索结果数据")
	}

	if DebugLog {
		jsonStr := string(matches[1])
		if len(jsonStr) > 200 {
			jsonStr = jsonStr[:200] + "..."
		}
		fmt.Printf("[Gying] 提取的JSON数据: %s\n", jsonStr)
	}

	var searchData SearchData
	if err := json.Unmarshal(matches[1], &searchData); err != nil {
		if DebugLog {
			fmt.Printf("[Gying] JSON解析失败: %v\n", err)
			fmt.Printf("[Gying] 原始JSON: %s\n", string(matches[1]))
		}
		return nil, fmt.Errorf("解析搜索数据失败: %w", err)
	}

	if DebugLog {
		fmt.Printf("[Gying] 搜索数据解析成功:\n")
		fmt.Printf("[Gying]   - 关键词: %s\n", searchData.Q)
		fmt.Printf("[Gying]   - 结果数量字符串: %s\n", searchData.N)
		fmt.Printf("[Gying]   - 资源ID数组长度: %d\n", len(searchData.L.I))
		fmt.Printf("[Gying]   - 标题数组长度: %d\n", len(searchData.L.Title))
		if len(searchData.L.I) > 0 {
			fmt.Printf("[Gying]   - 前3个资源ID: %v\n", searchData.L.I[:min(3, len(searchData.L.I))])
			fmt.Printf("[Gying]   - 前3个标题: %v\n", searchData.L.Title[:min(3, len(searchData.L.Title))])
		}
	}

	// 3. 刷新防爬cookies（关键！访问详情页触发vrg_sc、vrg_go等防爬cookies）
	if DebugLog {
		fmt.Printf("[Gying] 刷新防爬cookies...\n")
	}

View on GitHub (pinned to beaa561337)