{"record":{"id":"ef6c4bf2a4aa76be","repo":"fish2018/pansou","slug":"s-html-w-ef6c4b","errorCode":null,"errorMessage":"[%s] 网页搜索 HTML 解析失败: %w","messagePattern":"\\[(.+?)\\] 网页搜索 HTML 解析失败: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"plugin/feikuai/feikuai.go","lineNumber":193,"sourceCode":"\treq, err := http.NewRequestWithContext(ctx, http.MethodGet, searchURL, nil)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"[%s] 创建网页搜索请求失败: %w\", p.Name(), err)\n\t}\n\treq.Header.Set(\"User-Agent\", \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 Chrome/136.0.0.0 Safari/537.36\")\n\treq.Header.Set(\"Accept\", \"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\")\n\treq.Header.Set(\"Accept-Language\", \"zh-CN,zh;q=0.9,en;q=0.8\")\n\treq.Header.Set(\"Referer\", \"https://feikuai.in/\")\n\tresp, err := client.Do(req)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"[%s] API 失败且网页搜索请求失败: %v (API: %v)\", p.Name(), err, apiErr)\n\t}\n\tdefer resp.Body.Close()\n\tif resp.StatusCode != http.StatusOK {\n\t\treturn nil, fmt.Errorf(\"[%s] API 失败且网页搜索返回状态码 %d (API: %v)\", p.Name(), resp.StatusCode, apiErr)\n\t}\n\tdoc, err := goquery.NewDocumentFromReader(resp.Body)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"[%s] 网页搜索 HTML 解析失败: %w\", p.Name(), err)\n\t}\n\n\tresults := make([]model.SearchResult, 0, 64)\n\tseen := make(map[string]struct{})\n\tadd := func(linkURL, title, content string, datetime time.Time) {\n\t\tlinkURL = strings.TrimSpace(linkURL)\n\t\tif linkURL == \"\" {\n\t\t\treturn\n\t\t}\n\t\tlinkType := util.GetLinkType(linkURL)\n\t\tif linkType == \"\" || linkType == \"others\" {\n\t\t\treturn\n\t\t}\n\t\tif _, ok := seen[linkURL]; ok {\n\t\t\treturn\n\t\t}\n\t\tseen[linkURL] = struct{}{}\n\t\tif title == \"\" {","sourceCodeStart":175,"sourceCodeEnd":211,"githubUrl":"https://github.com/fish2018/pansou/blob/beaa56133755a548ebc51b090b3816e2ae044aa6/plugin/feikuai/feikuai.go#L175-L211","documentation":"The feikuai plugin falls back to scraping a web search page when its primary API call fails. goquery could not parse the HTML response body returned by the fallback search endpoint. This indicates the response was not valid/complete HTML (blocked page, truncated body, or non-HTML content type).","triggerScenarios":"searchWeb: the API attempt failed (apiErr set), the fallback web-search HTTP response returned status 200, but goquery.NewDocumentFromReader failed to parse resp.Body's HTML.","commonSituations":"Site returns a captcha/challenge page with malformed HTML, CDN truncates the response, server returns JSON or binary instead of HTML on the fallback URL, network proxy corrupts the body.","solutions":["Retry the request; transient truncation often succeeds on a second attempt.","Check the fallback search endpoint with curl to see what content is actually returned (captcha page, JSON, empty body).","Set a realistic User-Agent/headers so the site does not serve an anti-bot page.","Improve API reliability (credentials, rate limits) so the HTML fallback is not needed."],"exampleFix":"// before\ndoc, err := goquery.NewDocumentFromReader(resp.Body)\nif err != nil {\n    return nil, fmt.Errorf(\"[%s] 网页搜索 HTML 解析失败: %w\", p.Name(), err)\n}\n// after\ndoc, err := goquery.NewDocumentFromReader(io.LimitReader(resp.Body, 5<<20))\nif err != nil {\n    log.Printf(\"[%s] fallback HTML parse failed (status %d, content-type %s)\", p.Name(), resp.StatusCode, resp.Header.Get(\"Content-Type\"))\n    return nil, fmt.Errorf(\"[%s] 网页搜索 HTML 解析失败: %w\", p.Name(), err)\n}","handlingStrategy":"try-catch","validationCode":"// before calling search, check the site responds with HTML\nresp, err := http.Get(fallbackURL)\nif err == nil {\n    ct := resp.Header.Get(\"Content-Type\")\n    if !strings.Contains(ct, \"text/html\") {\n        // skip fallback scrape\n    }\n}","typeGuard":"func isHTMLResponse(resp *http.Response) bool {\n    return resp != nil && strings.Contains(resp.Header.Get(\"Content-Type\"), \"text/html\")\n}","tryCatchPattern":"results, err := plugin.Search(keyword)\nif err != nil {\n    var parseErr *goqueryError // or use errors.Is/As on wrapped causes\n    log.Printf(\"search failed (fallback HTML unparseable): %v\", err)\n    results = []model.SearchResult{} // degrade gracefully\n}","preventionTips":["Set a browser-like User-Agent to avoid anti-bot pages","Validate Content-Type is text/html before parsing","Retry transient failures before giving up","Keep the primary API healthy so the HTML fallback is rarely used"],"tags":["html-parsing","go","web-scraping","goquery"],"backgroundTag":"invalid-html-response","analyzedSha":"beaa56133755a548ebc51b090b3816e2ae044aa6","analyzedAt":"2026-09-07T00:31:18.025Z","contentChangedAt":"2026-09-07T00:31:18.025Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}