{"record":{"id":"fd05b7a8f14b7600","repo":"fish2018/pansou","slug":"parse-web-search-page-failed-w","errorCode":null,"errorMessage":"parse web search page failed: %w","messagePattern":"parse web search page failed: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"plugin/sousou/sousou.go","lineNumber":192,"sourceCode":"\n\tsearchURL := SousouWebURL + \"?q=\" + url.QueryEscape(strings.TrimSpace(keyword))\n\treq, err := http.NewRequestWithContext(ctx, http.MethodGet, searchURL, nil)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"create web search request failed: %w\", err)\n\t}\n\tsetSousouWebHeaders(req, \"https://www.panso.vip/\")\n\tresp, err := client.Do(req)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"web search request failed: %w\", err)\n\t}\n\tdefer resp.Body.Close()\n\tif resp.StatusCode != http.StatusOK {\n\t\treturn nil, fmt.Errorf(\"web search returned status %d\", resp.StatusCode)\n\t}\n\n\tdoc, err := goquery.NewDocumentFromReader(resp.Body)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"parse web search page failed: %w\", err)\n\t}\n\n\titems := make([]pansoSearchItem, 0, 20)\n\tdoc.Find(\"div.search-item\").Each(func(_ int, item *goquery.Selection) {\n\t\tanchor := item.Find(\"a.search-item-title[href]\").First()\n\t\thref := strings.TrimSpace(anchor.AttrOr(\"href\", \"\"))\n\t\tif href == \"\" {\n\t\t\treturn\n\t\t}\n\t\titems = append(items, pansoSearchItem{\n\t\t\tDocURL:   absolutePansoURL(href),\n\t\t\tTitle:    strings.TrimSpace(anchor.Text()),\n\t\t\tContent:  strings.TrimSpace(item.Find(\".search-item-info\").Text()),\n\t\t\tDiskType: strings.TrimSpace(item.Find(\".search-item-logo\").AttrOr(\"alt\", \"\")),\n\t\t\tDatetime: parsePansoDatetime(item.Find(\".search-item-meta-item\").Text()),\n\t\t\tPassword: parsePansoPassword(item),\n\t\t})\n\t})","sourceCodeStart":174,"sourceCodeEnd":210,"githubUrl":"https://github.com/fish2018/pansou/blob/beaa56133755a548ebc51b090b3816e2ae044aa6/plugin/sousou/sousou.go#L174-L210","documentation":"Once a 200 response is received, searchWeb parses it with goquery. If the body cannot be turned into an HTML document, 'parse web search page failed' is returned. It indicates the 200 response body was not valid/readable HTML for the panso search page.","triggerScenarios":"goquery.NewDocumentFromReader(resp.Body) errors inside searchWeb after a successful 200 response — truncated body, encoding problems, or non-HTML content served with 200.","commonSituations":"Anti-bot served a 200 with a JS-challenge page or empty body; transfer cut off; charset/encoding mismatch breaking the parser.","solutions":["Dump the first bytes of the body and Content-Type to confirm what was actually returned.","Retry the request; intermittent truncation is common under load.","Verify setSousouWebHeaders includes Accept/Accept-Encoding that yield plain HTML.","If the page now requires JavaScript, switch to a headless-browser fetch for this source."],"exampleFix":"// before\ndoc, err := goquery.NewDocumentFromReader(resp.Body)\nif err != nil {\n    return nil, fmt.Errorf(\"parse web search page failed: %w\", err)\n}\n// after\nbody, _ := io.ReadAll(resp.Body)\nif len(bytes.TrimSpace(body)) == 0 {\n    return nil, fmt.Errorf(\"parse web search page failed: empty body\")\n}\ndoc, err := goquery.NewDocumentFromReader(bytes.NewReader(body))","handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":"results, err := plugin.Search(keyword)\nif err != nil && strings.Contains(err.Error(), \"parse web search page failed\") {\n    // capture body sample for debugging, retry once, then fall back to another source\n}","preventionTips":["Read the body fully before parsing so failures are diagnosable","Verify Accept/Accept-Encoding headers yield plain HTML","Detect captcha/challenge pages explicitly before parsing","Alert on sudden parse-failure increases (site redesign signal)"],"tags":["html-parsing","goquery","scraping","go"],"backgroundTag":"unexpected-response-shape","analyzedSha":"beaa56133755a548ebc51b090b3816e2ae044aa6","analyzedAt":"2026-09-07T00:31:18.025Z","contentChangedAt":"2026-09-07T00:31:18.025Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}