{"record":{"id":"6526e65a3af30d4e","repo":"fish2018/pansou","slug":"dtoken","errorCode":null,"errorMessage":"未找到DToken","messagePattern":"未找到DToken","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"plugin/xys/xys.go","lineNumber":195,"sourceCode":"\t// 查找script标签中的DToken定义\n\tvar token string\n\tdoc.Find(\"script\").Each(func(i int, s *goquery.Selection) {\n\t\tscriptContent := s.Text()\n\t\tif strings.Contains(scriptContent, \"DToken\") {\n\t\t\t// 使用正则表达式提取token\n\t\t\tre := regexp.MustCompile(`const\\s+DToken\\s*=\\s*\"([^\"]+)\"`)\n\t\t\tmatches := re.FindStringSubmatch(scriptContent)\n\t\t\tif len(matches) > 1 {\n\t\t\t\ttoken = matches[1]\n\t\t\t\tif p.debugMode {\n\t\t\t\t\tlog.Printf(\"[XYS] 从script中提取到token: %s\", token[:10]+\"...\")\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t})\n\n\tif token == \"\" {\n\t\treturn \"\", fmt.Errorf(\"未找到DToken\")\n\t}\n\n\t// 缓存token\n\tp.tokenCache.Store(cacheKey, TokenCache{\n\t\tToken:     token,\n\t\tTimestamp: time.Now(),\n\t})\n\n\treturn token, nil\n}\n\n// doRequestWithRetry 带重试机制的HTTP请求\nfunc (p *XysPlugin) doRequestWithRetry(req *http.Request, client *http.Client) (*http.Response, error) {\n\tmaxRetries := 3\n\tvar lastErr error\n\t\n\tfor i := 0; i < maxRetries; i++ {\n\t\tif i > 0 {","sourceCodeStart":177,"sourceCodeEnd":213,"githubUrl":"https://github.com/fish2018/pansou/blob/beaa56133755a548ebc51b090b3816e2ae044aa6/plugin/xys/xys.go#L177-L213","documentation":"getToken scraped the token page successfully but no script tag contained a DToken definition, so the token variable stayed empty. The plugin depends on this client-side token to build authenticated search requests; without it searching is impossible. This is a page-structure/anti-bot change on the upstream site rather than a network fault.","triggerScenarios":"After iterating doc.Find(\"script\"), token == \"\" — the DToken regex found no match in any inline script (site renamed the variable, moved token into external JS/headers, or served a challenge page with 200).","commonSituations":"Upstream site updated its front-end and renamed/moved DToken; served a consent/anti-bot page with HTTP 200; token now delivered via cookie or API endpoint instead of inline script.","solutions":["Fetch the token page manually and inspect script contents to find the new token variable name or delivery mechanism","Update the extraction regex/selector in getToken to match the current script pattern","Check whether the token now comes from a cookie (e.g. DToken cookie) or a JSON endpoint and scrape that instead","Detect challenge pages (markers like 'captcha'/'verify') and surface a clearer error instead of '未找到DToken'"],"exampleFix":"// before\nif token == \"\" {\n    return \"\", fmt.Errorf(\"未找到DToken\")\n}\n// after\nif token == \"\" {\n    return \"\", fmt.Errorf(\"未找到DToken: token页面结构可能已变更或返回了验证页面 (url=%s, len=%d)\", tokenURL, len(pageHTML))\n}","handlingStrategy":"fallback","validationCode":"html, _ := io.ReadAll(resp.Body)\nif !strings.Contains(string(html), \"DToken\") {\n    return fmt.Errorf(\"token page no longer exposes DToken; scraper needs update\")\n}","typeGuard":null,"tryCatchPattern":"token, err := getToken(ctx)\nif errors.Is(err, errDTokenNotFound) {\n    return scrapeTokenFromCookieJar(client) // alternate extraction path\n}","preventionTips":["Add a canary test that fetches the token page and asserts DToken extraction works","Detect challenge/consent pages explicitly and report a distinct error","Cache tokens to reduce scraping frequency and lower detection risk"],"tags":["scraping","token","page-structure-change"],"backgroundTag":"resource-not-found","analyzedSha":"beaa56133755a548ebc51b090b3816e2ae044aa6","analyzedAt":"2026-09-07T00:31:18.025Z","contentChangedAt":"2026-09-07T00:31:18.025Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}