What causes "[%s] 创建详情页请求失败: %w" in fish2018/pansou?

Resolve extracted hrefs against the base URL with resp.Request.URL.Parse or the resolve reference pattern before requesting.

fish2018/pansou · error

[ ] 创建详情页请求失败

Error message

[%s] 创建详情页请求失败: %w

What it means

fetchDetailData failed to build the GET request for a scraped detail page URL via http.NewRequestWithContext. Since detailURL comes from parsed search-result hrefs, this almost always means an extracted link was relative, malformed, or contained invalid characters.

Solutions

Resolve extracted hrefs against the base URL with resp.Request.URL.Parse or the resolve reference pattern before requesting.
Skip/normalize invalid detail URLs during extraction instead of failing.
Log the offending detailURL to confirm whether extraction or the site is at fault.

Example fix

// before
detailURL := s.Find("a").AttrOr("href", "")
// after
href := s.Find("a").AttrOr("href", "")
u, err := url.Parse(href)
if err != nil || u.Host == "" {
    return
}
detailURL := baseURL.ResolveReference(u).String()

Defensive patterns

Strategy: validation

Validate before calling

u, err := url.Parse(detailHref)
if err != nil || u.Host == "" {
    return fmt.Errorf("invalid detail URL: %q", detailHref)
}
detailURL = baseURL.ResolveReference(u).String()

Try / catch

data, err := fetchDetailData(client, detailURL)
if err != nil {
    log.Printf("skipping detail %q: %v", detailURL, err)
    return nil // skip this item, continue with others
}

Prevention

Always resolve relative hrefs against the base URL before requesting.
Skip items with unusable hrefs instead of failing the whole search.
Validate scraped URLs with url.Parse during extraction.

When it happens

Trigger: searchImpl's anonymous worker extracts a detail URL from search results, then http.NewRequestWithContext rejects it — relative path without base, spaces/control chars, or empty href.

Common situations: Site serving protocol-relative or relative hrefs; href attributes with unescaped characters; extraction selector grabbing a non-URL attribute value.

Understand the failure class

Background: "Invalid URL" errors: why new URL(), URI.parse, and reqwest::Url reject your string — missing scheme, whitespace, and bad path format — this error's family across 39 libraries.

Related errors

AI-assisted analysis of fish2018/pansou@beaa561337 (2026-09-07). Data as JSON: /api/errors/7a6d2755e78b467d. Report an issue: GitHub.

Appendix: source

Thrown at plugin/mizixing/mizixing.go:243

	})

	return items, nil
}

type detailData struct {
	links       []model.Link
	datetime    time.Time
	tags        []string
	description string
}

func (p *MizixingPlugin) fetchDetailData(client *http.Client, detailURL string) (detailData, error) {
	ctx, cancel := context.WithTimeout(context.Background(), detailTimeout)
	defer cancel()

	req, err := http.NewRequestWithContext(ctx, http.MethodGet, detailURL, nil)
	if err != nil {
		return detailData{}, fmt.Errorf("[%s] 创建详情页请求失败: %w", p.Name(), err)
	}
	setHTMLHeaders(req, detailURL)

	resp, err := p.doRequestWithRetry(req, client, maxRequestRetries)
	if err != nil {
		return detailData{}, err
	}
	defer resp.Body.Close()

	if resp.StatusCode != http.StatusOK {
		return detailData{}, fmt.Errorf("[%s] 详情页返回状态码: %d", p.Name(), resp.StatusCode)
	}

	doc, err := goquery.NewDocumentFromReader(resp.Body)
	if err != nil {
		return detailData{}, fmt.Errorf("[%s] 解析详情页失败: %w", p.Name(), err)
	}

View on GitHub (pinned to beaa561337)