fish2018/pansou · error

创建gzip reader失败

Error message

创建gzip reader失败: %w

What it means

getResponseReader detected Content-Encoding: gzip on the HTTP response but gzip.NewReader failed to wrap resp.Body. gzip.NewReader fails on the first bytes if the body is not actually a valid gzip stream. This error propagates to all callers: searchImpl, fetchDetailPageLinks, and fetchPanLink.

Solutions

  1. Log the first bytes of resp.Body (after Peek) to verify whether the stream is really gzip
  2. Handle double-decompression: sniff the body magic (0x1f 0x8b) instead of trusting the header, and pass through if not gzip
  3. Disable automatic transparent decompression that conflicts with manual gzip handling (avoid custom Transport DisableCompression mismatches)
  4. Retry the request once, since transient proxies can send mismatched bodies
  5. Check brotli/deflate: if Content-Encoding is br or deflate, add appropriate decoders instead of falling through

Example fix

// before
if contentEncoding == "gzip" {
    gzReader, err := gzip.NewReader(resp.Body)
    if err != nil {
        return nil, fmt.Errorf("创建gzip reader失败: %w", err)
    }
    reader = gzReader
}
// after
if contentEncoding == "gzip" {
    br := bufio.NewReader(resp.Body)
    magic, _ := br.Peek(2)
    if len(magic) == 2 && magic[0] == 0x1f && magic[1] == 0x8b {
        gzReader, err := gzip.NewReader(br)
        if err != nil {
            return nil, fmt.Errorf("创建gzip reader失败: %w", err)
        }
        reader = gzReader
    } else {
        reader = br // server lied about gzip; use raw body
    }
}
Defensive patterns

Strategy: validation

Validate before calling

// Validate gzip magic bytes before wrapping
br := bufio.NewReader(resp.Body)
magic, _ := br.Peek(2)
if resp.Header.Get("Content-Encoding") == "gzip" && !(len(magic) == 2 && magic[0] == 0x1f && magic[1] == 0x8b) {
    return fmt.Errorf("声明gzip但内容非法: % x", magic)
}

Type guard

func isGzipStream(r io.Reader) (bool, io.Reader) {
    br := bufio.NewReader(r)
    magic, _ := br.Peek(2)
    return len(magic) == 2 && magic[0] == 0x1f && magic[1] == 0x8b, br
}

Try / catch

doc, err := fetchAndParse(url)
if err != nil && strings.Contains(err.Error(), "创建gzip reader失败") {
    // encoding mismatch: retry once with a fresh client, or sniff-and-decode manually
    doc, err = fetchAndParseWithSniffing(url)
}

Prevention

When it happens

Trigger: Server sent Content-Encoding: gzip header but the body is not gzip (e.g. a proxy/WAF already decompressed it, body is an error page, or an encoding mismatch like brotli mislabeled as gzip).

Common situations: Reverse proxy or CDN strips/alters encoding; site misconfigures Content-Encoding; middleware double-decompresses; server returns compressed error page with wrong header.

Related errors

AI-assisted analysis of fish2018/pansou@beaa561337 (2026-09-07). Data as JSON: /api/errors/0d4e1b34155962ad. Report an issue: GitHub.

Appendix: source

Thrown at plugin/libvio/libvio.go:199

	return filteredResults, nil
}

// getResponseReader 获取响应读取器（处理gzip压缩）
func (p *LibvioPlugin) getResponseReader(resp *http.Response) (io.Reader, error) {
	var reader io.Reader = resp.Body

	// 检查Content-Encoding
	contentEncoding := resp.Header.Get("Content-Encoding")
	if p.debugMode {
		log.Printf("[Libvio] Content-Encoding: %s", contentEncoding)
	}

	// 如果是gzip压缩，手动解压
	if contentEncoding == "gzip" {
		gzReader, err := gzip.NewReader(resp.Body)
		if err != nil {
			return nil, fmt.Errorf("创建gzip reader失败: %w", err)
		}
		// 注意：不要在这里关闭gzReader，它需要在外部使用
		reader = gzReader
	}

	return reader, nil
}

// extractSearchResults 从HTML中提取搜索结果
func (p *LibvioPlugin) extractSearchResults(doc *goquery.Document, keyword string) []model.SearchResult {
	var results []model.SearchResult

	// 选择所有搜索结果项
	doc.Find("ul.stui-vodlist li").Each(func(i int, s *goquery.Selection) {
		// 提取标题和详情页链接
		titleElem := s.Find(".stui-vodlist__detail h4 a")
		title := strings.TrimSpace(titleElem.Text())
		if title == "" {

View on GitHub (pinned to beaa561337)