What causes "cannot decompress response body from %s: %w" in VictoriaMetrics/VictoriaMetrics?

Curl the scrape URL with gzip and inspect whether the body is valid gzip (file/gunzip).

cannot decompress response body from %s: %w

Error message

cannot decompress response body from %s: %w

What it means

During a scrape, readFromBuffer decompresses the target's gzip-encoded response body via protoparserutil.GetUncompressedReader. This error means the gzip stream from the scrape target is malformed or truncated and cannot be decompressed; the scrape for that target fails. The scrape URL is included for identification.

Source

Thrown at lib/promscrape/scrapework.go:506

	leveledbytebufferpool.Put(body)

	<-processScrapedDataConcurrencyLimitCh

	return err
}

var processScrapedDataConcurrencyLimitCh = make(chan struct{}, cgroup.AvailableCPUs())

func (sw *scrapeWork) readFromBuffer(dst *bytesutil.ByteBuffer, src *chunkedbuffer.Buffer, isGzipped bool) error {
	if !isGzipped {
		src.MustWriteTo(dst)
		return nil
	}

	r := src.NewReader()
	reader, err := protoparserutil.GetUncompressedReader(r, "gzip")
	if err != nil {
		return fmt.Errorf("cannot decompress response body from %s: %w", sw.Config.ScrapeURL, err)
	}

	lr := ioutil.GetLimitedReader(reader, sw.Config.MaxScrapeSize+1)
	_, err = dst.ReadFrom(lr)
	ioutil.PutLimitedReader(lr)

	protoparserutil.PutUncompressedReader(reader)
	if err != nil {
		return fmt.Errorf("cannot read gzipped response body from %s: %w", sw.Config.ScrapeURL, err)
	}

	if int64(dst.Len()) > sw.Config.MaxScrapeSize {
		maxScrapeSizeExceeded.Inc()
		return fmt.Errorf("the uncompressed response from %q exceeds -promscrape.maxScrapeSize or max_scrape_size in the scrape config (%d bytes). "+
			"Possible solutions are: reduce the response size for the target, increase -promscrape.maxScrapeSize command-line flag, "+
			"increase max_scrape_size value in scrape config for the given target", sw.Config.ScrapeURL, sw.Config.MaxScrapeSize)
	}
	return nil

View on GitHub (pinned to 5079fb58f1)

Solutions

Curl the scrape URL with gzip and inspect whether the body is valid gzip (file/gunzip).
Check intermediate proxies/load balancers for response rewriting or compression bugs.
Verify the target does not send conflicting Content-Encoding; fix server compression settings.
Retry if caused by transient network truncation of the response.

Example fix

// before: server sends plain text while client requests gzip
Content-Encoding: identity
// after: target must honor gzip negotiation or send valid gzip
Content-Encoding: gzip (with actual gzip-compressed body)

Defensive patterns

Strategy: retry

Validate before calling

// verify the target serves valid gzip
curl -sH 'Accept-Encoding: gzip' -o /tmp/m.gz $SCRAPE_URL && gunzip -t /tmp/m.gz && echo OK

Try / catch

if strings.Contains(err.Error(), "cannot decompress response body") {
    // re-scrape once; if persistent, inspect proxy/server encoding
}

Prevention

Test gzip output of exporters with curl + gunzip -t
Check proxies don't corrupt or double-encode bodies
Avoid scraping non-metrics endpoints by mistake

When it happens

Trigger: A scrape target (through getTargetResponse/scrapeInternal) returns a body advertised as gzip ('Content-Encoding: gzip') that is invalid gzip data, corrupted, or cut off mid-stream.

Common situations: Target behind a proxy that re-encodes or corrupts responses; a server that gzips only headers but sends plain body (double-encoding bugs); scraping an endpoint that returns binary/non-gzip data while gzip is requested; network truncation of large responses.

Related errors

AI-assisted analysis of VictoriaMetrics/VictoriaMetrics@5079fb58f1 (2026-09-03). Data as JSON: /api/errors/70c7e3a9a3b93afe. Report an issue: GitHub.