thanos-io/thanos · error

encoding with snappy

Error message

encoding with snappy

What it means

Before storing fetched postings into the index cache, the reader compresses them with streamed snappy (snappyStreamedEncode). If that compression fails, the postings were read fine but cannot be cached, and the error aborts this postings-fetch goroutine. This is rare and points to a snappy encoder-level failure (e.g. corrupt in-memory buffer or unsupported size) rather than to user input.

Solutions

1. Retry the query — this is usually transient and postings will be re-read from storage
2. If persistent, disable postings caching (or lower cached postings limits) to stop encoding failures from failing queries
3. Upgrade Thanos to a version matching the vendored golang/snappy library and check for known encoding bugs
4. Report/persist stats.cachedPostingsCompressionErrors to correlate with specific blocks or posting sizes

Example fix

// before
dataToCache, err := snappyStreamedEncode(int(postingsCount), diffVarintPostings)
if err != nil {
	stats.cachedPostingsCompressionErrors += 1
	return errors.Wrap(err, "encoding with snappy")
}
// after
dataToCache, err := snappyStreamedEncode(int(postingsCount), diffVarintPostings)
if err != nil {
	stats.cachedPostingsCompressionErrors += 1
	level.Warn(logger).Log("msg", "skipping postings cache fill; encode failed", "err", err)
	return nil // serve result without caching instead of failing the query
}

if postingsCount == 0 || len(diffVarintPostings) == 0 || len(diffVarintPostings) > maxCacheEntrySize {
	// skip caching; serve postings without cache fill
}

func canCachePostings(count int, blob []byte) bool { return count > 0 && len(blob) > 0 && len(blob) <= maxCacheEntrySize }

dataToCache, err := snappyStreamedEncode(postingsCount, diffVarintPostings)
if err != nil {
	stats.cachedPostingsCompressionErrors += 1
	logger.Warn("snappy encode failed; skipping cache fill", "err", err)
	return nil // do not fail the query for a cache-fill problem
}

When it happens

Trigger: snappyStreamedEncode(postingsCount, diffVarintPostings) returns an error during cache-fill after successfully reading postings from object storage; typically internal buffer/stream error in the snappy streaming encoder.

Common situations: Very large postings lists stressing the streaming encoder; corrupted in-memory diffVarint postings buffer; a Thanos/snappy library version incompatibility.

Related errors

• decode postings
• endpoint
• unsupported compression type
• iterate series
• repaired block is invalid

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/caa7ceb9b5a13cc8. Report an issue: GitHub.

Appendix: source

Thrown at pkg/store/bucket.go:3233

			}
			defer runutil.CloseWithLogOnErr(r.logger, partReader, "readIndexRange close range reader")
			brdr.Reset(partReader)

			rdr := newPostingsReaderBuilder(ctx, brdr, ptrs[i:j], start, length)

			stats.postingsFetchCount++
			stats.add(PostingsFetched, j-i, int(length))

			for rdr.Next() {
				diffVarintPostings, postingsCount, keyID := rdr.AtDiffVarint()

				output[keyID] = newDiffVarintPostings(diffVarintPostings, nil)

				startCompression := time.Now()
				dataToCache, err := snappyStreamedEncode(int(postingsCount), diffVarintPostings)
				if err != nil {
					stats.cachedPostingsCompressionErrors += 1
					return errors.Wrap(err, "encoding with snappy")
				}

				stats.cachedPostingsCompressions += 1
				stats.CachedPostingsOriginalSizeSum += units.Base2Bytes(len(diffVarintPostings))
				stats.CachedPostingsCompressedSizeSum += units.Base2Bytes(len(dataToCache))
				stats.CachedPostingsCompressionTimeSum += time.Since(startCompression)
				stats.add(PostingsTouched, 1, len(diffVarintPostings))

				r.block.indexCache.StorePostings(r.block.meta.ULID, keys[keyID], dataToCache, tenant)
			}

			stats.PostingsFetchDurationSum += time.Since(begin)

			if err := rdr.Error(); err != nil {
				return errors.Wrap(err, "reading postings")
			}
			return nil
		})

View on GitHub (pinned to 35b8b99117)