thanos-io/thanos · error

creating snappy compressor

Error message

creating snappy compressor: %w

What it means

The encoder wraps any error from extsnappy.Compressor.Compress (constructing the snappy stream writer) with this message. It means the snappy compression writer could not be initialized before encoding postings.

Solutions

1. Check the underlying wrapped error (%w) for the root cause
2. Verify the snappy dependency (github.com/golang/snappy / extsnappy) builds correctly in your binary
3. Retry the encoding; this is typically transient

// ensure the compressor is initialized once and healthy at startup
if extsnappy.Compressor == nil {
    return errors.New("snappy compressor not initialized")
}

b, err := diffVarintSnappyStreamedEncode(p, length)
if err != nil {
    var wrapped *fmt.wrapError
    if errors.As(err, &wrapped) && strings.Contains(err.Error(), "creating snappy compressor") {
        return retryEncoder(p, length) // retry after transient failure
    }
    return nil, err
}

When it happens

Trigger: encodePostingsToCache -> diffVarintSnappyStreamedEncode when extsnappy.Compressor.Compress(compressedBuf) returns an error (e.g., failure setting up the streaming compressor).

Common situations: Faulty or misconfigured external snappy compressor build (extsnappy is a vendored Prometheus snappy wrapper); resource exhaustion during writer setup.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• encoding with snappy
• writing streamed snappy header
• short-write streamed snappy header
• writing uvarint encoded byte
• short-write for uvarint encoded byte

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/affe0ffdb7261c0c. Report an issue: GitHub.

Appendix: source

Thrown at pkg/store/postings_codec.go:105

		ret += snappy.MaxEncodedLen(length)
	}

	return ret
}

func diffVarintSnappyStreamedEncode(p index.Postings, length int) ([]byte, error) {
	compressedBuf := bytes.NewBuffer(make([]byte, 0, estimateSnappyStreamSize(length)))
	if n, err := compressedBuf.WriteString(codecHeaderStreamedSnappy); err != nil {
		return nil, fmt.Errorf("writing streamed snappy header")
	} else if n != len(codecHeaderStreamedSnappy) {
		return nil, fmt.Errorf("short-write streamed snappy header")
	}

	uvarintEncodeBuf := make([]byte, binary.MaxVarintLen64)

	sw, err := extsnappy.Compressor.Compress(compressedBuf)
	if err != nil {
		return nil, fmt.Errorf("creating snappy compressor: %w", err)
	}

	prev := storage.SeriesRef(0)
	for p.Next() {
		v := p.At()
		if v < prev {
			return nil, errors.Errorf("postings entries must be in increasing order, current: %d, previous: %d", v, prev)
		}

		uvarintSize := binary.PutUvarint(uvarintEncodeBuf, uint64(v-prev))
		if written, err := sw.Write(uvarintEncodeBuf[:uvarintSize]); err != nil {
			return nil, errors.Wrap(err, "writing uvarint encoded byte")
		} else if written != uvarintSize {
			return nil, errors.Wrap(err, "short-write for uvarint encoded byte")
		}

		prev = v
	}

View on GitHub (pinned to 35b8b99117)