thanos-io/thanos · error

missing magic snappy marker

Error message

missing magic snappy marker

What it means

A chunk of type chunkTypeCompressedData (Snappy-compressed postings) was encountered before the stream's Snappy identifier chunk was read. The format requires the "sNaPpY" magic identifier chunk to appear first so the decoder initializes its Snappy state; without it the compressed chunk cannot be trusted, so the iterator fails with this error.

Solutions

1. Ensure the full postings blob including the leading Snappy identifier chunk is passed to the decoder; do not slice off the first bytes.
2. Re-fetch/rebuild the postings data if the header was lost (re-compact the TSDB block).
3. Restart iteration from the beginning of the blob rather than from an offset after an earlier error.
4. Check the writing side (postings codec version) matches the reading side.

if !isDiffVarintSnappyEncodedPostings(blob) {
    return fmt.Errorf("stream must start with snappy identifier chunk")
}

When it happens

Trigger: Iterating a postings stream that starts directly with a compressed-data chunk because the leading identifier/magic chunk is missing, was skipped, or the buffer was sliced past it.

Common situations: Hand-built or truncated postings buffers, tools that strip the header before re-serving data, or resuming iteration mid-stream after a previous error left readSnappyIdentifier unset.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

• got bad identifier
• mismatched checksum (got , expected )
• unsupported chunk type
• invalid chunk type
• snappy decode

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/0c57c9028b1922f0. Report an issue: GitHub.

Appendix: source

Thrown at pkg/store/postings_codec.go:265

	it.input = it.input[3:]

	switch chunkType {
	case chunkTypeStreamIdentifier:
		const magicBody = "sNaPpY"
		if chunkLen != len(magicBody) {
			it.err = fmt.Errorf("corrupted identifier")
			return false
		}
		if string(it.input[:len(magicBody)]) != magicBody {
			it.err = fmt.Errorf("got bad identifier %s", string(it.input[:6]))
			return false
		}
		it.input = it.input[6:]
		it.readSnappyIdentifier = true
		return it.readNextChunk(nil)
	case chunkTypeCompressedData:
		if !it.readSnappyIdentifier {
			it.err = fmt.Errorf("missing magic snappy marker")
			return false
		}
		if len(it.input) < 4 {
			it.err = io.ErrUnexpectedEOF
			return false
		}
		checksum := uint32(it.input[0]) | uint32(it.input[1])<<8 | uint32(it.input[2])<<16 | uint32(it.input[3])<<24
		if len(it.input) < chunkLen {
			it.err = io.ErrUnexpectedEOF
			return false
		}

		if it.buf == nil {
			if it.disablePooling {
				it.buf = make([]byte, it.maximumDecodedLen)
			} else {
				b, err := decodedBufPool.Get(it.maximumDecodedLen)
				if err != nil {

View on GitHub (pinned to 35b8b99117)