What causes "read got %d bytes instead of 4" in thanos-io/thanos?

Fix or replace the io.Reader wrapper violating the io.Reader contract (short read without EOF error).

read got bytes instead of 4

Error message

read got %d bytes instead of 4

What it means

getInt32 also returns fmt.Errorf("read got %d bytes instead of 4") when io.ReadFull returned successfully but the count differs — a defensive check that should only trigger for readers behaving inconsistently (short reads without proper error). It indicates malformed postings binary data.

Solutions

Fix or replace the io.Reader wrapper violating the io.Reader contract (short read without EOF error).
Verify the postings data offsets are in sync — a corrupted stream likely desynchronized the decoder.
Regenerate/re-download the block index data.
Add a buffered reader (bufio) to normalize read behavior over the underlying source.

Example fix

// before
r := myCustomReader(raw)
// after
r := bufio.NewReader(io.MultiReader(bytes.NewReader(header), raw))

Defensive patterns

Strategy: try-catch

Validate before calling

// ensure underlying reader honors io.Reader contract
var probe [4]byte
if n, err := io.ReadFull(reader, probe[:]); err != nil || n != 4 { return err }

Try / catch

if err != nil {
  var serr interface{ ShortRead() bool } // or match on message
  if strings.Contains(err.Error(), "instead of 4") {
    // reader contract violation: replace reader or resync stream offsets
  }
}

Prevention

Wrap raw sources in bufio.Reader to normalize short reads
Test custom io.Reader implementations against io.ReadFull
Never reuse a postings reader after an error without resyncing

When it happens

Trigger: A reader implementation returning fewer than 4 bytes without returning an error from Read, corrupting the postings count decoding in postingsReaderBuilder.Next.

Common situations: Custom/wrapped io.Reader implementations with buggy Read semantics; corrupted mmap regions where ReadFull semantics degrade; desynchronized offsets in hand-crafted postings streams.

Related errors

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/b8e56c0938b5c16c. Report an issue: GitHub.

Appendix: source

Thrown at pkg/store/postings.go:57

		r:                r,
		readBuf:          make([]byte, 4),
		start:            start,
		length:           length,
		postings:         postings,
		uvarintEncodeBuf: make([]byte, binary.MaxVarintLen64),
		ctx:              ctx,
	}

	return prb
}

func getInt32(r io.Reader, buf []byte) (uint32, error) {
	read, err := io.ReadFull(r, buf)
	if err != nil {
		return 0, errors.Wrap(err, "reading")
	}
	if read != 4 {
		return 0, fmt.Errorf("read got %d bytes instead of 4", read)
	}
	return binary.BigEndian.Uint32(buf), nil
}

func (r *postingsReaderBuilder) Next() bool {
	if r.ctx.Err() != nil {
		r.e = r.ctx.Err()
		return false
	}
	if r.repeatFor > 0 {
		r.keyID = r.postings[r.pi-r.repeatFor].keyID
		r.repeatFor--
		return true
	}
	if r.pi >= len(r.postings) {
		return false
	}
	if r.Error() != nil {

View on GitHub (pinned to 35b8b99117)