thanos-io/thanos · error

get posting offset table from object storage of

Error message

get posting offset table from object storage of %s

What it means

CopyPostingsOffsets streams the postings offset table (bytes from toc.PostingsTable to end of file) from the index in object storage to the writer, again via a ranged GET. A failing GetRange is wrapped with this message — the posting offset table could not be fetched from the bucket.

Solutions

1. Check the wrapped error type: range/416 errors mean the index is truncated or the TOC is wrong — re-upload or restore the block.
2. Confirm IAM allows ranged GetObject; re-check bucket/prefix configuration.
3. Apply retries with backoff for transient storage errors (retrying bucket middleware).
4. Verify blocks with thanos tools bucket verify and quarantine corrupt ones.

if toc.PostingsTable >= uint64(attrs.Size) {
    return fmt.Errorf("postings table offset %d beyond object %s", toc.PostingsTable, path)
}

var apiErr s3types.NoSuchKey
if errors.As(err, &apiErr) { /* block gone: re-sync */ }
else if isThrottling(err) { retry.WithBackoff(op) }

When it happens

Trigger: CopyPostingsOffsets when bkt.GetRange(ctx, path, toc.PostingsTable, size-toc.PostingsTable) errors: missing object, permissions, throttling, or an invalid range when toc.PostingsTable exceeds the real object size (corrupt TOC or truncated index).

Common situations: S3 InvalidRange on truncated index objects; IAM lacking GetObject on the prefix; throttling (503/SlowDown) during bulk header builds across many blocks; block removed by retention mid-copy.

Understand the failure class

Background: 'Something went wrong' / 'Request failed (500)' / 'HTTP error! status: 404' — what failed HTTP requests actually mean and how to find the real cause — this error's family across 28 libraries.

Related errors

• get TOC from object storage of
• get symbols from object storage of
• copy posting offsets
• compaction
• sync before first pass of downsampling

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/6b9549aabd0157da. Report an issue: GitHub.

Appendix: source

Thrown at pkg/block/indexheader/binary_reader.go:269

func (r *chunkedIndexReader) CopySymbols(w io.Writer, buf []byte) (err error) {
	rc, err := r.bkt.GetRange(r.ctx, r.path, int64(r.toc.Symbols), int64(r.toc.Series-r.toc.Symbols))
	if err != nil {
		return errors.Wrapf(err, "get symbols from object storage of %s", r.path)
	}
	defer runutil.CloseWithErrCapture(&err, rc, "close symbol reader")

	if _, err := io.CopyBuffer(w, rc, buf); err != nil {
		return errors.Wrap(err, "copy symbols")
	}

	return nil
}

func (r *chunkedIndexReader) CopyPostingsOffsets(w io.Writer, buf []byte) (err error) {
	rc, err := r.bkt.GetRange(r.ctx, r.path, int64(r.toc.PostingsTable), int64(r.size-r.toc.PostingsTable))
	if err != nil {
		return errors.Wrapf(err, "get posting offset table from object storage of %s", r.path)
	}
	defer runutil.CloseWithErrCapture(&err, rc, "close posting offsets reader")

	if _, err := io.CopyBuffer(w, rc, buf); err != nil {
		return errors.Wrap(err, "copy posting offsets")
	}

	return nil
}

// TODO(bwplotka): Add padding for efficient read.
type binaryWriter struct {
	writer PosWriter

	toc BinaryTOC

	// Reusable memory.
	buf encoding.Encbuf

View on GitHub (pinned to 35b8b99117)