thanos-io/thanos · error

get symbols from object storage of

Error message

get symbols from object storage of %s

What it means

CopySymbols streams the symbol table section of the index (bytes [toc.Symbols, toc.Series)) from the object-storage index file to a writer via a ranged GET. A failing GetRange is wrapped with this message. It is raised while an index-header is being built and symbols copied from the raw index.

Solutions

1. Check the wrapped error: InvalidRange/416 indicates corrupt/truncated index — re-upload or restore the block.
2. Verify IAM credentials allow ranged GetObject on the bucket prefix.
3. Retry transient/throttling errors with backoff or via a retrying bucket client.
4. Ensure retention/lifecycle rules are not deleting the block while index-headers are being built.

if toc.Series < toc.Symbols || toc.Series > uint64(attrs.Size) {
    return fmt.Errorf("TOC offsets inconsistent with object size; corrupt index %s", path)
}

if strings.Contains(err.Error(), "InvalidRange") {
    // truncated index: restore block from replica
} else { retry.WithBackoff(copySymbols) }

When it happens

Trigger: CopySymbols when bkt.GetRange(ctx, path, toc.Symbols, toc.Series-toc.Symbols) errors: missing object, denied access, throttling, or an invalid range because the TOC offsets do not fit the actual object (corrupt TOC, wrong object size).

Common situations: S3 416/InvalidRange when TOC offsets exceed the real object (truncated index); throttling under heavy store-gateway load; missing IAM GetObject permissions; block deleted concurrently by retention while being read.

Understand the failure class

Background: 'Something went wrong' / 'Request failed (500)' / 'HTTP error! status: 404' — what failed HTTP requests actually mean and how to find the real cause — this error's family across 28 libraries.

Related errors

• get TOC from object storage of
• get posting offset table from object storage of
• compaction
• sync before first pass of downsampling
• sync before second pass of downsampling

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/9e96e686a1f09d07. Report an issue: GitHub.

Appendix: source

Thrown at pkg/block/indexheader/binary_reader.go:255

		runutil.CloseWithErrCapture(&err, rc, "close toc reader")
		return nil, errors.Wrapf(err, "get TOC from object storage of %s", r.path)
	}

	if err := rc.Close(); err != nil {
		return nil, errors.Wrap(err, "close toc reader")
	}

	toc, err := index.NewTOCFromByteSlice(realByteSlice(tocBytes))
	if err != nil {
		return nil, errors.Wrap(err, "new TOC")
	}
	return toc, nil
}

func (r *chunkedIndexReader) CopySymbols(w io.Writer, buf []byte) (err error) {
	rc, err := r.bkt.GetRange(r.ctx, r.path, int64(r.toc.Symbols), int64(r.toc.Series-r.toc.Symbols))
	if err != nil {
		return errors.Wrapf(err, "get symbols from object storage of %s", r.path)
	}
	defer runutil.CloseWithErrCapture(&err, rc, "close symbol reader")

	if _, err := io.CopyBuffer(w, rc, buf); err != nil {
		return errors.Wrap(err, "copy symbols")
	}

	return nil
}

func (r *chunkedIndexReader) CopyPostingsOffsets(w io.Writer, buf []byte) (err error) {
	rc, err := r.bkt.GetRange(r.ctx, r.path, int64(r.toc.PostingsTable), int64(r.size-r.toc.PostingsTable))
	if err != nil {
		return errors.Wrapf(err, "get posting offset table from object storage of %s", r.path)
	}
	defer runutil.CloseWithErrCapture(&err, rc, "close posting offsets reader")

	if _, err := io.CopyBuffer(w, rc, buf); err != nil {

View on GitHub (pinned to 35b8b99117)