thanos-io/thanos · error

copy symbols

Error message

copy symbols

What it means

CopySymbols fetched the symbols range successfully but io.CopyBuffer failed while streaming it into the destination writer, wrapped with 'copy symbols'. The underlying reader is closed with error capture via defer.

Solutions

1. Inspect the wrapped error: ENOSPC/EACCES on the writer means free disk space or fix permissions on the index-header cache directory.
2. If the error is a read/network failure, retry the copy; enable objstore retries.
3. Enlarge the local cache volume or cap its usage to avoid disk-full during header builds.
4. Check context deadlines if the copy is canceled by timeouts/shutdown.

if free, _ := diskFree(cacheDir); free < minRequiredBytes {
    return fmt.Errorf("insufficient disk for index-header cache: %s", cacheDir)
}

var perr *fs.PathError
if errors.As(err, &perr) {
    // local write failed: check disk space/permissions on cache dir
} else if errors.Is(err, context.DeadlineExceeded) {
    // retry copy with larger deadline
}

When it happens

Trigger: CopySymbols when io.CopyBuffer(w, rc, buf) errors: destination writer (the index-header file being written) fails — disk full, local I/O error — or the source body read fails mid-stream (network drop, context cancel).

Common situations: Disk full on the store-gateway volume where index-headers are cached; local filesystem permission or I/O errors; network interruption during a large symbols section read; context deadline from a slow copy.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• copy posting offsets
• read meta file
• get header from object storage of
• close reader
• close toc reader

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/78da323a77f9c215. Report an issue: GitHub.

Appendix: source

Thrown at pkg/block/indexheader/binary_reader.go:260

		return nil, errors.Wrap(err, "close toc reader")
	}

	toc, err := index.NewTOCFromByteSlice(realByteSlice(tocBytes))
	if err != nil {
		return nil, errors.Wrap(err, "new TOC")
	}
	return toc, nil
}

func (r *chunkedIndexReader) CopySymbols(w io.Writer, buf []byte) (err error) {
	rc, err := r.bkt.GetRange(r.ctx, r.path, int64(r.toc.Symbols), int64(r.toc.Series-r.toc.Symbols))
	if err != nil {
		return errors.Wrapf(err, "get symbols from object storage of %s", r.path)
	}
	defer runutil.CloseWithErrCapture(&err, rc, "close symbol reader")

	if _, err := io.CopyBuffer(w, rc, buf); err != nil {
		return errors.Wrap(err, "copy symbols")
	}

	return nil
}

func (r *chunkedIndexReader) CopyPostingsOffsets(w io.Writer, buf []byte) (err error) {
	rc, err := r.bkt.GetRange(r.ctx, r.path, int64(r.toc.PostingsTable), int64(r.size-r.toc.PostingsTable))
	if err != nil {
		return errors.Wrapf(err, "get posting offset table from object storage of %s", r.path)
	}
	defer runutil.CloseWithErrCapture(&err, rc, "close posting offsets reader")

	if _, err := io.CopyBuffer(w, rc, buf); err != nil {
		return errors.Wrap(err, "copy posting offsets")
	}

	return nil
}

View on GitHub (pinned to 35b8b99117)