thanos-io/thanos · error

write index header TOC

Error message

write index header TOC

What it means

WriteTOC writes the Table-Of-Contents section (offsets to symbols, postings table, etc.) at the end of the index-header stream. This error wraps a failure from bw.WriteTOC(), typically a write/flush error inside the TOC serialization (buffered writer write failure) rather than a data-content problem.

Solutions

1. Check remaining disk space; TOC writes fail when the volume filled during earlier writes.
2. Inspect storage health for late-stage I/O errors (dmesg).
3. Retry the operation after freeing space; delete stale .tmp index-header files.
4. If it persists, verify the Thanos version for known binary-writer bugs and upgrade.

Example fix

// before
if err := bw.WriteTOC(); err != nil {
    return nil, errors.Wrap(err, "write index header TOC")
}
// after
if err := bw.WriteTOC(); err != nil {
    return nil, errors.Wrapf(err, "write index header TOC for block %s (disk full?)", id)
}

// Go: inspect the wrapped cause for space/device errors
_, err := indexheader.WriteBinary(ctx, bkt, id, dst)
if err != nil && strings.Contains(err.Error(), "write index header TOC") {
    switch {
    case errors.Is(err, syscall.ENOSPC):
        log.Error("disk full while writing TOC")
    case errors.Is(err, syscall.EIO):
        log.Error("I/O error while writing TOC; check device health")
    }
    return err
}

When it happens

Trigger: WriteBinary -> bw.WriteTOC() failing because the underlying bufio.Writer write inside WriteTOC fails (disk full, EIO, closed writer). Reached only after both earlier flushes in WriteBinary succeeded.

Common situations: Persistent volume filled up mid-write (earlier flushes consumed the last free bytes); storage device failure late in the write; test/benchmark environments with a tmpfs that ran out of space.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• add index meta
• read meta file
• open index file
• flush
• sync

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/47f8df803bdbeb52. Report an issue: GitHub.

Appendix: source

Thrown at pkg/block/indexheader/binary_reader.go:154

	if err := ir.CopySymbols(bw.SymbolsWriter(), buf); err != nil {
		return nil, err
	}

	if err := bw.writer.Flush(); err != nil {
		return nil, errors.Wrap(err, "flush")
	}

	if err := ir.CopyPostingsOffsets(bw.PostingOffsetsWriter(), buf); err != nil {
		return nil, err
	}

	if err := bw.writer.Flush(); err != nil {
		return nil, errors.Wrap(err, "flush")
	}

	if err := bw.WriteTOC(); err != nil {
		return nil, errors.Wrap(err, "write index header TOC")
	}

	if err := bw.writer.Flush(); err != nil {
		return nil, errors.Wrap(err, "flush")
	}

	if err := bw.writer.Sync(); err != nil {
		return nil, errors.Wrap(err, "sync")
	}

	if tmpFilename != "" {
		// Create index-header in atomic way, to avoid partial writes (e.g during restart or crash of store GW).
		return nil, os.Rename(tmpFilename, filename)
	}

	return bw.Buffer(), nil
}

View on GitHub (pinned to 35b8b99117)