thanos-io/thanos · error

add index meta

Error message

add index meta

What it means

WriteBinary serializes an index-header file for a TSDB block. The first serialization step is bw.AddIndexMeta(indexVersion, ir.toc.PostingsTable), which writes the index version and postings-table offset into the buffered writer. This error wraps any failure returned by that call, most commonly an underlying write/flush error from the bufio.Writer backing the binary writer (e.g. disk full, I/O error) or an unsupported index version.

Solutions

1. Check free disk space on the volume holding the block directory (df -h) and clean up if full.
2. Verify write permissions on the directory containing the index-header temp file (<filename>.tmp).
3. Check dmesg / filesystem health for underlying I/O errors on the disk.
4. Re-download or re-upload the block's index file if the source index is corrupted (invalid version/TOC).
5. Retry WriteBinary; transient I/O failures on network-backed storage can clear on retry.

Example fix

// before: no preflight space/permission check, error surfaces only here
bw, err := newBinaryWriter(id, tmpFilename, buf)
...
// after: fail early with a clear message
if err := checkWritable(tmpFilename); err != nil {
    return nil, errors.Wrapf(err, "index-header dir %s not writable", filepath.Dir(tmpFilename))
}

// Go: preflight writable temp dir with enough space before WriteBinary
func canWriteIndexHeader(dir string) error {
    info, err := os.Stat(dir)
    if err != nil { return err }
    if !info.IsDir() { return fmt.Errorf("%s is not a directory", dir) }
    probe := filepath.Join(dir, ".write-probe")
    if err := os.WriteFile(probe, []byte("x"), 0o644); err != nil { return err }
    return os.Remove(probe)
}

// Go
ir, err := indexheader.WriteBinary(ctx, bkt, id, dst)
if err != nil {
    if strings.Contains(err.Error(), "add index meta") {
        // disk/permission problem writing the index-header temp file
        log.Error("index-header build failed at AddIndexMeta; check disk space/permissions", "err", err)
    }
    return err
}

When it happens

Trigger: Calling WriteBinary (directly or via NewBinaryReader / NewLazyBinaryReader) on a block whose on-disk or in-memory index write fails during the very first write to the bufio.Writer: AddIndexMeta writes the magic/version/postings-offset and the buffered writer's first flush to the underlying file or buffer fails (disk full, EIO, permission issue on the temp file).

Common situations: Store Gateway or Compactor rebuilding an index-header on a disk that is full or failing; writing into a directory without write permission (the .tmp temp file); a corrupted or unsupported index version being passed through.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• flush
• write index header TOC
• unable to access path
• unable to open tbc directory
• read dir

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/2de1b1018b055e8e. Report an issue: GitHub.

Appendix: source

Thrown at pkg/block/indexheader/binary_reader.go:134

	if err != nil {
		return nil, errors.Wrap(err, "new index reader")
	}
	tmpFilename := ""
	if filename != "" {
		tmpFilename = filename + ".tmp"
	}

	// Buffer for copying and encbuffers.
	// This also will control the size of file writer buffer.
	buf := make([]byte, 32*1024)
	bw, err := newBinaryWriter(id, tmpFilename, buf)
	if err != nil {
		return nil, errors.Wrap(err, "new binary index header writer")
	}
	defer runutil.CloseWithErrCapture(&err, bw, "close binary writer for %s", tmpFilename)

	if err := bw.AddIndexMeta(indexVersion, ir.toc.PostingsTable); err != nil {
		return nil, errors.Wrap(err, "add index meta")
	}

	if err := ir.CopySymbols(bw.SymbolsWriter(), buf); err != nil {
		return nil, err
	}

	if err := bw.writer.Flush(); err != nil {
		return nil, errors.Wrap(err, "flush")
	}

	if err := ir.CopyPostingsOffsets(bw.PostingOffsetsWriter(), buf); err != nil {
		return nil, err
	}

	if err := bw.writer.Flush(); err != nil {
		return nil, errors.Wrap(err, "flush")
	}

View on GitHub (pinned to 35b8b99117)