thanos-io/thanos · error

flush

Error message

flush

What it means

After copying symbols into the binary writer, WriteBinary flushes the bufio.Writer (bw.writer.Flush()) to push all buffered bytes into the underlying output (file or in-memory buffer). This error wraps a flush failure right after CopySymbols — i.e. writing the accumulated index-meta plus symbol data to the backing file failed.

Solutions

1. Check disk space and inode availability on the target filesystem (df -h, df -i).
2. Confirm the volume is writable and not remounted read-only (mount | grep <path>).
3. Check filesystem/device errors in dmesg and repair or replace failing storage.
4. Retry the operation; for block bucket-backed flows, deleting a partial <index-header>.tmp and re-running WriteBinary is safe.

Example fix

// before
if err := bw.writer.Flush(); err != nil {
    return nil, errors.Wrap(err, "flush")
}
// after: surface quota/space cause explicitly
if err := bw.writer.Flush(); err != nil {
    if st, statErr := os.Stat(filepath.Dir(tmpFilename)); statErr == nil {
        _ = st
    }
    return nil, errors.Wrapf(err, "flush index-header writer (check disk space on %s)", filepath.Dir(tmpFilename))
}

// Go: verify disk has headroom before the flush-heavy build
func hasDiskHeadroom(dir string, minBytes uint64) error {
    var st syscall.Statfs_t
    if err := syscall.Statfs(dir, &st); err != nil { return err }
    if uint64(st.Bavail)*uint64(st.Bsize) < minBytes {
        return fmt.Errorf("less than %d bytes free in %s", minBytes, dir)
    }
    return nil
}

// Go
if err := retry.Do(func() error {
    _, err := indexheader.WriteBinary(ctx, bkt, id, dst)
    return err
}, retry.OnRetry(func(n uint, err error) { log.Warn("retrying index-header write after flush failure", "attempt", n, "err", err) }), retry.Attempts(3)); err != nil {
    return errors.Wrap(err, "index-header write failed after retries")
}

When it happens

Trigger: WriteBinary -> bw.writer.Flush() returning a non-nil error after symbol copy: the underlying os.File write failed (disk full, EIO), or the file was closed/invalid. Triggered by any caller of WriteBinary (NewBinaryReader, NewLazyBinaryReader, benchmarks, tests).

Common situations: Disk-quota exceeded on store-gateway/compactor data volume; filesystem remounted read-only; NFS/ObjectStorage-mounted volume with intermittent I/O errors while materializing the index-header.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• add index meta
• flush
• unable to access path
• unable to open tbc directory
• read dir

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/69c8e318865218b0. Report an issue: GitHub.

Appendix: source

Thrown at pkg/block/indexheader/binary_reader.go:142

	// Buffer for copying and encbuffers.
	// This also will control the size of file writer buffer.
	buf := make([]byte, 32*1024)
	bw, err := newBinaryWriter(id, tmpFilename, buf)
	if err != nil {
		return nil, errors.Wrap(err, "new binary index header writer")
	}
	defer runutil.CloseWithErrCapture(&err, bw, "close binary writer for %s", tmpFilename)

	if err := bw.AddIndexMeta(indexVersion, ir.toc.PostingsTable); err != nil {
		return nil, errors.Wrap(err, "add index meta")
	}

	if err := ir.CopySymbols(bw.SymbolsWriter(), buf); err != nil {
		return nil, err
	}

	if err := bw.writer.Flush(); err != nil {
		return nil, errors.Wrap(err, "flush")
	}

	if err := ir.CopyPostingsOffsets(bw.PostingOffsetsWriter(), buf); err != nil {
		return nil, err
	}

	if err := bw.writer.Flush(); err != nil {
		return nil, errors.Wrap(err, "flush")
	}

	if err := bw.WriteTOC(); err != nil {
		return nil, errors.Wrap(err, "write index header TOC")
	}

	if err := bw.writer.Flush(); err != nil {
		return nil, errors.Wrap(err, "flush")
	}

View on GitHub (pinned to 35b8b99117)