thanos-io/thanos · error

open temporary block dir

Error message

open temporary block dir

What it means

Flush fails when fileutil.OpenDir cannot open the temporary block directory for reading. Flush finalizes the block by syncing, closing writers, and renaming the tmp dir; if the tmp dir cannot even be opened, finalization cannot proceed and the block is not published.

Solutions

1. Ensure no other process concurrently removes the <block>/tmp-for-creation directory.
2. Confirm Flush is called on the same DiskWriter instance/dir used for writes.
3. Check filesystem permissions and open-file limits (ulimit -n).
4. If the block write failed earlier, discard and re-create the block instead of flushing.

Example fix

// before
// janitor goroutine: os.RemoveAll(tmpDir) while Flush is running
// after
// guard: only remove tmpDir if writer flush failed and is not in progress

tmpDir := filepath.Join(bdir, "tmp-for-creation")
if fi, err := os.Stat(tmpDir); err != nil || !fi.IsDir() {
    return fmt.Errorf("tmp dir %s missing before flush", tmpDir)
}

if _, err := d.Flush(ctx); err != nil {
    if strings.Contains(err.Error(), "open temporary block dir") {
        // transient removal race: rebuild block
        return fmt.Errorf("tmp dir vanished during flush; re-run compaction: %w", err)
    }
    return err
}

When it happens

Trigger: fileutil.OpenDir(d.bTmp) failing because the temp block dir was deleted (e.g. by concurrent cleanup), was never created, the path changed, or an OS-level open error occurred (permissions, fd exhaustion).

Common situations: Two compactions/uploads racing on the same block dir and one removing the tmp folder; tmp dir cleaned by a janitor mid-flush; too many open files; wrong bdir passed so bTmp never existed.

Related errors

• open chunk writer
• open index writer
• sync temporary dir file
• close temporary dir
• replace index file

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/4be40bfe4de59538. Report an issue: GitHub.

Appendix: source

Thrown at pkg/block/writer.go:114

		return nil, errors.Wrap(err, "open index writer")
	}
	d.closers = append(d.closers, indexw)
	d.statsGatheringSeriesWriter = statsGatheringSeriesWriter{iw: indexw, cw: chunkw}
	return d, nil
}

func (d *DiskWriter) Flush() (_ tsdb.BlockStats, err error) {
	defer func() {
		if err != nil {
			err = tsdb_errors.NewMulti(err, tsdb_errors.CloseAll(d.closers)).Err()
			if err := os.RemoveAll(d.bTmp); err != nil {
				level.Error(d.logger).Log("msg", "removed tmp folder failed after block(s) write", "err", err.Error())
			}
		}
	}()
	df, err := fileutil.OpenDir(d.bTmp)
	if err != nil {
		return tsdb.BlockStats{}, errors.Wrap(err, "open temporary block dir")
	}
	defer func() {
		if df != nil {
			err = tsdb_errors.NewMulti(err, df.Close()).Err()
		}
	}()

	if err := df.Sync(); err != nil {
		return tsdb.BlockStats{}, errors.Wrap(err, "sync temporary dir file")
	}

	// Close temp dir before rename block dir (for windows platform).
	if err = df.Close(); err != nil {
		return tsdb.BlockStats{}, errors.Wrap(err, "close temporary dir")
	}
	df = nil

	if err := tsdb_errors.CloseAll(d.closers); err != nil {

View on GitHub (pinned to 35b8b99117)