thanos-io/thanos · error

read dir

Error message

read dir %v

What it means

GatherFileStats enumerates the chunks/ directory of a local TSDB block to build per-file metadata (size, hash). This error wraps os.ReadDir on <blockDir>/chunks failing — the directory does not exist or is unreadable. It surfaces when uploading a block whose on-disk structure is broken.

Solutions

1. Verify the block directory actually contains a chunks/ subdirectory (ls <blockDir>).
2. Check filesystem permissions on the block dir and that the process user can read it.
3. Confirm --data-dir points at a healthy Prometheus/TSDB data directory.
4. Exclude/remove the corrupted block from the upload queue (or repair it from a replica).

Example fix

// before
files, err := os.ReadDir(filepath.Join(blockDir, ChunksDirname))
if err != nil {
	return nil, errors.Wrapf(err, "read dir %v", filepath.Join(blockDir, ChunksDirname))
}
// after
if _, err := os.Stat(filepath.Join(blockDir, ChunksDirname)); os.IsNotExist(err) {
	return nil, errors.Errorf("block dir %s has no chunks directory; block is incomplete", blockDir)
}
files, err := os.ReadDir(filepath.Join(blockDir, ChunksDirname))

chunksDir := filepath.Join(blockDir, "chunks")
if fi, err := os.Stat(chunksDir); err != nil || !fi.IsDir() {
	return fmt.Errorf("block %s unusable: chunks dir missing", blockDir)
}
if _, err := os.Stat(filepath.Join(blockDir, "meta.json")); err != nil {
	return fmt.Errorf("block %s unusable: meta.json missing", blockDir)
}
if _, err := os.Stat(filepath.Join(blockDir, "index")); err != nil {
	return fmt.Errorf("block %s unusable: index missing", blockDir)
}

func isCompleteBlockDir(dir string) bool {
	for _, p := range []string{"meta.json", "index", "chunks"} {
		if _, err := os.Stat(filepath.Join(dir, p)); err != nil { return false }
	}
	return true
}

files, err := block.GatherFileStats(blockDir, hashFunc, logger)
if err != nil {
	if os.IsNotExist(errors.Unwrap(err)) {
		return fmt.Errorf("block %s incomplete, skipping upload", blockDir)
	}
	return err
}

When it happens

Trigger: upload() calls GatherFileStats on a block directory that lacks a chunks/ subdirectory, or where ReadDir fails due to permissions or an I/O error (e.g., disk failure, NFS issue).

Common situations: Prometheus data directory partially deleted or corrupted; wrong --data-dir path; block dir being concurrently deleted while being uploaded; volume mount issues in containers.

Related errors

• stat
• open block
• start remote write agent db
• open TSDB
• remove storage lock files

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/33775c4e39002165. Report an issue: GitHub.

Appendix: source

Thrown at pkg/block/block.go:330

func GetSegmentFiles(blockDir string) []string {
	files, err := os.ReadDir(filepath.Join(blockDir, ChunksDirname))
	if err != nil {
		return nil
	}

	// ReadDir returns files in sorted order already.
	var result []string
	for _, f := range files {
		result = append(result, f.Name())
	}
	return result
}

// GatherFileStats returns metadata.File entry for files inside TSDB block (index, chunks, meta.json).
func GatherFileStats(blockDir string, hf metadata.HashFunc, logger log.Logger) (res []metadata.File, _ error) {
	files, err := os.ReadDir(filepath.Join(blockDir, ChunksDirname))
	if err != nil {
		return nil, errors.Wrapf(err, "read dir %v", filepath.Join(blockDir, ChunksDirname))
	}
	for _, f := range files {
		fi, err := f.Info()
		if err != nil {
			return nil, errors.Wrapf(err, "getting file info %v", filepath.Join(ChunksDirname, f.Name()))
		}

		mf := metadata.File{
			RelPath:   filepath.Join(ChunksDirname, f.Name()),
			SizeBytes: fi.Size(),
		}
		if hf != metadata.NoneFunc && !f.IsDir() {
			h, err := metadata.CalculateHash(filepath.Join(blockDir, ChunksDirname, f.Name()), hf, logger)
			if err != nil {
				return nil, errors.Wrapf(err, "calculate hash %v", filepath.Join(ChunksDirname, f.Name()))
			}
			mf.Hash = &h
		}

View on GitHub (pinned to 35b8b99117)