What causes "create working compact directory" in thanos-io/thanos?

Ensure the data dir exists and is writable by the thanos process user (chown/chmod).

create working compact directory

Error message

create working compact directory

What it means

runCompact creates the local working directory <dataDir>/compact via os.MkdirAll and wraps any OS error with this message. MkdirAll fails if the path cannot be created: permission denied, parent is a file, or disk errors.

Solutions

Ensure the data dir exists and is writable by the thanos process user (chown/chmod).
Check that <dataDir>/compact is not a regular file and remove it if so.
Verify disk space and that the volume is mounted read-write.

Example fix

// before
mkdir: cannot create directory '/data/compact': Permission denied
// after
mkdir -p /data && chown thanos:thanos /data

Defensive patterns

Strategy: validation

Validate before calling

if err := os.MkdirAll(dataDir, 0o755); err != nil {
    return fmt.Errorf("cannot prepare data-dir %q: %w", dataDir, err)
}

Try / catch

if err := runCompact(...); err != nil {
    if strings.Contains(err.Error(), "create working compact directory") {
        logger.Error(err, "dataDir unusable; check permissions, mount and free space")
    }
    return err
}

Prevention

Create and chown the data dir before the process starts (init container/entrypoint)
Ensure the volume is mounted read-write with enough free space
Never point --data-dir at a path occupied by a regular file

When it happens

Trigger: os.MkdirAll(compactDir, os.ModePerm) returns an error because dataDir doesn't exist or isn't writable, a file already occupies the path, or the filesystem is read-only/full.

Common situations: Mounting an emptyDir/PV with wrong permissions; --data-dir pointing to a path owned by root while compact runs as non-root; disk full on the node.

Understand the failure class

Background: mkdir permission denied (EACCES): failed to create directory errors explained — this error's family across 32 libraries.

Related errors

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/c7a8282c153d1be1. Report an issue: GitHub.

Appendix: source

Thrown at cmd/thanos/compact.go:356

	default:
		return errors.Errorf("unsupported deduplication func, got %s", conf.dedupFunc)
	}

	// Instantiate the compactor with different time slices. Timestamps in TSDB
	// are in milliseconds.
	comp, err := tsdb.NewLeveledCompactor(ctx, reg, logutil.GoKitLogToSlog(logger), levels, downsample.NewPool(), mergeFunc)
	if err != nil {
		return errors.Wrap(err, "create compactor")
	}

	var (
		compactDir      = path.Join(conf.dataDir, "compact")
		downsamplingDir = path.Join(conf.dataDir, "downsample")
	)

	if err := os.MkdirAll(compactDir, os.ModePerm); err != nil {
		return errors.Wrap(err, "create working compact directory")
	}

	if err := os.MkdirAll(downsamplingDir, os.ModePerm); err != nil {
		return errors.Wrap(err, "create working downsample directory")
	}

	grouper := compact.NewDefaultGrouper(
		logger,
		insBkt,
		conf.acceptMalformedIndex,
		enableVerticalCompaction,
		reg,
		compactMetrics.blocksMarked.WithLabelValues(metadata.DeletionMarkFilename, ""),
		compactMetrics.garbageCollectedBlocks,
		compactMetrics.blocksMarked.WithLabelValues(metadata.NoCompactMarkFilename, metadata.OutOfOrderChunksNoCompactReason),
		metadata.HashFunc(conf.hashFunc),
		conf.blockFilesConcurrency,
		conf.compactBlocksFetchConcurrency,

View on GitHub (pinned to 35b8b99117)