hashicorp/nomad · error

failed to create WAL directory: %w

Error message

failed to create WAL directory: %w

What it means

MigrateToWAL creates the destination WAL directory at <raftDir>/wal via os.MkdirAll after opening the source BoltDB store. This error wraps the OS error when that directory cannot be created. Migration aborts, the BoltDB store is closed, and the original raft.db is left untouched, so the operation is safe to retry.

Source

Thrown at helper/raftutil/migrate.go:91

	}
	defer os.Remove(markerPath) // Clean up marker on completion or failure.

	// Open the source BoltDB store.
	src, err := raftboltdb.New(raftboltdb.Options{
		Path: boltPath,
		BoltOptions: &bbolt.Options{
			Timeout: 5 * time.Second,
		},
		MsgpackUseNewTimeFormat: true,
	})
	if err != nil {
		return fmt.Errorf("failed to open BoltDB store: %w", err)
	}

	// Create the destination WAL store.
	if err := os.MkdirAll(walDir, 0o700); err != nil {
		src.Close()
		return fmt.Errorf("failed to create WAL directory: %w", err)
	}

	dst, err := raftwal.Open(walDir)
	if err != nil {
		src.Close()
		cleanupWAL(walDir)
		return fmt.Errorf("failed to open WAL store: %w", err)
	}

	// Copy logs.
	logProgress := make(chan string, 64)
	wg.Add(1)
	go drainProgress(logProgress, progress, &wg)
	if err := migrate.CopyLogs(ctx, dst, src, migrateBatchBytes, logProgress); err != nil {
		dst.Close()
		src.Close()
		cleanupWAL(walDir)
		return fmt.Errorf("failed to copy logs: %w", err)

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Check that the process user has write permission on the raft directory (ls -ld <raftDir>; chown/chmod as needed).
2. Verify no non-directory file named 'wal' exists at <raftDir>/wal and remove/rename it.
3. Confirm the raft volume is mounted read-write (mount | grep <raftDir>; remount or fix container volume config).
4. Fix the wrapped OS error indicated in the %w suffix (e.g. ENOSPC, ENAMETOOLONG) and re-run MigrateToWAL.

Example fix

// before (migrating inside a container with read-only data volume)
volumes:
  - /opt/nomad/data:ro
// after
volumes:
  - /opt/nomad/data:/opt/nomad/data

walDir := filepath.Join(raftDir, "wal")
if fi, err := os.Stat(walDir); err == nil && !fi.IsDir() {
    return fmt.Errorf("%s exists and is not a directory; remove it first", walDir)
}
if err := unix.Access(raftDir, unix.W_OK); err != nil {
    return fmt.Errorf("no write access to %s: %w", raftDir, err)
}

err := raftutil.MigrateToWAL(ctx, raftDir, progress)
var pathErr *os.PathError
if errors.As(err, &pathErr) && errors.Is(pathErr.Err, syscall.EACCES) {
    // fix permissions or run as another user
}

When it happens

Trigger: os.MkdirAll(walDir, 0o700) returns an error: the parent raft directory is read-only or not writable by the process user, a file (not directory) named 'wal' exists at the target path despite preflight, the path is too long, or an I/O error occurs on the underlying filesystem.

Common situations: Running the Nomad server or migration under a service account that lacks write access to the raft data dir; raft dir on a read-only mounted volume (e.g. container image or read-only PVC); a stale 'wal' file left by a previous failed run; SELinux/AppArmor denials.

Understand the failure class

Background: mkdir permission denied (EACCES): failed to create directory errors explained — this error's family across 32 libraries.

Related errors

• failed to create migration marker: %w
• failed to open WAL store: %w
• plugin not executable
• failed to snapshot %s: %w
• error creating task %q dir: %w

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/40aef83ebfb68a34. Report an issue: GitHub.