hashicorp/nomad · error

error moving data dir

Error message

error moving data dir: %w

What it means

Move renames the previous allocation's SharedDataDir into the new allocation dir with os.Rename. If the rename fails (cross-device filesystems, permissions, source vanished), the error is wrapped as 'error moving data dir'. Rename requires both paths on the same mounted filesystem.

Solutions

1. Ensure old and new alloc dirs are on the same filesystem/device
2. Check directory permissions on both alloc dirs
3. Verify the source data dir still exists (not GC'd) at migration time
4. Copy-then-delete as a fallback when cross-device moves are unavoidable

Example fix

// before
if err := os.Rename(otherDataDir, dataDir); err != nil {
  return fmt.Errorf("error moving data dir: %w", err)
}
// after
if err := os.Rename(otherDataDir, dataDir); err != nil {
  if err2 := copyDir(otherDataDir, dataDir); err2 != nil {
    return fmt.Errorf("error moving data dir: %w", err)
  }
  os.RemoveAll(otherDataDir)
}

od, err := os.Stat(otherDataDir)
if err == nil && od != nil {
  sameDev := syscall.Stat_t{ }
  _ = syscall.Stat(otherDataDir, &sameDev) // compare Device with target's before rename
}

if err := allocDir.Move(other, tasks); err != nil && strings.Contains(err.Error(), "error moving data dir") {
  // fall back to copy+delete or surface EXDEV/mount misconfiguration
}

When it happens

Trigger: os.Rename of otherDataDir fails — source alloc dir on a different device/mount than the target, permission denial, or the source data dir removed concurrently by GC.

Common situations: Alloc dirs spanned across volumes/bind-mounts (host volume layouts); node filesystem remounts read-only; GC racing with migration; permission mismatches after node re-provisioning.

Related errors

• error moving task local dir
• cannot install new nomad binary
• error building alloc dir for previous alloc
• error creating task dir
• migration succeeded but failed to rename

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/4bdfa826cdaf0239. Report an issue: GitHub.

Appendix: source

Thrown at client/allocdir/alloc_dir.go:294

// Move other alloc directory's shared path and local dir to this alloc dir.
func (a *AllocDir) Move(other Interface, tasks []*structs.Task) error {
	a.mu.RLock()
	if !a.built {
		// Enforce the invariant that Build is called before Move
		a.mu.RUnlock()
		return fmt.Errorf("unable to move to %q - alloc dir is not built", a.AllocDir)
	}

	// Moving is slow and only reads immutable fields, so unlock during heavy IO
	a.mu.RUnlock()

	// Move the data directory
	otherDataDir := filepath.Join(other.ShareDirPath(), SharedDataDir)
	dataDir := filepath.Join(a.SharedDir, SharedDataDir)
	if fileInfo, err := os.Stat(otherDataDir); fileInfo != nil && err == nil {
		os.Remove(dataDir) // remove an empty data dir if it exists
		if err := os.Rename(otherDataDir, dataDir); err != nil {
			return fmt.Errorf("error moving data dir: %w", err)
		}
	}

	// Move the task directories
	for _, task := range tasks {
		otherTaskDir := filepath.Join(other.AllocDirPath(), task.Name)
		otherTaskLocal := filepath.Join(otherTaskDir, TaskLocal)

		fileInfo, err := os.Stat(otherTaskLocal)
		if fileInfo != nil && err == nil {
			// TaskDirs haven't been built yet, so create it
			newTaskDir := filepath.Join(a.AllocDir, task.Name)
			if err := os.MkdirAll(newTaskDir, fileMode777); err != nil {
				return fmt.Errorf("error creating task %q dir: %w", task.Name, err)
			}
			localDir := filepath.Join(newTaskDir, TaskLocal)
			os.Remove(localDir) // remove an empty local dir if it exists
			if err := os.Rename(otherTaskLocal, localDir); err != nil {

View on GitHub (pinned to 482b49bf1a)