hashicorp/nomad · error

failed to create allocation directory for volume (%s): %v

Error message

failed to create allocation directory for volume (%s): %v

What it means

ensureAllocDir in Nomad's CSI volume manager creates the per-allocation directory (mode 0700) that a published volume is mounted into. If os.MkdirAll fails with any error other than 'already exists', publishVolume fails with this message wrapping the OS error.

Source

Thrown at client/pluginmanager/csimanager/volume.go:466

	if err != nil {
		return "", false, fmt.Errorf("mount point detection failed for volume (%s): %v", vol.ID, err)
	}

	return hostStagingPath, !isNotMount, nil
}

// ensureAllocDir attempts to create a directory for use when publishing a volume
// and then validates that the path is not already a mount point (e.g when reattaching
// to existing allocs).
//
// Returns whether the directory is a pre-existing mountpoint, the publish path,
// and any errors that occurred.
func (v *volumeManager) ensureAllocDir(vol *structs.CSIVolume, alloc *structs.Allocation, usage *UsageOptions) (string, bool, error) {
	allocPath := v.allocDirForVolume(v.mountRoot, vol.ID, alloc.ID)

	// Make the alloc path, owned by the Nomad User
	if err := os.MkdirAll(allocPath, 0700); err != nil && !os.IsExist(err) {
		return "", false, fmt.Errorf("failed to create allocation directory for volume (%s): %v", vol.ID, err)
	}

	// Validate that the target is not already a mount point
	targetPath := v.targetForVolume(v.mountRoot, vol.ID, alloc.ID, usage)

	m := mount.New()
	isNotMount, err := m.IsNotAMountPoint(targetPath)

	switch {
	case errors.Is(err, os.ErrNotExist):
		// ignore; path does not exist and as such is not a mount
	case err != nil:
		return "", false, fmt.Errorf("mount point detection failed for volume (%s): %v", vol.ID, err)
	}

	return targetPath, !isNotMount, nil
}

View on GitHub (pinned to 482b49bf1a)

Solutions

  1. Inspect the wrapped OS error and the alloc path on the host; remove/fix any non-directory file occupying the path
  2. Correct ownership/permissions on the Nomad client data dir for the Nomad user
  3. Free disk space or remount read-write if ENOSPC/EROFS is the cause
  4. Restart the Nomad client after fixing the filesystem so the alloc dir is recreated

Example fix

// before (host)
ls -l /var/lib/nomad/client/<alloc_id>  # regular file
// after (host)
rm /var/lib/nomad/client/<alloc_id>
# restart nomad client; alloc dir recreated with mode 0700
Defensive patterns

Strategy: validation

Validate before calling

allocPath := filepath.Join(mountRoot, allocID)
if fi, err := os.Stat(allocPath); err == nil && !fi.IsDir() {
    return fmt.Errorf("alloc path %s is not a directory", allocPath)
}
if err := unix.Access(mountRoot, unix.W_OK); err != nil {
    return fmt.Errorf("mount root %s not writable: %v", mountRoot, err)
}

Try / catch

path, published, err := vm.EnsureAllocDir(vol, alloc, usage)
if err != nil {
    return fmt.Errorf("alloc dir setup failed: %w", err) // fix OS-level cause before retry
}

Prevention

When it happens

Trigger: os.MkdirAll(allocPath, 0700) returns a non-IsExist error during publishVolume — e.g. the alloc dir path component is a file, the mount root is unwritable, disk full, or read-only filesystem.

Common situations: Nomad client data dir permissions broken after host migration/restore; alloc directory corrupted (file where directory expected); ENOSPC on the node; security policies (SELinux/AppArmor) blocking writes under the client data dir.

Understand the failure class

Background: mkdir permission denied (EACCES): failed to create directory errors explained — this error's family across 32 libraries.

Related errors

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/67247d847b4459f2. Report an issue: GitHub.