hashicorp/nomad · error

failed to create allocation directory for volume

Error message

failed to create allocation directory for volume (%s): %v

What it means

ensureAllocDir in Nomad's CSI volume manager creates the per-allocation directory (mode 0700) that a published volume is mounted into. If os.MkdirAll fails with any error other than 'already exists', publishVolume fails with this message wrapping the OS error.

Solutions

1. Inspect the wrapped OS error and the alloc path on the host; remove/fix any non-directory file occupying the path
2. Correct ownership/permissions on the Nomad client data dir for the Nomad user
3. Free disk space or remount read-write if ENOSPC/EROFS is the cause
4. Restart the Nomad client after fixing the filesystem so the alloc dir is recreated

Example fix

// before (host)
ls -l /var/lib/nomad/client/<alloc_id>  # regular file
// after (host)
rm /var/lib/nomad/client/<alloc_id>
# restart nomad client; alloc dir recreated with mode 0700

allocPath := filepath.Join(mountRoot, allocID)
if fi, err := os.Stat(allocPath); err == nil && !fi.IsDir() {
    return fmt.Errorf("alloc path %s is not a directory", allocPath)
}
if err := unix.Access(mountRoot, unix.W_OK); err != nil {
    return fmt.Errorf("mount root %s not writable: %v", mountRoot, err)
}

path, published, err := vm.EnsureAllocDir(vol, alloc, usage)
if err != nil {
    return fmt.Errorf("alloc dir setup failed: %w", err) // fix OS-level cause before retry
}

When it happens

Trigger: os.MkdirAll(allocPath, 0700) returns a non-IsExist error during publishVolume — e.g. the alloc dir path component is a file, the mount root is unwritable, disk full, or read-only filesystem.

Common situations: Nomad client data dir permissions broken after host migration/restore; alloc directory corrupted (file where directory expected); ENOSPC on the node; security policies (SELinux/AppArmor) blocking writes under the client data dir.

Understand the failure class

Background: mkdir permission denied (EACCES): failed to create directory errors explained — this error's family across 32 libraries.

Related errors

• failed to create staging directory for volume
• CSIPluginConfig StagePublishBaseDir must not be a…
• mount point detection failed for volume
• alloc lookup failed
• alloc failed to claim volume

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/67247d847b4459f2. Report an issue: GitHub.

Appendix: source

Thrown at client/pluginmanager/csimanager/volume.go:466

	if err != nil {
		return "", false, fmt.Errorf("mount point detection failed for volume (%s): %v", vol.ID, err)
	}

	return hostStagingPath, !isNotMount, nil
}

// ensureAllocDir attempts to create a directory for use when publishing a volume
// and then validates that the path is not already a mount point (e.g when reattaching
// to existing allocs).
//
// Returns whether the directory is a pre-existing mountpoint, the publish path,
// and any errors that occurred.
func (v *volumeManager) ensureAllocDir(vol *structs.CSIVolume, alloc *structs.Allocation, usage *UsageOptions) (string, bool, error) {
	allocPath := v.allocDirForVolume(v.mountRoot, vol.ID, alloc.ID)

	// Make the alloc path, owned by the Nomad User
	if err := os.MkdirAll(allocPath, 0700); err != nil && !os.IsExist(err) {
		return "", false, fmt.Errorf("failed to create allocation directory for volume (%s): %v", vol.ID, err)
	}

	// Validate that the target is not already a mount point
	targetPath := v.targetForVolume(v.mountRoot, vol.ID, alloc.ID, usage)

	m := mount.New()
	isNotMount, err := m.IsNotAMountPoint(targetPath)

	switch {
	case errors.Is(err, os.ErrNotExist):
		// ignore; path does not exist and as such is not a mount
	case err != nil:
		return "", false, fmt.Errorf("mount point detection failed for volume (%s): %v", vol.ID, err)
	}

	return targetPath, !isNotMount, nil
}

View on GitHub (pinned to 482b49bf1a)