hashicorp/nomad · critical

failed creating alloc mounts dir: %w

Error message

failed creating alloc mounts dir: %w

What it means

If conf.AllocMountsDir is set to a custom path, the client runs os.MkdirAll(path, 0o711) during init to guarantee it exists. Failure (any MkdirAll error) aborts client startup with this message. The alloc mounts dir is where allocation mount data is staged, so it is mandatory.

Source

Thrown at client/client.go:736

		// node is accessible. Upgrade drops and logs corrupt state it
		// encounters, so failing to start the agent should be extremely
		// rare.
		return fmt.Errorf("failed to upgrade state database: %v", err)
	}

	c.stateDB = db

	// Ensure host_volumes_dir config is not empty.
	if conf.HostVolumesDir == "" {
		conf = c.UpdateConfig(func(c *config.Config) {
			c.HostVolumesDir = filepath.Join(conf.StateDir, "host_volumes")
		})
	}

	// Ensure the alloc mounts dir exists if we are configured with a custom path.
	if conf.AllocMountsDir != "" {
		if err := os.MkdirAll(conf.AllocMountsDir, 0o711); err != nil {
			return fmt.Errorf("failed creating alloc mounts dir: %w", err)
		}
	}

	// Ensure the alloc dir exists if we are configured with a custom path.
	if conf.AllocDir != "" {
		if err := os.MkdirAll(conf.AllocDir, 0o711); err != nil {
			return fmt.Errorf("failed creating alloc dir: %w", err)
		}
	} else {
		// Otherwise make a temp directory to use.
		p, err := os.MkdirTemp("", "NomadClient")
		if err != nil {
			return fmt.Errorf("failed creating temporary directory for the AllocDir: %v", err)
		}

		p, err = filepath.EvalSymlinks(p)
		if err != nil {
			return fmt.Errorf("failed to find temporary directory for the AllocDir: %v", err)

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Pre-create the directory manually with correct ownership: sudo mkdir -p <path> && sudo chown nomad:nomad <path> && chmod 711 <path>.
2. Check the wrapped %w error for ENOSPC/EROFS/EACCES and fix disk space, mount flags, or permissions accordingly.
3. Verify the configured alloc_mounts_dir path is a directory, not a file, and lives on a local writable filesystem.

Example fix

// before (config.hcl)
client { alloc_mounts_dir = "/mnt/ro-disk/nomad/mounts" }  // read-only mount
// after
client { alloc_mounts_dir = "/var/lib/nomad/alloc_mounts" }
# or fix permissions first:
# sudo mkdir -p /mnt/disk/nomad/mounts && sudo chown nomad:nomad /mnt/disk/nomad/mounts

func ensureDirWritable(p string) error {
    fi, err := os.Stat(p)
    if err == nil && !fi.IsDir() {
        return fmt.Errorf("%s exists and is not a directory", p)
    }
    return os.MkdirAll(p, 0o711)
}
// call before constructing client config
if err := ensureDirWritable(cfg.AllocMountsDir); err != nil { return err }

if err := clientInit(); err != nil {
    var pe *fs.PathError
    if errors.As(err, &pe) && strings.Contains(err.Error(), "failed creating alloc mounts dir") {
        log.Printf("fix path %s: %v", pe.Path, pe.Err)
    }
}

When it happens

Trigger: client init with alloc_mounts_dir (or derived alloc dir) configured: os.MkdirAll(conf.AllocMountsDir, 0o711) returns error.

Common situations: Custom path on a read-only mount or full disk, parent directory owned by another user with no write permission, SELinux/AppArmor denial, or the path existing as a regular file instead of a directory.

Understand the failure class

Background: mkdir permission denied (EACCES): failed to create directory errors explained — this error's family across 32 libraries.

Related errors

• failed creating alloc dir: %w
• failed to change directory permissions for the AllocDir: %v
• failed to create parent directories of %q: %w
• plugin not executable
• failed to snapshot %s: %w

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/ead62dd10a24f6bf. Report an issue: GitHub.