hashicorp/nomad · error

Failed to chown task mount directory: %v

Error message

Failed to chown task mount directory: %v

What it means

After creating the per-task mounts directory in Unveil mode, TaskDir.Build runs os.Chown(parent, uid, gid) so the task's runtime user owns it. This error wraps a chown failure. Nomad needs this ownership so the isolated task can traverse its mount points.

Source

Thrown at client/allocdir/task_dir.go:195

	// Only bind mount the task alloc/task dirs to the client.mounts_dir/<task>
	if fsi == fsisolation.Unveil {
		uid, gid, _, err := dynamic.LookupUser(username)
		if err != nil {
			return fmt.Errorf("Failed to lookup user: %v", err)
		}

		nobodyUID, nobodyGID, _, err := dynamic.LookupUser("nobody")
		if err != nil {
			return fmt.Errorf("Failed to lookup nobody user: %v", err)
		}

		// create the task unique directory under the client mounts path
		parent := filepath.Dir(t.MountsAllocDir)
		if err = os.MkdirAll(parent, fileMode710); err != nil {
			return fmt.Errorf("Failed to create task mount directory: %v", err)
		}
		if err = os.Chown(parent, uid, gid); err != nil {
			return fmt.Errorf("Failed to chown task mount directory: %v", err)
		}

		// create the taskdir mount point
		if err = mountDir(t.Dir, t.MountsTaskDir, uid, gid, fileMode710); err != nil {
			return fmt.Errorf("Failed to mount task dir: %v", err)
		}

		// create the allocdir mount point (owned by nobody)
		if err = mountDir(filepath.Join(t.AllocDir, "/alloc"), t.MountsAllocDir, nobodyUID, nobodyGID, fileMode777); err != nil {
			return fmt.Errorf("Failed to mount alloc dir: %v", err)
		}

		// create the secretsdir mount point
		if err = mountDir(t.SecretsDir, t.MountsSecretsDir, uid, gid, fileMode710); err != nil {
			return fmt.Errorf("Failed to mount secrets dir: %v", err)
		}
	}

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Run the Nomad client as root (or with CAP_CHOWN / CAP_DAC_OVERRIDE capability set)
2. Check the wrapped errno: EPERM on NFS → move data_dir off root-squashed NFS to local storage
3. If non-root operation is required, switch the task driver/fs-isolation mode that doesn't require chown (chroot as the same user)
4. Confirm the resolved task username exists and the intended uid/gid are correct in the agent logs

Example fix

// before: systemd unit runs nomad as non-root
User=nomad
# after
User=root
# or drop-in override with capability
[Service]
AmbientCapabilities=CAP_CHOWN CAP_DAC_OVERRIDE

// ensure the process can chown: must be root or have CAP_CHOWN
if os.Geteuid() != 0 {
    log.Fatal("Unveil fs isolation requires running the Nomad client as root (CAP_CHOWN)")
}

if err := taskDir.Build(fsi, chroot, username); err != nil {
    var perr *os.PathError
    if errors.As(err, &perr) && errors.Is(perr.Err, syscall.EPERM) {
        return fmt.Errorf("chown denied (EPERM): run client as root or grant CAP_CHOWN; avoid root-squashed NFS for data_dir: %w", err)
    }
    return err
}

When it happens

Trigger: os.Chown fails during TaskDir.Build(Unveil) — typically because the Nomad client process is not root (or lacks CAP_CHOWN) while the target uid/gid differ from its own, or the parent directory just created has restrictive ownership.

Common situations: Running nomad agent as a non-root user with Unveil/task isolation; containerized Nomad client dropped privileges (no CAP_CHOWN); root-squashed NFS-mounted data_dir where chown to another uid is denied (EPERM).

Related errors

• Couldn't change owner/group of %v to (uid: %v, gid: %v): %w
• Failed to create task mount directory: %v
• failed to recover vault token from %s: %v
• error chowning directory %w
• error chowning file %w

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/d143f9e4c6ac2457. Report an issue: GitHub.