hashicorp/nomad · error

Chmod(%v) failed: %w

Error message

Chmod(%v) failed: %w

What it means

dropDirPermissions attempts os.Chmod(path, desired|0777) to give all users full access to a shared allocation directory (typically the shared alloc dir exposed to chroot-isolated tasks). This error means the chmod syscall failed; the wrapped error carries the OS reason, usually EPERM because the process does not own the directory.

Source

Thrown at client/allocdir/fs_unix.go:38

var (
	// SharedAllocContainerPath is the path inside container for mounted
	// directory shared across tasks in a task group.
	SharedAllocContainerPath = filepath.Join("/", SharedAllocName)

	// TaskLocalContainerPath is the path inside a container for mounted directory
	// for local storage.
	TaskLocalContainerPath = filepath.Join("/", TaskLocal)

	// TaskSecretsContainerPath is the path inside a container for mounted
	// secrets directory
	TaskSecretsContainerPath = filepath.Join("/", TaskSecrets)
)

// dropDirPermissions gives full access to a directory to all users and sets
// the owner to nobody.
func dropDirPermissions(path string, desired os.FileMode) error {
	if err := os.Chmod(path, desired|fileMode777); err != nil {
		return fmt.Errorf("Chmod(%v) failed: %w", path, err)
	}

	// Can't change owner if not root.
	if unix.Geteuid() != 0 {
		return nil
	}

	u, err := users.Lookup("nobody")
	if err != nil {
		return fmt.Errorf("Unable to find nobody user: %w", err)
	}

	uid, err := getUid(u)
	if err != nil {
		return err
	}

	gid, err := getGid(u)

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Ensure the Nomad client process owns the alloc directory (chown -R the data_dir to the client user).
2. Clean stale alloc dirs from previous runs: nomad system gc or manually remove client data_dir/alloc.
3. Check that the alloc-dir mount is writable and supports chmod.
4. Inspect the wrapped error (lsattr, mount options) if permissions look correct.

Example fix

// before: client running as unprivileged user with root-owned alloc dir
ExecStart=/usr/bin/nomad agent -config /etc/nomad.d
// after: run client as a dedicated user that owns data_dir
chown -R nomad:nomad /var/lib/nomad
ExecStart=/usr/bin/nomad agent -config /etc/nomad.d
User=nomad

// preflight: client must own the alloc dir and be able to chmod
if st, err := os.Stat(allocDir); err != nil {
    return err
} else if st.Mode().Perm()&0o200 == 0 {
    return fmt.Errorf("alloc dir %s not writable by client user", allocDir)
}
// test:
probe := filepath.Join(allocDir, ".chmod-probe")
os.WriteFile(probe, nil, 0o600)
err := os.Chmod(probe, 0o700); os.Remove(probe)

if err := buildTaskDir(...); err != nil {
    var eperm syscall.Errno
    if errors.As(err, &eperm) && eperm == syscall.EPERM {
        log.Printf("chmod denied on %s; check ownership", allocDir)
    }
    return err
}

When it happens

Trigger: os.Chmod(path, desired|fileMode777) returned an error while relaxing permissions on a built alloc/task directory, e.g. during task directory Build for chroot filesystem isolation.

Common situations: Directory owned by another user (stale alloc dirs from a previous run under a different client user); read-only mounts; alloc-dir on a filesystem not supporting chmod (some FAT/overlay setups); immutable attribute set.

Related errors

• error chmoding file %w
• plugin not executable
• Couldn't change owner/group of %v to (uid: %v, gid: %v): %w
• unable to remove existing unix socket: %w
• failed to write vault token to secrets dir: %v

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/f0e6a44ae3fa2189. Report an issue: GitHub.