hashicorp/nomad · error

unable to set permissions on unix socket: %w

Error message

unable to set permissions on unix socket: %w

What it means

After creating the Consul HTTP unix socket, the hook chmods it to os.ModePerm (0777) so non-root task users can connect to the Consul HTTP API through the alloc's network namespace. If os.Chmod fails (usually a permissions or filesystem issue on the socket path), the socket would be unusable by unprivileged tasks, so the hook aborts with this wrapped error.

Source

Thrown at client/allocrunner/consul_http_sock_hook.go:236

		socketFile = filepath.Join(allocdir.SharedAllocName, allocdir.TmpDirName,
			"consul_"+p.config.Name+"_http.sock")
	}
	hostHTTPSockPath := filepath.Join(p.allocDir.AllocDirPath(), socketFile)
	if err := maybeRemoveOldSocket(hostHTTPSockPath); err != nil {
		return err
	}

	listener, err := net.Listen("unix", hostHTTPSockPath)
	if err != nil {
		return fmt.Errorf("unable to create unix socket for Consul HTTP endpoint: %w", err)
	}

	// The Consul HTTP socket should be usable by all users in case a task is
	// running as a non-privileged user. Unix does not allow setting domain
	// socket permissions when creating the file, so we must manually call
	// chmod afterwards.
	if err := os.Chmod(hostHTTPSockPath, os.ModePerm); err != nil {
		return fmt.Errorf("unable to set permissions on unix socket: %w", err)
	}

	go func() {
		proxy(p.ctx, p.logger, destAddr, listener)
		p.cancel()
		close(p.doneCh)
	}()

	p.runOnce = true
	return nil
}

func (p *httpSocketProxy) stop() error {
	p.cancel()

	// if proxy was never run, no need to wait before shutdown
	if !p.runOnce {
		return nil

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Run the Nomad client as root (or a user with ownership of the host_http_socket path) so chmod succeeds
2. Check the wrapped OS error: EPERM → fix ownership of hostHTTPSockPath's directory; ENOENT → investigate concurrent deletion
3. Move the client data_dir / socket path to a local filesystem (ext4/xfs) that supports chmod on sockets
4. Restart the alloc to recreate the socket and retry chmod

Example fix

// before
if err := os.Chmod(hostHTTPSockPath, os.ModePerm); err != nil {
	return fmt.Errorf("unable to set permissions on unix socket: %w", err)
}

// after: tolerate already-removed socket
if err := os.Chmod(hostHTTPSockPath, os.ModePerm); err != nil {
	if _, statErr := os.Lstat(hostHTTPSockPath); os.IsNotExist(statErr) {
		return errSocketRemoved
	}
	return fmt.Errorf("unable to set permissions on unix socket: %w", err)
}

// verify we can chmod in the socket directory before creating the listener
sockDir := filepath.Dir(hostHTTPSockPath)
probe := filepath.Join(sockDir, ".chmodprobe")
if err := os.WriteFile(probe, nil, 0o777); err != nil {
	return fmt.Errorf("cannot manage permissions in %s: %w", sockDir, err)
}
os.Remove(probe)

if err := os.Chmod(hostHTTPSockPath, os.ModePerm); err != nil {
	switch {
	case errors.Is(err, os.ErrPermission):
		// client lacks ownership; escalate/restart agent as root
	case errors.Is(err, os.ErrNotExist):
		// socket removed concurrently; recreate listener
	}
	return fmt.Errorf("unable to set permissions on unix socket: %w", err)
}

When it happens

Trigger: run() successfully creates the listener at hostHTTPSockPath but os.Chmod(hostHTTPSockPath, os.ModePerm) returns an error — e.g. the file was removed between listen and chmod, the client runs as an unprivileged user not owning the socket, or the filesystem disallows chmod.

Common situations: Nomad client agent running as a non-root user without ownership of the socket path; unusual filesystems (some network mounts) rejecting chmod on sockets; security software or containers mounting paths with restricted chmod; race where the socket file is deleted concurrently.

Understand the failure class

Background: Permission denied / not authorized / 403 Forbidden: access-control rejections when the caller lacks the required role, grant, or ownership — this error's family across 18 libraries.

Related errors

• Chmod(%v) failed: %w
• unable to create unix socket for Consul HTTP endpoint: %w
• unable to remove existing unix socket: %w
• error chmoding file %w
• failed to change directory permissions for the AllocDir: %v

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/c67f9e72a8d4805f. Report an issue: GitHub.