hashicorp/nomad · error

unable to set permissions on unix socket

Error message

unable to set permissions on unix socket: %w

What it means

After creating the Consul HTTP unix socket, the hook chmods it to os.ModePerm (0777) so non-root task users can connect to the Consul HTTP API through the alloc's network namespace. If os.Chmod fails (usually a permissions or filesystem issue on the socket path), the socket would be unusable by unprivileged tasks, so the hook aborts with this wrapped error.

Solutions

  1. Run the Nomad client as root (or a user with ownership of the host_http_socket path) so chmod succeeds
  2. Check the wrapped OS error: EPERM → fix ownership of hostHTTPSockPath's directory; ENOENT → investigate concurrent deletion
  3. Move the client data_dir / socket path to a local filesystem (ext4/xfs) that supports chmod on sockets
  4. Restart the alloc to recreate the socket and retry chmod

Example fix

// before
if err := os.Chmod(hostHTTPSockPath, os.ModePerm); err != nil {
	return fmt.Errorf("unable to set permissions on unix socket: %w", err)
}

// after: tolerate already-removed socket
if err := os.Chmod(hostHTTPSockPath, os.ModePerm); err != nil {
	if _, statErr := os.Lstat(hostHTTPSockPath); os.IsNotExist(statErr) {
		return errSocketRemoved
	}
	return fmt.Errorf("unable to set permissions on unix socket: %w", err)
}
Defensive patterns

Strategy: try-catch

Validate before calling

// verify we can chmod in the socket directory before creating the listener
sockDir := filepath.Dir(hostHTTPSockPath)
probe := filepath.Join(sockDir, ".chmodprobe")
if err := os.WriteFile(probe, nil, 0o777); err != nil {
	return fmt.Errorf("cannot manage permissions in %s: %w", sockDir, err)
}
os.Remove(probe)

Try / catch

if err := os.Chmod(hostHTTPSockPath, os.ModePerm); err != nil {
	switch {
	case errors.Is(err, os.ErrPermission):
		// client lacks ownership; escalate/restart agent as root
	case errors.Is(err, os.ErrNotExist):
		// socket removed concurrently; recreate listener
	}
	return fmt.Errorf("unable to set permissions on unix socket: %w", err)
}

Prevention

When it happens

Trigger: run() successfully creates the listener at hostHTTPSockPath but os.Chmod(hostHTTPSockPath, os.ModePerm) returns an error — e.g. the file was removed between listen and chmod, the client runs as an unprivileged user not owning the socket, or the filesystem disallows chmod.

Common situations: Nomad client agent running as a non-root user without ownership of the socket path; unusual filesystems (some network mounts) rejecting chmod on sockets; security software or containers mounting paths with restricted chmod; race where the socket file is deleted concurrently.

Understand the failure class

Background: Permission denied / not authorized / 403 Forbidden: access-control rejections when the caller lacks the required role, grant, or ownership — this error's family across 18 libraries.

Related errors


AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/c67f9e72a8d4805f. Report an issue: GitHub.

Appendix: source

Thrown at client/allocrunner/consul_http_sock_hook.go:236

		socketFile = filepath.Join(allocdir.SharedAllocName, allocdir.TmpDirName,
			"consul_"+p.config.Name+"_http.sock")
	}
	hostHTTPSockPath := filepath.Join(p.allocDir.AllocDirPath(), socketFile)
	if err := maybeRemoveOldSocket(hostHTTPSockPath); err != nil {
		return err
	}

	listener, err := net.Listen("unix", hostHTTPSockPath)
	if err != nil {
		return fmt.Errorf("unable to create unix socket for Consul HTTP endpoint: %w", err)
	}

	// The Consul HTTP socket should be usable by all users in case a task is
	// running as a non-privileged user. Unix does not allow setting domain
	// socket permissions when creating the file, so we must manually call
	// chmod afterwards.
	if err := os.Chmod(hostHTTPSockPath, os.ModePerm); err != nil {
		return fmt.Errorf("unable to set permissions on unix socket: %w", err)
	}

	go func() {
		proxy(p.ctx, p.logger, destAddr, listener)
		p.cancel()
		close(p.doneCh)
	}()

	p.runOnce = true
	return nil
}

func (p *httpSocketProxy) stop() error {
	p.cancel()

	// if proxy was never run, no need to wait before shutdown
	if !p.runOnce {
		return nil

View on GitHub (pinned to 482b49bf1a)