{"record":{"id":"5791e8829254ca05","repo":"hashicorp/nomad","slug":"unable-to-create-unix-socket-for-consul-http-endpo","errorCode":null,"errorMessage":"unable to create unix socket for Consul HTTP endpoint: %w","messagePattern":"unable to create unix socket for Consul HTTP endpoint: %w","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"error","filePath":"client/allocrunner/consul_http_sock_hook.go","lineNumber":228,"sourceCode":"\t// consul http dest addr\n\tdestAddr := p.config.Addr\n\tif destAddr == \"\" {\n\t\treturn errors.New(\"consul address must be set on nomad client\")\n\t}\n\n\tsocketFile := allocdir.AllocHTTPSocket\n\tif p.config.Name != structs.ConsulDefaultCluster && p.config.Name != \"\" {\n\t\tsocketFile = filepath.Join(allocdir.SharedAllocName, allocdir.TmpDirName,\n\t\t\t\"consul_\"+p.config.Name+\"_http.sock\")\n\t}\n\thostHTTPSockPath := filepath.Join(p.allocDir.AllocDirPath(), socketFile)\n\tif err := maybeRemoveOldSocket(hostHTTPSockPath); err != nil {\n\t\treturn err\n\t}\n\n\tlistener, err := net.Listen(\"unix\", hostHTTPSockPath)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"unable to create unix socket for Consul HTTP endpoint: %w\", err)\n\t}\n\n\t// The Consul HTTP socket should be usable by all users in case a task is\n\t// running as a non-privileged user. Unix does not allow setting domain\n\t// socket permissions when creating the file, so we must manually call\n\t// chmod afterwards.\n\tif err := os.Chmod(hostHTTPSockPath, os.ModePerm); err != nil {\n\t\treturn fmt.Errorf(\"unable to set permissions on unix socket: %w\", err)\n\t}\n\n\tgo func() {\n\t\tproxy(p.ctx, p.logger, destAddr, listener)\n\t\tp.cancel()\n\t\tclose(p.doneCh)\n\t}()\n\n\tp.runOnce = true\n\treturn nil","sourceCodeStart":210,"sourceCodeEnd":246,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/client/allocrunner/consul_http_sock_hook.go#L210-L246","documentation":"The hook creates a Unix domain socket at hostHTTPSockPath so Consul's HTTP API can be reached securely from within the alloc's network namespace. net.Listen(\"unix\", ...) failed — typically because the path already exists and is stale, the directory doesn't exist, or permission/filesystem constraints prevent socket creation. The underlying OS error is wrapped for diagnosis.","triggerScenarios":"run() calls maybeRemoveOldSocket then net.Listen(\"unix\", hostHTTPSockPath); listen fails with EADDRINUSE (stale socket file still present), ENOENT (parent dir missing), EACCES (permissions), or ENOTSUP (path on a filesystem that disallows unix sockets, e.g. some NFS/VFS mounts).","commonSituations":"Leftover socket from a crashed previous alloc; alloc dir on NFS or tmpfs without socket support; data_dir permissions changed; host path pointing at a read-only mount.","solutions":["Remove the stale socket file at hostHTTPSockPath manually (rm <path>) or ensure maybeRemoveOldSocket succeeded, then restart the alloc","Check the wrapped OS error: EACCES → fix directory permissions; ENOENT → create the parent directory","Move Nomad's client data_dir off filesystems that don't support unix sockets (NFS) to local disk","Restart the Nomad client agent to clean up leaked sockets from crashed allocs"],"exampleFix":"// before: listen fails on stale socket\nlistener, err := net.Listen(\"unix\", hostHTTPSockPath)\n\n// after: force-remove then listen\nif err := maybeRemoveOldSocket(hostHTTPSockPath); err != nil {\n\treturn err\n}\nlistener, err := net.Listen(\"unix\", hostHTTPSockPath)","handlingStrategy":"validation","validationCode":"sockDir := filepath.Dir(hostHTTPSockPath)\nif fi, err := os.Stat(sockDir); err != nil || !fi.IsDir() {\n\treturn fmt.Errorf(\"socket dir %s missing: %w\", sockDir, err)\n}\n// test unix socket support on this filesystem\ntestPath := filepath.Join(sockDir, \".socktest\")\nl, err := net.Listen(\"unix\", testPath)\nif err != nil {\n\treturn fmt.Errorf(\"filesystem %s does not support unix sockets: %w\", sockDir, err)\n}\nl.Close(); os.Remove(testPath)","typeGuard":null,"tryCatchPattern":"listener, err := net.Listen(\"unix\", hostHTTPSockPath)\nif err != nil {\n\tif errors.Is(err, syscall.EADDRINUSE) {\n\t\tos.Remove(hostHTTPSockPath) // stale socket; retry once\n\t\tlistener, err = net.Listen(\"unix\", hostHTTPSockPath)\n\t}\n\tif err != nil { return err }\n}","preventionTips":["Keep Nomad client data_dir on local disk, not NFS or socket-hostile filesystems","Run the client agent with permission to create files in the socket's parent directory","Restart clients cleanly so stale sockets from crashed allocs are removed","Pre-check the wrapped OS error (EADDRINUSE/ENOENT/EACCES) for targeted remediation"],"tags":["consul","unix-socket","network","filesystem"],"backgroundTag":"unix-socket-listen-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}