{"record":{"id":"4a07408636e26aa0","repo":"hashicorp/nomad","slug":"failed-to-write-consul-si-token-w","errorCode":null,"errorMessage":"failed to write Consul SI token: %w","messagePattern":"failed to write Consul SI token: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"client/allocrunner/taskrunner/consul_hook.go","lineNumber":78,"sourceCode":"\t\tfor tokenName, token := range t {\n\t\t\ts := strings.SplitN(tokenName, \"/\", 2)\n\t\t\tif len(s) < 2 {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tidentity := s[0]\n\t\t\ttaskName := s[1]\n\t\t\t// do not write tokens that do not belong to any of this task's\n\t\t\t// identities\n\t\t\tif taskName != h.task.Name || !slices.ContainsFunc(\n\t\t\t\th.task.Identities,\n\t\t\t\tfunc(id *structs.WorkloadIdentity) bool { return id.Name == identity }) &&\n\t\t\t\tidentity != h.task.Identity.Name {\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\ttokenPath := filepath.Join(h.tokenDir, consulTokenFilename)\n\t\t\tif err := os.WriteFile(tokenPath, []byte(token.SecretID), consulTokenFilePerms); err != nil {\n\t\t\t\tmErr.Errors = append(mErr.Errors, fmt.Errorf(\"failed to write Consul SI token: %w\", err))\n\t\t\t}\n\n\t\t\tenv := map[string]string{\n\t\t\t\t\"CONSUL_TOKEN\":      token.SecretID,\n\t\t\t\t\"CONSUL_HTTP_TOKEN\": token.SecretID,\n\t\t\t}\n\n\t\t\tresp.Env = env\n\t\t}\n\t}\n\n\treturn mErr.ErrorOrNil()\n}\n","sourceCodeStart":60,"sourceCodeEnd":92,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/client/allocrunner/taskrunner/consul_hook.go#L60-L92","documentation":"Nomad's Consul SI (service identity) task hook wraps the token for the workload into a file (consul_token) inside the task's token directory during Prestart. This error means the os.WriteFile of the Consul SI token SecretID to that file failed, and the underlying OS error is wrapped with %w. It is accumulated into a multi-error rather than returned immediately, so Prestart can report all token write failures at once.","triggerScenarios":"os.WriteFile(filepath.Join(h.tokenDir, consulTokenFilename), ...) fails: the token directory does not exist, has wrong permissions, the disk is full or read-only, or the task's token dir was removed mid-prestart. Triggered in Prestart when a workload identity of kind consul matches the task and the hook tries to materialize the token file.","commonSituations":"Disk full on the client data volume; tokenDir not created due to earlier filesystem errors; running with a chroot/env where the path is not writable; SELinux/AppArmor denying writes to the token directory.","solutions":["Check the wrapped underlying error (use errors.Unwrap / %v of the multi-error) to identify the OS cause","Verify the task's token directory exists and is writable by the Nomad agent user","Check disk space and mount ro flags on the client's data/state volume","Inspect SELinux/AppAudit policy denials if using enforcing security modules","Retry the allocation after fixing storage; the hook re-runs on restart"],"exampleFix":"// before: token dir assumed to exist\nos.WriteFile(tokenPath, []byte(token.SecretID), consulTokenFilePerms)\n// after: ensure directory exists first\nos.MkdirAll(h.tokenDir, 0o700)\nos.WriteFile(tokenPath, []byte(token.SecretID), consulTokenFilePerms)","handlingStrategy":"validation","validationCode":"// before Prestart relies on the token dir\nif info, err := os.Stat(tokenDir); err != nil || !info.IsDir() {\n    return fmt.Errorf(\"consul token dir %s unavailable: %w\", tokenDir, err)\n}","typeGuard":null,"tryCatchPattern":"if err := hook.Prestart(req); err != nil {\n    var pathErr *os.PathError\n    if errors.As(err, &pathErr) {\n        log.Printf(\"token write failed on %s: %v\", pathErr.Path, pathErr.Err)\n    }\n}","preventionTips":["Ensure the task token directory is created with 0700 before the hook runs","Monitor client disk usage and alert before volumes fill","Validate storage permissions in client setup scripts","Check audit logs for LSM denials on the Nomad data dir"],"tags":["consul","filesystem","nomad-client"],"backgroundTag":"file-write-permission-denied","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}