{"record":{"id":"96831c8f4cff3e24","repo":"hashicorp/nomad","slug":"failed-to-mount-secrets-dir-v","errorCode":null,"errorMessage":"Failed to mount secrets dir: %v","messagePattern":"Failed to mount secrets dir: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"client/allocdir/task_dir.go","lineNumber":210,"sourceCode":"\t\t\treturn fmt.Errorf(\"Failed to create task mount directory: %v\", err)\n\t\t}\n\t\tif err = os.Chown(parent, uid, gid); err != nil {\n\t\t\treturn fmt.Errorf(\"Failed to chown task mount directory: %v\", err)\n\t\t}\n\n\t\t// create the taskdir mount point\n\t\tif err = mountDir(t.Dir, t.MountsTaskDir, uid, gid, fileMode710); err != nil {\n\t\t\treturn fmt.Errorf(\"Failed to mount task dir: %v\", err)\n\t\t}\n\n\t\t// create the allocdir mount point (owned by nobody)\n\t\tif err = mountDir(filepath.Join(t.AllocDir, \"/alloc\"), t.MountsAllocDir, nobodyUID, nobodyGID, fileMode777); err != nil {\n\t\t\treturn fmt.Errorf(\"Failed to mount alloc dir: %v\", err)\n\t\t}\n\n\t\t// create the secretsdir mount point\n\t\tif err = mountDir(t.SecretsDir, t.MountsSecretsDir, uid, gid, fileMode710); err != nil {\n\t\t\treturn fmt.Errorf(\"Failed to mount secrets dir: %v\", err)\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// buildChroot takes a mapping of absolute directory or file paths on the host\n// to their intended, relative location within the task directory. This\n// attempts hardlink and then defaults to copying. If the path exists on the\n// host and can't be embedded an error is returned.\nfunc (t *TaskDir) buildChroot(entries map[string]string) error {\n\treturn t.embedDirs(entries)\n}\n\nfunc (t *TaskDir) embedDirs(entries map[string]string) error {\n\tsubdirs := make(map[string]string)\n\tfor source, dest := range entries {\n\t\tif t.skip.Contains(source) {","sourceCodeStart":192,"sourceCodeEnd":228,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/client/allocdir/task_dir.go#L192-L228","documentation":"TaskDir.Build (Unveil mode) bind-mounts the task's secrets directory into the mounts path with mode 0710 owned by the task user. This error wraps mountDir failure for the secrets dir. Secrets dir provisioning (MakeSecretsDirs, incl. size limit) precedes this, so a failure here is mount/permission related.","triggerScenarios":"mountDir(t.SecretsDir, t.MountsSecretsDir, uid, gid, 0710) fails during TaskDir.Build(Unveil) — mount(2) error or mount-point creation/chown failure.","commonSituations":"Client without CAP_SYS_ADMIN (containerized); stale mount at MountsSecretsDir from previous alloc; secrets tmpfs size limit misconfigured earlier leaving the dir in a bad state; data_dir on a filesystem that disallows bind mounts.","solutions":["Run the client as root or with CAP_SYS_ADMIN","Check the wrapped errno; EBUSY → unmount the stale secrets mount point and retry","Verify <data_dir> supports bind mounts (avoid exotic FUSE/network filesystems for data_dir)","Restart the client to clean the mounts tree: `rm -rf <data_dir>/client/mounts/<task>` while stopped"],"exampleFix":"# before\nFailed to mount secrets dir: device or resource busy\n# after\n$ umount /var/nomad/client/mounts/<alloc-task>/secrets\n$ systemctl restart nomad","handlingStrategy":"try-catch","validationCode":"// preflight bind-mount capability and a clean mount point\nif _, err := os.Stat(mountsSecretsDir); err == nil {\n    exec.Command(\"umount\", mountsSecretsDir).Run() // clear stale mount\n}\nif err := syscall.Mount(\"tmpfs-probe\", probesDir, \"tmpfs\", 0, \"\"); err != nil {\n    log.Fatalf(\"mounts unavailable (need root/CAP_SYS_ADMIN): %v\", err)\n}\nsyscall.Unmount(probesDir, 0)","typeGuard":null,"tryCatchPattern":"if err := taskDir.Build(fsi, chroot, username); err != nil {\n    if strings.Contains(err.Error(), \"Failed to mount secrets dir\") {\n        exec.Command(\"umount\", mountsSecretsDir).Run()\n        return retryBuild()\n    }\n    return err\n}","preventionTips":["Run the client with mount privileges (root or CAP_SYS_ADMIN)","Use local disk for data_dir (bind mounts on FUSE/network FS are fragile)","Clear stale secrets mounts when a client crashes","Validate secrets tmpfs size config before deploy"],"tags":["filesystem","mount","secrets"],"backgroundTag":"mount-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}