hashicorp/nomad · critical
could not read path from keystore
Error message
could not read path %s from keystore: %v
What it means
Encrypter startup walks the on-disk keystore directory to load encryption keys; if filepath.Walk reports any error for a path (unreadable directory, permission denied, etc.), Nomad returns this error naming the path and underlying cause. It signals the keystore cannot be fully read and keys may be missing.
Solutions
- Verify the keystore path in the server config exists and is readable by the Nomad process user: ls -ld <keystore_path>
- Fix permissions/ownership (chown/chmod) on the keystore directory and key files
- If the mount/volume hosting the keystore is missing, remount it and restart the server
- Correct or create the keystore directory, then re-run the server; keys are needed for workload encryption (JWT/Vault-adjacent features)
Example fix
// before: server config points at a missing dir
server { encryption { keystore_path = "/opt/nomad/keys" } } # /opt/nomad/keys missing
// after
mkdir -p /opt/nomad/keys && chown nomad:nomad /opt/nomad/keys && chmod 700 /opt/nomad/keys
Defensive patterns
Strategy: validation
Validate before calling
const fs = require('fs')
function keystoreReadable(p) {
try { fs.accessSync(p, fs.constants.R_OK | fs.constants.X_OK); return fs.statSync(p).isDirectory() }
catch { return false }
}
keystoreReadable('/var/lib/nomad/keystore') // true means safe to start server Try / catch
try {
encrypter.loadKeystore()
} catch (e) {
if (e.message.startsWith('could not read path')) {
const p = e.message.split(' ')[4]
// fix perms/mount then retry
fixKeystorePerms(p); encrypter.loadKeystore()
} else { throw e }
} Prevention
- Pre-provision the keystore directory with correct ownership for the Nomad user before startup
- Verify keystore_path in server config matches an existing, mounted volume
- After container/VM image updates, re-check permissions on the keystore
- Run startup health checks that stat the keystore path before launching nomad agent
When it happens
Trigger: During Encrypter initialization, filepath.Walk(e.keystorePath) invokes the callback with a non-nil err — e.g. the keystore directory doesn't exist, lacks read permission, or contains an unreadable entry.
Common situations: Misconfigured keystore_path in server config pointing to a missing or wrong directory; incorrect file ownership/permissions after migration or container image changes; keystore on a volume that failed to mount; running Nomad as a user without access to the key files.
Related errors
- Chmod( ) failed
- Couldn't change owner/group of
- error chmoding file
- error chowning directory
- error chowning file
AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04).
Data as JSON: /api/errors/b83a088a0b156711.
Report an issue: GitHub.
Appendix: source
Thrown at nomad/encrypter.go:159
// default to false as this will be parsed by the go-kms-wrapping package
skipVerify := ""
if vaultcfg.TLSSkipVerify != nil {
skipVerify = fmt.Sprintf("%v", *vaultcfg.TLSSkipVerify)
}
setFallback("tls_skip_verify", skipVerify, "VAULT_SKIP_VERIFY", "false")
}
func (e *Encrypter) loadKeystore() error {
if err := os.MkdirAll(e.keystorePath, 0o700); err != nil {
return err
}
keyErrors := map[string]error{}
filepath.Walk(e.keystorePath, func(path string, info fs.FileInfo, err error) error {
if err != nil {
return fmt.Errorf("could not read path %s from keystore: %v", path, err)
}
// skip over subdirectories and non-key files; they shouldn't
// be here but there's no reason to fail startup for it if the
// administrator has left something there
if path != e.keystorePath && info.IsDir() {
return filepath.SkipDir
}
if !strings.HasSuffix(path, nomadKeystoreExtension) {
return nil
}
idWithIndex := strings.TrimSuffix(filepath.Base(path), nomadKeystoreExtension)
id, _, _ := strings.Cut(idWithIndex, ".")
if !helper.IsUUID(id) {
return nil
}
e.keyringLock.RLock()View on GitHub (pinned to 482b49bf1a)