hashicorp/nomad · critical

could not read path %s from keystore: %v

Error message

could not read path %s from keystore: %v

What it means

Encrypter startup walks the on-disk keystore directory to load encryption keys; if filepath.Walk reports any error for a path (unreadable directory, permission denied, etc.), Nomad returns this error naming the path and underlying cause. It signals the keystore cannot be fully read and keys may be missing.

Source

Thrown at nomad/encrypter.go:159

	// default to false as this will be parsed by the go-kms-wrapping package
	skipVerify := ""
	if vaultcfg.TLSSkipVerify != nil {
		skipVerify = fmt.Sprintf("%v", *vaultcfg.TLSSkipVerify)
	}
	setFallback("tls_skip_verify", skipVerify, "VAULT_SKIP_VERIFY", "false")
}

func (e *Encrypter) loadKeystore() error {

	if err := os.MkdirAll(e.keystorePath, 0o700); err != nil {
		return err
	}

	keyErrors := map[string]error{}

	filepath.Walk(e.keystorePath, func(path string, info fs.FileInfo, err error) error {
		if err != nil {
			return fmt.Errorf("could not read path %s from keystore: %v", path, err)
		}

		// skip over subdirectories and non-key files; they shouldn't
		// be here but there's no reason to fail startup for it if the
		// administrator has left something there
		if path != e.keystorePath && info.IsDir() {
			return filepath.SkipDir
		}
		if !strings.HasSuffix(path, nomadKeystoreExtension) {
			return nil
		}
		idWithIndex := strings.TrimSuffix(filepath.Base(path), nomadKeystoreExtension)
		id, _, _ := strings.Cut(idWithIndex, ".")
		if !helper.IsUUID(id) {
			return nil
		}

		e.keyringLock.RLock()

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Verify the keystore path in the server config exists and is readable by the Nomad process user: ls -ld <keystore_path>
2. Fix permissions/ownership (chown/chmod) on the keystore directory and key files
3. If the mount/volume hosting the keystore is missing, remount it and restart the server
4. Correct or create the keystore directory, then re-run the server; keys are needed for workload encryption (JWT/Vault-adjacent features)

Example fix

// before: server config points at a missing dir
server { encryption { keystore_path = "/opt/nomad/keys" } }  # /opt/nomad/keys missing
// after
mkdir -p /opt/nomad/keys && chown nomad:nomad /opt/nomad/keys && chmod 700 /opt/nomad/keys

const fs = require('fs')
function keystoreReadable(p) {
  try { fs.accessSync(p, fs.constants.R_OK | fs.constants.X_OK); return fs.statSync(p).isDirectory() }
  catch { return false }
}
keystoreReadable('/var/lib/nomad/keystore') // true means safe to start server

try {
  encrypter.loadKeystore()
} catch (e) {
  if (e.message.startsWith('could not read path')) {
    const p = e.message.split(' ')[4]
    // fix perms/mount then retry
    fixKeystorePerms(p); encrypter.loadKeystore()
  } else { throw e }
}

When it happens

Trigger: During Encrypter initialization, filepath.Walk(e.keystorePath) invokes the callback with a non-nil err — e.g. the keystore directory doesn't exist, lacks read permission, or contains an unreadable entry.

Common situations: Misconfigured keystore_path in server config pointing to a missing or wrong directory; incorrect file ownership/permissions after migration or container image changes; keystore on a volume that failed to mount; running Nomad as a user without access to the key files.

Related errors

• plugin not executable
• Chmod(%v) failed: %w
• Couldn't change owner/group of %v to (uid: %v, gid: %v): %w
• unable to remove existing unix socket: %w
• failed to write vault token to secrets dir: %v

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/b83a088a0b156711. Report an issue: GitHub.