What causes "error reading %s: %w" in hashicorp/nomad?

Verify the PemKeyFile path exists on every Nomad server and is readable: `ls -l && sudo -u nomad cat `.

error reading %s: %w

Error message

error reading %s: %w

What it means

Fires in getCassPrivateKey when os.ReadFile fails on the OIDC client assertion's configured PemKeyFile — the file path is wrong, missing, or unreadable, so the private key for the client assertion JWT cannot be loaded.

Source

Thrown at lib/auth/oidc/client_assertion.go:128

		return cass.NewJWTWithRSAKey(clientID, as.Audience, algo, rsaKey, opts...)

	default: // this shouldn't happen, but just in case
		return nil, fmt.Errorf("unknown OIDC KeySource %q", as.KeySource)
	}
}

// getCassPrivateKey parses the structs.OIDCClientAssertionKey PemKeyFile
// or PemKey, depending on which is set.
func getCassPrivateKey(k *structs.OIDCClientAssertionKey) (key *rsa.PrivateKey, err error) {
	var bts []byte
	var source string // for informative error messages

	// pem file on disk
	if k.PemKeyFile != "" {
		source = "PemKeyFile"
		bts, err = os.ReadFile(k.PemKeyFile)
		if err != nil {
			return nil, fmt.Errorf("error reading %s: %w", source, err)
		}
	}
	// or pem string
	if k.PemKey != "" {
		source = "PemKey"
		bts = []byte(k.PemKey)
	}

	// ensure newlines around pem header/footer
	bts = newlineHeaders(bts)

	key, err = gojwt.ParseRSAPrivateKeyFromPEM(bts)
	if err != nil {
		return nil, fmt.Errorf("error parsing %s: %w", source, err)
	}
	if err := key.Validate(); err != nil {
		return nil, fmt.Errorf("error validating %s: %w", source, err)
	}

View on GitHub (pinned to 482b49bf1a)

Solutions

Verify the PemKeyFile path exists on every Nomad server and is readable: `ls -l <path> && sudo -u nomad cat <path>`.
Use an absolute path and ensure the key is provisioned/templated (e.g. Vault agent, volume mount) on server nodes.
Alternatively embed the key inline via PemKey instead of a file path.

Example fix

// before
key := &structs.OIDCClientAssertionKey{PemKeyFile: "client.key"}
// after
key := &structs.OIDCClientAssertionKey{PemKeyFile: "/etc/nomad.d/tls/oidc-client.key"}

Defensive patterns

Strategy: validation

Validate before calling

if k.PemKeyFile != "" {
  if _, err := os.ReadFile(k.PemKeyFile); err != nil {
    return fmt.Errorf("PemKeyFile unreadable before assertion build: %w", err)
  }
}

Try / catch

key, err := getCassPrivateKey(k)
if err != nil && strings.Contains(err.Error(), "error reading") {
  return fmt.Errorf("check PemKeyFile path/permissions on Nomad servers: %w", err)
}

Prevention

Use absolute paths and provision key files on all server nodes.
Check file ownership/permissions (readable by the nomad user) after provisioning.
Prefer inline PemKey when file distribution is unreliable.

When it happens

Trigger: BuildClientAssertionJWT → getCassPrivateKey when OIDCClientAssertionKey.PemKeyFile is set and the file cannot be read (missing path, wrong permissions, unreadable volume).

Common situations: Path typo or relative path that doesn't resolve on Nomad servers; file only mounted on clients, not servers; permissions preventing the nomad user from reading; secret not provisioned by the deployment tool.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/1510914f4c878d35. Report an issue: GitHub.