{"record":{"id":"0e40b5ac098af5db","repo":"hashicorp/nomad","slug":"w-root-key-w","errorCode":null,"errorMessage":"%w (root key): %w","messagePattern":"%w \\(root key\\): %w","errorType":"console","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"nomad/encrypter.go","lineNumber":577,"sourceCode":"// channel.\n//\n// The error returned is only for testing and diagnostics.\nfunc (e *Encrypter) decryptWrappedKeyTask(\n\tctx context.Context, wrapper kms.Wrapper, meta *structs.RootKeyMeta,\n\twrappedKey *structs.WrappedKey, respCh chan *cipherSet) error {\n\n\tvar key []byte\n\tvar rsaKey []byte\n\n\tminBackoff := time.Second\n\tmaxBackoff := time.Second * 5\n\n\terr := helper.WithBackoffFunc(ctx, minBackoff, maxBackoff, func() error {\n\t\twrappedDEK := wrappedKey.WrappedDataEncryptionKey\n\t\tvar err error\n\t\tkey, err = wrapper.Decrypt(e.srv.shutdownCtx, wrappedDEK)\n\t\tif err != nil {\n\t\t\terr := fmt.Errorf(\"%w (root key): %w\", ErrDecryptFailed, err)\n\t\t\te.log.Error(err.Error(), \"key_id\", meta.KeyID)\n\t\t\treturn err\n\t\t}\n\t\treturn nil\n\t})\n\tif err != nil {\n\t\treturn err\n\t}\n\n\terr = helper.WithBackoffFunc(ctx, minBackoff, maxBackoff, func() error {\n\t\tvar err error\n\n\t\t// Decrypt RSAKey for Workload Identity JWT signing if one exists. Prior to\n\t\t// 1.7 an ed25519 key derived from the root key was used instead of an RSA\n\t\t// key.\n\t\tif wrappedKey.WrappedRSAKey != nil && len(wrappedKey.WrappedRSAKey.Ciphertext) > 0 {\n\t\t\trsaKey, err = wrapper.Decrypt(e.srv.shutdownCtx, wrappedKey.WrappedRSAKey)\n\t\t\tif err != nil {","sourceCodeStart":559,"sourceCodeEnd":595,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/encrypter.go#L559-L595","documentation":"Nomad's Encrypter failed to decrypt a wrapped root key's data-encryption key (DEK) using the KMS wrapper during keyring restoration. The error is wrapped as ErrDecryptFailed with the underlying crypto error, meaning the ciphertext could not be opened with the supplied root key material. It is retried with backoff first, since transient issues (e.g. KMS unavailability) are possible.","triggerScenarios":"wrapper.Decrypt(shutdownCtx, wrappedDEK) returns an error inside the backoff retry while restoring a keyring key — typically because the wrapped DEK ciphertext was produced by a different/rotated root key, or the external KMS (Vault Transit, AWS KMS, etc.) rejects the decrypt operation.","commonSituations":"Key material changed or was rotated out from under stored wrapped keys; external KMS credentials expired or the KMS is unreachable; keyring state was copied between clusters or environments with different root keys; corrupted raft state after a restore.","solutions":["Check the underlying wrapped error for the real cause (auth failure vs bad ciphertext vs KMS unreachable) and fix that dependency first","Verify the external KMS credentials/config used by the kms wrapper are valid and the key still exists","Confirm the wrapped key ciphertext belongs to the same root key (KeyID in meta matches); restore a consistent keyring snapshot if state was copied across clusters","If keyring state is unrecoverable, reinitialize the keyring (keyring rotate/bootstrap) — note this affects decryptability of previously encrypted data"],"exampleFix":"// no caller code fix; fix the KMS dependency\n// before: wrapper can't authenticate to Vault Transit\n// after: export VAULT_TOKEN=<valid token> and ensure transit key <key_id> exists, then restart Nomad agent","handlingStrategy":"retry","validationCode":"// before relying on decrypt: verify the external KMS is reachable/authorized\nerr := wrapper.Encrypt(ctx, []byte(\"probe\"))\nif err != nil { /* fix KMS credentials/connectivity first */ }","typeGuard":null,"tryCatchPattern":"if err != nil {\n    if errors.Is(err, ErrDecryptFailed) {\n        // inspect wrapped cause; retry with backoff or re-rotate keyring\n    }\n}","preventionTips":["Monitor external KMS credentials/tokens for expiry","Never copy keyring raft state between clusters with different root keys","Test keyring restore procedures in staging"],"tags":["encryption","kms","keyring","decrypt-failed"],"backgroundTag":"decrypt-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}