{"record":{"id":"e61699f037bff175","repo":"apache/hadoop","slug":"can-t-recover-key-for-name-from-keystore-path","errorCode":null,"errorMessage":"Can't recover key for ${name} from keystore ${path}","messagePattern":"Can't recover key for (.+?) from keystore (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/JavaKeyStoreProvider.java","lineNumber":425,"sourceCode":"      try {\n        if (!keyStore.containsAlias(name)) {\n          return null;\n        }\n        Metadata meta = ((KeyMetadata) keyStore.getKey(name, password)).metadata;\n        cache.put(name, meta);\n        return meta;\n      } catch (ClassCastException e) {\n        throw new IOException(\"Can't cast key for \" + name + \" in keystore \" +\n            path + \" to a KeyMetadata. Key may have been added using \" +\n            \" keytool or some other non-Hadoop method.\", e);\n      } catch (KeyStoreException e) {\n        throw new IOException(\"Can't get metadata for \" + name +\n            \" from keystore \" + path, e);\n      } catch (NoSuchAlgorithmException e) {\n        throw new IOException(\"Can't get algorithm for \" + name +\n            \" from keystore \" + path, e);\n      } catch (UnrecoverableKeyException e) {\n        throw new IOException(\"Can't recover key for \" + name +\n            \" from keystore \" + path, e);\n      }\n    } finally {\n      readLock.unlock();\n    }\n  }\n\n  @Override\n  public KeyVersion createKey(String name, byte[] material,\n                               Options options) throws IOException {\n    Preconditions.checkArgument(name.equals(StringUtils.toLowerCase(name)),\n        \"Uppercase key names are unsupported: %s\", name);\n    writeLock.lock();\n    try {\n      try {\n        if (keyStore.containsAlias(name) || cache.containsKey(name)) {\n          throw new IOException(\"Key \" + name + \" already exists in \" + this);\n        }","sourceCodeStart":407,"sourceCodeEnd":443,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/JavaKeyStoreProvider.java#L407-L443","documentation":"In getMetadata(), getKey() threw UnrecoverableKeyException: the configured keystore password cannot decrypt the metadata entry for the named key. The metadata entry is stored as a password-protected key, so a store-password mismatch shows up here rather than at file load.","triggerScenarios":"Any key operation that reads metadata (hadoop key list/roll/delete, KMS requests) when the password file (KEYSTORE_PASSWORD_FILE_KEY) does not match the keystore's actual store password, or the entry was protected with a different key password.","commonSituations":"Password rotated on one host but not all KMS nodes; multiple KMS instances configured with different password files; keystore restored from backup made under another password","solutions":["Verify and fix the password file contents against keytool -list using that exact file","Align the password configuration across all KMS hosts and restart","If the store password was intentionally changed, re-create entries so metadata is recoverable under the new password"],"exampleFix":"# before: hadoop key roll mykey fails: Can't recover key for mykey\n\n# after: validate the exact password KMS uses, then align\nprintf '%s' \"$(cat /etc/security/ks.pwd)\" | wc -c   # check for stray newline\nkeytool -list -keystore /etc/security/ks.jks -storetype jceks \\\n  -storepass:file /etc/security/ks.pwd\n# fix file or keystore until this lists, restart KMS","handlingStrategy":"validation","validationCode":"// Prove metadata entries decrypt before serving key ops\nKeyStore ks = KeyStore.getInstance(\"jceks\");\nks.load(in, pwd);\nks.getKey(name + \"_metadata\", pwd); // UnrecoverableKeyException here = password mismatch","typeGuard":null,"tryCatchPattern":"try {\n  meta = provider.getMetadata(name);\n} catch (IOException e) {\n  if (e.getCause() instanceof UnrecoverableKeyException) {\n    // store password wrong: align KEYSTORE_PASSWORD_FILE_KEY across hosts, restart\n  }\n}","preventionTips":["Rotate keystore passwords atomically: file and config together, all hosts","Script a keytool -list validation as part of KMS config deployment","Treat one key failing while others work as an entry-password anomaly worth investigating"],"tags":["keystore","kms","crypto","password","metadata"],"backgroundTag":"keystore-wrong-password","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}