apache/hadoop · critical · IOException
Can't store keystore ${this}
Error message
Can't store keystore ${this} What it means
writeToNew serializes the whole keystore to the *_NEW file during flush; KeyStore.store() threw KeyStoreException, meaning the in-memory KeyStore could not be encoded at all (an uninitialized or internally inconsistent store instance, rather than a filesystem problem - FS errors surface as plain IOExceptions from the stream). The flush aborts before the old file is backed up, so the previous on-disk keystore is untouched but all pending updates are lost.
Source
Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/JavaKeyStoreProvider.java:612
LOG.debug("KeyStore resetting to previously flushed state !!");
} catch (Exception e) {
LOG.debug("Could not reset Keystore to previous state", e);
}
}
private void cleanupNewAndOld(Path newPath, Path oldPath) throws IOException {
// Rename _NEW to CURRENT
renameOrFail(newPath, path);
// Delete _OLD
fs.delete(oldPath, true);
}
protected void writeToNew(Path newPath) throws IOException {
try (FSDataOutputStream out =
FileSystem.create(fs, newPath, permissions);) {
keyStore.store(out, password);
} catch (KeyStoreException e) {
throw new IOException("Can't store keystore " + this, e);
} catch (NoSuchAlgorithmException e) {
throw new IOException(
"No such algorithm storing keystore " + this, e);
} catch (CertificateException e) {
throw new IOException(
"Certificate exception storing keystore " + this, e);
}
}
protected boolean backupToOld(Path oldPath)
throws IOException {
try {
renameOrFail(path, oldPath);
return true;
} catch (FileNotFoundException e) {
return false;
}
}View on GitHub (pinned to 2add963021)
Solutions
- Check that the current keystore file is loadable and not corrupt (`hadoop key list -provider <uri>`)
- Remove leftover *_NEW/_OLD files and retry the flush
- If the store is unrecoverable, recreate the keystore and re-add keys from secure backups
- Confirm a stable, standard JCE configuration in the JVM
Defensive patterns
Strategy: try-catch
Validate before calling
// preflight: prove the store serializes before relying on flush
try { provider.getKeys(); } catch (IOException e) { throw new IllegalStateException("keystore unhealthy: " + e.getMessage(), e); } Try / catch
try { provider.flush(); } catch (IOException e) { if (String.valueOf(e.getMessage()).startsWith("Can't store keystore")) { // CRITICAL: pending keys NOT persisted; keep process alive if keys remain in cache, alert ops, repair store and re-flush } throw e; } Prevention
- Never assume a createKey/rollNewVersion succeeded durably until flush returns
- Canary-test keystore write+flush at service startup
- Back up keystore files and verify restore path
- Monitor for *_NEW/_OLD residue as crash evidence
When it happens
Trigger: A KeyStore instance that was never load()ed successfully; entries in an encoding-incompatible state after corruption; provider-level failure during serialization.
Common situations: Keystore file unreadable at provider construction so the store is empty/uninitialized; corrupt store loaded partially; JVM provider bugs.
Related errors
- Problem removing ${versionName} from ${this}
- Problem removing ${name} from ${this}
- Can't store key ${versionName} in ${this}
- Can't set metadata key ${entry.getKey()}
- Certificate exception storing keystore ${this}
AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22).
Data as JSON: /api/errors/008ebfeab7d2a293.
Report an issue: GitHub.