apache/hadoop · error · IOException

Algorithm 'MD5' not found

Error message

Algorithm 'MD5' not found

What it means

FSNamesystem initialization needs an MD5 MessageDigest (for the per-storage-dir digest it reports). On any standard JDK/JRE MD5 is built in, so this IOException ('Algorithm MD5 not found') essentially never fires; it indicates a broken security-provider setup — a stripped or altered java.security configuration, a JCE policy that removed MD5, or an exotic JVM whose providers do not expose MD5.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java:962

      // Get the checksum type from config
      String checksumTypeStr = conf.get(DFS_CHECKSUM_TYPE_KEY,
          DFS_CHECKSUM_TYPE_DEFAULT);
      this.isSnapshotTrashRootEnabled = conf.getBoolean(
          DFS_NAMENODE_SNAPSHOT_TRASHROOT_ENABLED,
          DFS_NAMENODE_SNAPSHOT_TRASHROOT_ENABLED_DEFAULT);
      DataChecksum.Type checksumType;
      try {
         checksumType = DataChecksum.Type.valueOf(checksumTypeStr);
      } catch (IllegalArgumentException iae) {
         throw new IOException("Invalid checksum type in "
            + DFS_CHECKSUM_TYPE_KEY + ": " + checksumTypeStr);
      }

      try {
        digest = MessageDigest.getInstance("MD5");
      } catch (NoSuchAlgorithmException e) {
        throw new IOException("Algorithm 'MD5' not found");
      }

      this.serverDefaults = new FsServerDefaults(
          conf.getLongBytes(DFS_BLOCK_SIZE_KEY, DFS_BLOCK_SIZE_DEFAULT),
          conf.getInt(DFS_BYTES_PER_CHECKSUM_KEY, DFS_BYTES_PER_CHECKSUM_DEFAULT),
          conf.getInt(DFS_CLIENT_WRITE_PACKET_SIZE_KEY, DFS_CLIENT_WRITE_PACKET_SIZE_DEFAULT),
          (short) conf.getInt(DFS_REPLICATION_KEY, DFS_REPLICATION_DEFAULT),
          conf.getInt(IO_FILE_BUFFER_SIZE_KEY, IO_FILE_BUFFER_SIZE_DEFAULT),
          conf.getBoolean(DFS_ENCRYPT_DATA_TRANSFER_KEY, DFS_ENCRYPT_DATA_TRANSFER_DEFAULT),
          conf.getLong(FS_TRASH_INTERVAL_KEY, FS_TRASH_INTERVAL_DEFAULT),
          checksumType,
          conf.getTrimmed(
              CommonConfigurationKeysPublic.HADOOP_SECURITY_KEY_PROVIDER_PATH,
              ""),
          blockManager.getStoragePolicySuite().getDefaultPolicy().getId(),
          isSnapshotTrashRootEnabled);

      this.maxFsObjects = conf.getLong(DFS_NAMENODE_MAX_OBJECTS_KEY, 

View on GitHub (pinned to 2add963021)

Solutions

1. Run the NameNode on a stock, supported JDK distribution
2. Verify MD5 availability in that JVM: a one-liner calling MessageDigest.getInstance("MD5") or 'java -XshowSettings:properties -version'
3. If MD5 was deliberately disabled (FIPS policy), use a JVM/provider configuration that still exposes it for the NN process — Hadoop's NameNode requires it
4. Reinstall or repair the JDK if the provider files were corrupted

Example fix

# before: NN JVM runs with a stripped java.security (MD5 absent)
# after: point the NN at a stock JDK
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk && hdfs --daemon start namenode

// deploy-time JVM check on the NameNode host
MessageDigest.getInstance("MD5"); // throws NoSuchAlgorithmException if the JVM lacks MD5

When it happens

Trigger: MessageDigest.getInstance("MD5") throwing NoSuchAlgorithmException during FSNamesystem construction — only possible when the JVM's registered security providers supply no MD5 implementation.

Common situations: Hardened/FIPS-mode JVM images with MD5 disabled; a customized java.security file that dropped the default providers; a corrupted JDK installation.

Related errors

• Unexpected HAServiceStateProto:
• Unrecognized section {}
• Image file is not found in {}
• Edits file is not found in {}
• Inconsistent storage detected, image and edits checkpoint ti

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/38300c6efba868a7. Report an issue: GitHub.