apache/hadoop · error · IOException

Unexpected configuration parameters: dfs.replication.max = {

Error message

Unexpected configuration parameters: dfs.replication.max = {maxR} > 32767

What it means

BlockManager.initMaxReplication validates dfs.replication.max during NameNode startup. Replication factors are stored in a 16-bit signed field, so the maximum legal value is Short.MAX_VALUE = 32767; anything larger throws IOException 'Unexpected configuration parameters: dfs.replication.max = N > 32767' and the NameNode does not start.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java:642

  }

  private int initMinReplication(Configuration conf) throws IOException {
    final int minR = conf.getInt(
        DFSConfigKeys.DFS_NAMENODE_REPLICATION_MIN_KEY,
        DFSConfigKeys.DFS_NAMENODE_REPLICATION_MIN_DEFAULT);
    if (minR <= 0) {
      throw new IOException("Unexpected configuration parameters: "
          + DFSConfigKeys.DFS_NAMENODE_REPLICATION_MIN_KEY
          + " = " + minR + " <= 0");
    }
    return minR;
  }

  private int initMaxReplication(Configuration conf) throws IOException {
    final int maxR = conf.getInt(DFSConfigKeys.DFS_REPLICATION_MAX_KEY,
        DFSConfigKeys.DFS_REPLICATION_MAX_DEFAULT);
    if (maxR > Short.MAX_VALUE) {
      throw new IOException("Unexpected configuration parameters: "
          + DFSConfigKeys.DFS_REPLICATION_MAX_KEY
          + " = " + maxR + " > " + Short.MAX_VALUE);
    }
    if (minReplication > maxR) {
      throw new IOException("Unexpected configuration parameters: "
          + DFSConfigKeys.DFS_NAMENODE_REPLICATION_MIN_KEY
          + " = " + minReplication + " > "
          + DFSConfigKeys.DFS_REPLICATION_MAX_KEY
          + " = " + maxR);
    }
    return maxR;
  }

  private int initMinReplicationToBeInMaintenance(Configuration conf)
      throws IOException {
    final int minMaintenanceR = conf.getInt(
        DFSConfigKeys.DFS_NAMENODE_MAINTENANCE_REPLICATION_MIN_KEY,
        DFSConfigKeys.DFS_NAMENODE_MAINTENANCE_REPLICATION_MIN_DEFAULT);

View on GitHub (pinned to 2add963021)

Solutions

Set dfs.replication.max to 32767 or lower (values in the low hundreds are typical for real clusters)
Restart the NameNode

Example fix

<!-- before -->
<property><name>dfs.replication.max</name><value>1000000</value></property>

<!-- after -->
<property><name>dfs.replication.max</name><value>32767</value></property>

Defensive patterns

Strategy: validation

Validate before calling

int maxR = conf.getInt(DFSConfigKeys.DFS_REPLICATION_MAX_KEY, DFSConfigKeys.DFS_REPLICATION_MAX_DEFAULT);
if (maxR > Short.MAX_VALUE) throw new IOException("dfs.replication.max must be <= 32767, got " + maxR);

Try / catch

catch (IOException e) { if (e.getMessage().contains("dfs.replication.max")) failDeployment(e); else throw e; }

Prevention

Cap dfs.replication.max at a realistic ceiling (hundreds), never near 32767
Add a config lint rule rejecting replication keys above Short.MAX_VALUE

When it happens

Trigger: hdfs-site.xml sets dfs.replication.max above 32767 (e.g. 100000 or 2147483647), then the NameNode process is started.

Common situations: Someone 'removing the cap' by setting a huge number; global find-replace across config templates bumping replication values; confusion between max replication and an ID or port field.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/44c2ddc6c7f0afc1. Report an issue: GitHub.