apache/hadoop · error · IOException

Specified block size {} is less than configured minimum valu

Error message

Specified block size {} is less than configured minimum value dfs.namenode.fs-limits.min-block-size={}

What it means

startFile enforces a floor on block size: dfs.namenode.fs-limits.min-block-size configured on the NameNode (default 1024 bytes; production clusters often raise it to 1MB). A create whose blockSize is below the floor is refused with IOException naming both the requested value and the configured key.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java:2833

    }

    INodesInPath iip = null;
    boolean skipSync = true; // until we do something that might create edits
    HdfsFileStatus stat = null;
    BlocksMapUpdateInfo toRemoveBlocks = null;

    checkOperation(OperationCategory.WRITE);
    final FSPermissionChecker pc = getPermissionChecker();
    writeLock(RwLockMode.FS);
    try {
      checkOperation(OperationCategory.WRITE);
      checkNameNodeSafeMode("Cannot create file" + src);

      iip = FSDirWriteFileOp.resolvePathForStartFile(
          dir, pc, src, flag, createParent);

      if (blockSize < minBlockSize) {
        throw new IOException("Specified block size " + blockSize +
            " is less than configured minimum value " +
            DFSConfigKeys.DFS_NAMENODE_MIN_BLOCK_SIZE_KEY + "=" + minBlockSize);
      }

      if (shouldReplicate) {
        blockManager.verifyReplication(src, replication, clientMachine);
      } else {
        final ErasureCodingPolicy ecPolicy = FSDirErasureCodingOp
            .getErasureCodingPolicy(this, ecPolicyName, iip);
        if (ecPolicy != null && (!ecPolicy.isReplicationPolicy())) {
          checkErasureCodingSupported("createWithEC");
          if (blockSize < ecPolicy.getCellSize()) {
            throw new IOException("Specified block size (" + blockSize
                + ") is less than the cell size (" + ecPolicy.getCellSize()
                +") of the erasure coding policy (" + ecPolicy + ").");
          }
        } else {
          blockManager.verifyReplication(src, replication, clientMachine);

View on GitHub (pinned to 2add963021)

Solutions

1. Raise the requested blockSize to at least the cluster's dfs.namenode.fs-limits.min-block-size (when unsure, default to 128MB or pass 0 to use the configured default)
2. Only if genuinely required, lower dfs.namenode.fs-limits.min-block-size on the NameNode and restart — accepting the NameNode memory cost of many tiny blocks
3. Mirror the NameNode's fs-limits in the client config so the check can run before the RPC

Example fix

// before
fs.create(path, true, 4096, (short) 3, 1024L);
// after
long minBlock = conf.getLong(DFSConfigKeys.DFS_NAMENODE_MIN_BLOCK_SIZE_KEY,
                             DFSConfigKeys.DFS_NAMENODE_MIN_BLOCK_SIZE_DEFAULT);
long blockSize = Math.max(1024L, minBlock);
fs.create(path, true, 4096, (short) 3, blockSize);

long minBlock = conf.getLong(DFSConfigKeys.DFS_NAMENODE_MIN_BLOCK_SIZE_KEY,
                             DFSConfigKeys.DFS_NAMENODE_MIN_BLOCK_SIZE_DEFAULT);
if (blockSize > 0 && blockSize < minBlock) blockSize = minBlock;

When it happens

Trigger: DFSClient.create(..., blockSize) with blockSize smaller than the NameNode's dfs.namenode.fs-limits.min-block-size — e.g., an app requests 512B/64KB while the cluster enforces 1MB.

Common situations: Small-file test tooling carrying tiny block sizes into a hardened cluster; distro/perf guidance raised min-block-size and legacy apps kept old values; client-side hdfs-site.xml diverges from the NameNode's fs-limits.

Related errors

• Requested replication factor of {replication}{err} for {src}
• Specified block size ({}) is less than the cell size ({}) of
• Internal error: default blockSize is not a multiple of defau
• Can not create a Path from a null string
• Can not create a Path from an empty string

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/f7596db3df88f05c. Report an issue: GitHub.