apache/hadoop · error · IOException

Specified block size ({}) is less than the cell size ({}) of

Error message

Specified block size ({}) is less than the cell size ({}) of the erasure coding policy ({}).

What it means

For erasure-coded files each write unit must cover at least one full cell so all data and parity shards receive payload. When the effective EC policy is not a replication policy, startFile rejects blockSize < ecPolicy.getCellSize() with IOException naming the block size, the cell size, and the policy (e.g., 1MB cells for RS-6-3-1024k).

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java:2846

      iip = FSDirWriteFileOp.resolvePathForStartFile(
          dir, pc, src, flag, createParent);

      if (blockSize < minBlockSize) {
        throw new IOException("Specified block size " + blockSize +
            " is less than configured minimum value " +
            DFSConfigKeys.DFS_NAMENODE_MIN_BLOCK_SIZE_KEY + "=" + minBlockSize);
      }

      if (shouldReplicate) {
        blockManager.verifyReplication(src, replication, clientMachine);
      } else {
        final ErasureCodingPolicy ecPolicy = FSDirErasureCodingOp
            .getErasureCodingPolicy(this, ecPolicyName, iip);
        if (ecPolicy != null && (!ecPolicy.isReplicationPolicy())) {
          checkErasureCodingSupported("createWithEC");
          if (blockSize < ecPolicy.getCellSize()) {
            throw new IOException("Specified block size (" + blockSize
                + ") is less than the cell size (" + ecPolicy.getCellSize()
                +") of the erasure coding policy (" + ecPolicy + ").");
          }
        } else {
          blockManager.verifyReplication(src, replication, clientMachine);
        }
      }

      FileEncryptionInfo feInfo = null;
      if (!iip.isRaw() && provider != null) {
        EncryptionKeyInfo ezInfo = FSDirEncryptionZoneOp.getEncryptionKeyInfo(
            this, iip, supportedVersions);
        // if the path has an encryption zone, the lock was released while
        // generating the EDEK.  re-resolve the path to ensure the namesystem
        // and/or EZ has not mutated
        if (ezInfo != null) {
          checkOperation(OperationCategory.WRITE);
          iip = FSDirWriteFileOp.resolvePathForStartFile(

View on GitHub (pinned to 2add963021)

Solutions

1. Use blockSize >= cell size of the effective EC policy (1MB for RS-6-3-1024k; legal cells 64KB-4MB)
2. If small blocks are a hard requirement, write the file replicated instead: non-EC directory or CreateFlag.SHOULD_REPLICATE

Example fix

// before: EC directory + small blocks
fs.create(new Path("/ec/out"), true, 4096, (short) 3, 512L * 1024);
// after
DistributedFileSystem dfs = (DistributedFileSystem) fs;
ErasureCodingPolicy ec = dfs.getErasureCodingPolicy(new Path("/ec"));
long cell = (ec != null && !ec.isReplicationPolicy()) ? ec.getCellSize() : 0L;
fs.create(new Path("/ec/out"), true, 4096, (short) 3, Math.max(512L * 1024, cell));

DistributedFileSystem dfs = (DistributedFileSystem) fs;
ErasureCodingPolicy p = dfs.getErasureCodingPolicy(path.getParent());
if (p != null && !p.isReplicationPolicy()) {
  long cell = p.getCellSize();
  if (blockSize < cell) blockSize = cell;
}

When it happens

Trigger: Creating a file that resolves to an EC policy (via ecPolicyName or the parent directory's policy) while passing a blockSize smaller than that policy's cell size — e.g., 512KB blocks under the default RS-6-3-1024k policy.

Common situations: Apps tuned for replicated HDFS (small blocks for small files) start writing into EC-enabled directories after tiering changes; the cluster's default EC policy was switched to a larger cell (e.g., 4MB); migration code carries over legacy block sizes.

Related errors

• Specified block size {} is less than configured minimum valu
• FileSystem ${item.fs.getUri()} does not support Erasure Codi
• Requested replication factor of {replication}{err} for {src}
• All negative block group IDs are used, growing into positive
• Unknown BlockChecksumType: " + groupChecksumType

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/7566f5d133fd8056. Report an issue: GitHub.