apache/hadoop · error · HadoopIllegalArgumentException

"Number of data and parity blocks in an EC group " + blocksI

Error message

"Number of data and parity blocks in an EC group " + blocksInGroup + " should not exceed maximum " + HdfsServerConstants.MAX_BLOCKS_IN_GROUP

What it means

Error ""Number of data and parity blocks in an EC group " + blocksInGroup + " should not exceed maximum " + HdfsServerConstants.MAX_BLOCKS_IN_GROUP" thrown in apache/hadoop.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ErasureCodingPolicyManager.java:310

  /**
   * Add an erasure coding policy.
   * @return the added policy
   */
  public synchronized ErasureCodingPolicy addPolicy(
      ErasureCodingPolicy policy) {
    if (!userDefinedAllowed) {
      throw new HadoopIllegalArgumentException(
          "Addition of user defined erasure coding policy is disabled.");
    }

    if (!CodecUtil.hasCodec(policy.getCodecName())) {
      throw new HadoopIllegalArgumentException("Codec name "
          + policy.getCodecName() + " is not supported");
    }

    int blocksInGroup = policy.getNumDataUnits() + policy.getNumParityUnits();
    if (blocksInGroup > HdfsServerConstants.MAX_BLOCKS_IN_GROUP) {
      throw new HadoopIllegalArgumentException("Number of data and parity blocks in an EC group " +
          blocksInGroup + " should not exceed maximum " + HdfsServerConstants.MAX_BLOCKS_IN_GROUP);
    }

    if (policy.getCellSize() > maxCellSize) {
      throw new HadoopIllegalArgumentException("Cell size " +
          policy.getCellSize() + " should not exceed maximum " +
          maxCellSize + " bytes");
    }

    String assignedNewName = ErasureCodingPolicy.composePolicyName(
        policy.getSchema(), policy.getCellSize());
    for (ErasureCodingPolicyInfo info : getPolicies()) {
      final ErasureCodingPolicy p = info.getPolicy();
      if (p.getName().equals(assignedNewName)) {
        LOG.info("The policy name " + assignedNewName + " already exists");
        return p;
      }
      if (p.getSchema().equals(policy.getSchema()) &&

View on GitHub (pinned to 2add963021)

Solutions

1. Choose an EC schema whose data+parity block count does not exceed HdfsServerConstants.MAX_BLOCKS_IN_GROUP.
2. Reduce the number of data or parity blocks in the policy being added.

When it happens

Trigger: Adding a custom erasure coding policy whose total number of data plus parity blocks exceeds HdfsServerConstants.MAX_BLOCKS_IN_GROUP.

Common situations: See trigger scenarios.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/fbb7c7e19fd330f0. Report an issue: GitHub.