apache/hadoop · error · HadoopIllegalArgumentException

"Adding erasure coding policy failed because the number of p

Error message

"Adding erasure coding policy failed because the number of policies stored in the system already reached the threshold, which is " + ErasureCodeConstants.MAX_POLICY_ID

What it means

Error ""Adding erasure coding policy failed because the number of policies stored in the system already reached the threshold, which is " + ErasureCodeConstants.MAX_POLICY_ID" thrown in apache/hadoop.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ErasureCodingPolicyManager.java:338

    String assignedNewName = ErasureCodingPolicy.composePolicyName(
        policy.getSchema(), policy.getCellSize());
    for (ErasureCodingPolicyInfo info : getPolicies()) {
      final ErasureCodingPolicy p = info.getPolicy();
      if (p.getName().equals(assignedNewName)) {
        LOG.info("The policy name " + assignedNewName + " already exists");
        return p;
      }
      if (p.getSchema().equals(policy.getSchema()) &&
          p.getCellSize() == policy.getCellSize()) {
        LOG.info("A policy with same schema "
            + policy.getSchema().toString() + " and cell size "
            + p.getCellSize() + " already exists");
        return p;
      }
    }

    if (getCurrentMaxPolicyID() == ErasureCodeConstants.MAX_POLICY_ID) {
      throw new HadoopIllegalArgumentException("Adding erasure coding " +
          "policy failed because the number of policies stored in the " +
          "system already reached the threshold, which is " +
          ErasureCodeConstants.MAX_POLICY_ID);
    }

    policy = new ErasureCodingPolicy(assignedNewName, policy.getSchema(),
        policy.getCellSize(), getNextAvailablePolicyID());
    final ErasureCodingPolicyInfo pi = new ErasureCodingPolicyInfo(policy);
    this.policiesByName.put(policy.getName(), pi);
    this.policiesByID.put(policy.getId(), pi);
    allPolicies =
        policiesByName.values().toArray(new ErasureCodingPolicyInfo[0]);
    allPersistedPolicies.put(policy.getId(),
        new ErasureCodingPolicyInfo(policy));
    LOG.info("Added erasure coding policy " + policy);
    return policy;
  }

View on GitHub (pinned to 2add963021)

Solutions

1. Remove unused user-defined EC policies before adding new ones.
2. Reuse an existing policy instead of creating a new one; the number of policies is bounded by ErasureCodeConstants.MAX_POLICY_ID.

When it happens

Trigger: Adding a new erasure coding policy when the number of user-defined policies already reached ErasureCodeConstants.MAX_POLICY_ID, leaving no free policy IDs.

Common situations: See trigger scenarios.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/421ab69977145fdf. Report an issue: GitHub.