apache/hadoop · error · IndexOutOfBoundsException

adding an additional class would exceed the maximum number…

Error message

adding an additional class would exceed the maximum number allowed

What it means

Dynamically registered classes (those not pre-assigned a fixed id) get consecutive positive byte ids via ++newClasses. Once the count would exceed Byte.MAX_VALUE (127), addToMap(clazz) throws IndexOutOfBoundsException - the wire format only has one byte for the id, so a MapWritable cannot carry more than 127 distinct dynamic types.

Solutions

1. Reduce distinct value types: encode values as Text/BytesWritable instead of one Writable class per type
2. Pre-register your types with fixed negative ids (outside -127..-113) if you control the subclass, since predefined ids do not count against the 127
3. Switch containers: ObjectWritable, a custom Writable with an explicit type tag, or a serialization framework without the byte-id cap

Example fix

// before: each domain type eats one dynamic id -> cap at 127
map.put(new Text("user"), new UserWritable(...));
map.put(new Text("order"), new OrderWritable(...));

// after: normalize values to Text, no per-type registration
map.put(new Text("user"), new Text(jsonUser));
map.put(new Text("order"), new Text(jsonOrder));

Set<Class<?>> distinct = map.keySet().stream()
    .map(Object::getClass).collect(Collectors.toSet());
// plus value types actually stored; if distinct types approach 127, normalize instead:
if (distinct.size() > 100) {
  throw new IllegalStateException("MapWritable nearing 127-class cap; encode values as Text");
}

try {
  map.put(key, newValue);
} catch (IndexOutOfBoundsException e) {
  if (e.getMessage().contains("maximum number allowed")) {
    // no more dynamic ids: serialize values as Text/BytesWritable instead
  }
}

When it happens

Trigger: Using MapWritable/SortedMapWritable as a heterogeneous bag and putting values of more than 127 distinct Writable classes into one instance (each new class consumes one id); long-lived accumulators that keep adding new value types.

Common situations: Generic ETL code that boxes many domain types into a MapWritable; migrating maps of many types into Hadoop serialization without pre-registering.

Related errors

• Class already registered but maps to and not
• Id exists but maps to and not
• map cannot be copied
• source map cannot be null
• Component type is set as the output type, but no encoding…

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/c3a2eaa073a7eed4. Report an issue: GitHub.

Appendix: source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/AbstractMapWritable.java:96

      if (!c.equals(clazz)) {
        throw new IllegalArgumentException("Id " + id + " exists but maps to " +
            c.getName() + " and not " + clazz.getName());
      }
    }
    classToIdMap.put(clazz, id);
    idToClassMap.put(id, clazz);
  }
  
  /**
   * Add a Class to the maps if it is not already present.
   * @param clazz clazz.
   */
  protected synchronized void addToMap(Class<?> clazz) {
    if (classToIdMap.containsKey(clazz)) {
      return;
    }
    if (newClasses + 1 > Byte.MAX_VALUE) {
      throw new IndexOutOfBoundsException("adding an additional class would" +
      " exceed the maximum number allowed");
    }
    byte id = ++newClasses;
    addToMap(clazz, id);
  }

  /**
   * the Class class for the specified id.
   * @param id id.
   * @return the Class class for the specified id.
   */
  protected Class<?> getClass(byte id) {
    return idToClassMap.get(id);
  }

  /**
   * get id.
   * @return the id for the specified Class.

View on GitHub (pinned to 2add963021)