apache/hadoop · error · IOException

Unexpected counter group type: ${groupType}

Error message

Unexpected counter group type: ${groupType}

What it means

While deserializing the framework counter groups, AbstractCounters.readFields switches over the GroupType enum (FILESYSTEM, FRAMEWORK). The 'default' branch throwing IOException("Unexpected counter group type") is a defensive guard for an enum constant the switch does not handle; with the current two-constant enum it is effectively unreachable because the group type is read as an in-range index into GroupType.values(). It fires only when the enum and the switch fall out of sync (a fork adds a GroupType without a case) or the stream is corrupted/misaligned.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/AbstractCounters.java:305

    if (version != groupFactory.version()) {
      throw new IOException("Counters version mismatch, expected "+
          groupFactory.version() +" got "+ version);
    }
    int numFGroups = WritableUtils.readVInt(in);
    fgroups.clear();
    GroupType[] groupTypes = GroupType.values();
    while (numFGroups-- > 0) {
      GroupType groupType = groupTypes[WritableUtils.readVInt(in)];
      G group;
      switch (groupType) {
        case FILESYSTEM: // with nothing
          group = groupFactory.newFileSystemGroup();
          break;
        case FRAMEWORK:  // with group id
          group = groupFactory.newFrameworkGroup(WritableUtils.readVInt(in));
          break;
        default: // Silence dumb compiler, as it would've thrown earlier
          throw new IOException("Unexpected counter group type: "+ groupType);
      }
      group.readFields(in);
      fgroups.put(group.getName(), group);
    }
    int numGroups = WritableUtils.readVInt(in);
    while (numGroups-- > 0) {
      limits.checkGroups(groups.size() + 1);
      G group = groupFactory.newGenericGroup(
          StringInterner.weakIntern(Text.readString(in)), null, limits);
      group.readFields(in);
      groups.put(group.getName(), group);
    }
  }

  /**
   * Return textual representation of the counter values.
   * @return the string
   */

View on GitHub (pinned to 2add963021)

Solutions

1. If you maintain a Hadoop fork, handle every GroupType constant in the readFields switch (map the new constant to a factory method).
2. Rebuild both producer and consumer from the same source so the enum and switch stay in sync.
3. Treat the counters as lost on corruption: discard the stream and rebuild counters from task reports instead of replaying bytes.

Example fix

// fork patch pattern: keep switch exhaustive over GroupType
switch (groupType) {
  case FILESYSTEM:
    group = groupFactory.newFileSystemGroup();
    break;
  case FRAMEWORK:
    group = groupFactory.newFrameworkGroup(WritableUtils.readVInt(in));
    break;
  case MY_NEW_TYPE: // added constant must be handled here too
    group = groupFactory.newMyNewGroup(WritableUtils.readVInt(in));
    break;
  default:
    throw new IOException("Unexpected counter group type: " + groupType);
}

catch (IOException e) when deserializing counters: log the stream source, fall back to empty Counters, and continue - the payload is unreadable and retrying the same bytes cannot succeed.

When it happens

Trigger: Deserializing a counters stream written by a patched Hadoop that added a GroupType constant not handled by this switch; or a corrupted/misaligned stream that decodes to an unhandled switch path. Stock Hadoop cannot produce this from a well-formed stream.

Common situations: Forked Hadoop builds that extend the counters framework; counters payloads corrupted in transit or truncated then misread; cross-version deserialization after the enum layout changed.

Related errors

• Counters version mismatch, expected ${groupFactory.version()
• bad framework group id: ${id}
• Unknown mode: ${mode}
• Counters are enabled, Reporter cannot be NULL
• bad framework group name: ${name}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/fbd0821ff0c3f943. Report an issue: GitHub.