apache/hadoop · error · IllegalArgumentException

too many schemes? ${schemes.size()} when process scheme: ${s

Error message

too many schemes? ${schemes.size()} when process scheme: ${scheme}

What it means

FileSystemCounterGroup interns every distinct filesystem scheme it sees in counter names (uppercased) and checkScheme enforces a sanity cap of MAX_NUM_SCHEMES = 100 (FileSystemCounterGroup.java:56). Exceeding it throws IllegalArgumentException("too many schemes?") - the guard exists because malformed names create a pseudo-scheme per distinct prefix and blow up memory/serialization.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/FileSystemCounterGroup.java:239

    Object[] counters = map.get(canonicalScheme);
    int ord = key.ordinal();
    if (counters == null) {
      counters = new Object[FileSystemCounter.values().length];
      map.put(canonicalScheme, counters);
      counters[ord] = newCounter(canonicalScheme, key);
    }
    else if (counters[ord] == null) {
      counters[ord] = newCounter(canonicalScheme, key);
    }
    return (C) counters[ord];
  }

  private String checkScheme(String scheme) {
    String fixed = StringUtils.toUpperCase(scheme);
    String interned = schemes.putIfAbsent(fixed, fixed);
    if (schemes.size() > MAX_NUM_SCHEMES) {
      // mistakes or abuses
      throw new IllegalArgumentException("too many schemes? "+ schemes.size() +
                                         " when process scheme: "+ scheme);
    }
    return interned == null ? fixed : interned;
  }

  /**
   * Abstract factory method to create a file system counter
   * @param scheme of the file system
   * @param key the enum of the file system counter
   * @return a new file system counter
   */
  protected abstract C newCounter(String scheme, FileSystemCounter key);

  @Override
  public synchronized int size() {
    int n = 0;
    if (map != null) {
      for (Object[] counters : map.values()) {

View on GitHub (pinned to 2add963021)

Solutions

1. Normalize fs counter names to a fixed, small set of real schemes (HDFS, FILE, S3A, ...) before adding.
2. Fix the name generator: dynamic segments belong after the '_', or in a separate generic counter group.
3. Route high-cardinality metrics to a metrics system (Metrics2/Dropwizard) instead of Hadoop counters.
4. Audit with counters.countCounters()/group names before submission.

Example fix

// before: every tenant looks like a new scheme
counters.getGroup("File System Counters")
    .findCounter("TENANT" + tenantId + "_BYTES_READ").increment(n); // explodes past 100 schemes

// after: fixed scheme, dynamic part in a generic group
counters.getGroup("File System Counters").findCounter("HDFS_BYTES_READ").increment(n);
counters.findCounter("tenant-bytes", "TENANT" + tenantId).increment(n);

// before adding fs counters, keep your own scheme vocabulary bounded
static final Set<String> ALLOWED_SCHEMES = Set.of("HDFS", "FILE", "S3A");
String scheme = name.substring(0, name.indexOf('_'));
if (!ALLOWED_SCHEMES.contains(scheme)) {
  // route to a generic group instead of creating a pseudo-scheme
  counters.findCounter("fs-other", name);
}

catch (IllegalArgumentException e) around counter merge: on "too many schemes" log the offending scheme name and quarantine the source feeding pseudo-schemes; do not retry unchanged.

When it happens

Trigger: More than 100 distinct scheme prefixes flowing through fs counter names - e.g. a generator emitting names like "TENANT42_BYTES_READ", "JOB7_BYTES_READ", each counted as a new scheme; merging counters from many synthetic filesystems into one Counters object.

Common situations: User/lib code fabricating dynamic first-segment names in the fs group instead of a fixed scheme vocabulary; counters aggregation across many jobs/tenants into one object; frameworks registering a filesystem per workload with unique scheme names.

Related errors

• bad fs counter name
• bad framework group id: ${id}
• bad framework group name: ${name}
• Counters are enabled, Reporter cannot be NULL
• Counters version mismatch, expected ${groupFactory.version()

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/520fc2ff0be26a76. Report an issue: GitHub.