apache/hadoop · error · IllegalArgumentException

bad fs counter name

Error message

bad fs counter name

What it means

FileSystemCounterGroup stores the per-scheme filesystem counters ("File System Counters"). Names are encoded as SCHEME_KEY, e.g. HDFS_BYTES_READ or FILE_BYTES_READ, and parseCounterName splits on the first '_'. When a counter name passed to findCounter/addCounter contains no underscore, no scheme/key split is possible and IllegalArgumentException("bad fs counter name") is thrown.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/FileSystemCounterGroup.java:186

    if (ours != null) {
      ours.setValue(counter.getValue());
    }
  }

  @Override
  public C addCounter(String name, String displayName, long value) {
    C counter = findCounter(name);
    if (counter != null) {
      counter.setValue(value);
    }
    return counter;
  }

  // Parse generic counter name into [scheme, key]
  private String[] parseCounterName(String counterName) {
    int schemeEnd = counterName.indexOf('_');
    if (schemeEnd < 0) {
      throw new IllegalArgumentException("bad fs counter name");
    }
    return new String[]{counterName.substring(0, schemeEnd),
                        counterName.substring(schemeEnd + 1)};
  }

  @Override
  public C findCounter(String counterName, String displayName) {
    return findCounter(counterName);
  }

  @Override
  public C findCounter(String counterName, boolean create) {
    try {
      String[] pair = parseCounterName(counterName);
      return findCounter(pair[0], FileSystemCounter.valueOf(pair[1]));
    }
    catch (Exception e) {
      if (create) throw new IllegalArgumentException(e);

View on GitHub (pinned to 2add963021)

Solutions

1. Use canonical SCHEME_KEY names: uppercase filesystem scheme + '_' + key, e.g. "HDFS_BYTES_READ".
2. Put user metrics in their own group via counters.findCounter("mygroup", "BYTES_READ") instead of the fs group.
3. Let the framework populate filesystem counters itself; do not hand-write into that group.
4. Sanitize third-party counter names at merge time: names without '_' go to a generic group.

Example fix

// before
Counter c = counters.getGroup("File System Counters").findCounter("BYTES_READ");
// IllegalArgumentException: bad fs counter name

// after
Counter c = counters.getGroup("File System Counters").findCounter("HDFS_BYTES_READ");
// or keep user metrics in their own group:
Counter mine = counters.findCounter("myapp", "BYTES_READ");

static boolean isValidFsCounterName(String name) {
  int i = name.indexOf('_');
  return i > 0 && i < name.length() - 1; // SCHEME_KEY with non-empty scheme and key
}

if (isValidFsCounterName(name)) { fsGroup.findCounter(name); }
else { counters.findCounter("mygroup", name); }

When it happens

Trigger: Calling findCounter(name) or addCounter(name, displayName, value) on the file system counter group with a name lacking '_', such as "BYTES_READ" instead of "HDFS_BYTES_READ"; merging user-invented names into the fs group; addCounter(String, String, long) wiring user counters through this group.

Common situations: User code writing its own counters directly into the "File System Counters" group; libraries that emit scheme-less filesystem counter names; counters aggregation from non-Hadoop sources that drops the scheme prefix.

Related errors

• too many schemes? ${schemes.size()} when process scheme: ${s
• bad framework group id: ${id}
• bad framework group name: ${name}
• Name cannot be have a '{ch}' char
• Name cannot be NULL or emtpy

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/9a82be6890c19585. Report an issue: GitHub.