What causes "Metrics source {} already exists!" in apache/hadoop?

Pass dupOK=true where exact duplicate names are acceptable (e.g., per-attempt records)

apache/hadoop · error · MetricsException

Metrics source {} already exists!

Error message

Metrics source {} already exists!

What it means

DefaultMetricsSystem.newSourceName(name, dupOK), reached via sourceName(name, dupOK), throws MetricsException('Metrics source <name> already exists!') when the name is already in sourceNames, dupOK is false, and miniClusterMode is off. Passing dupOK=true tolerates exact duplicates instead of throwing.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/DefaultMetricsSystem.java:152

    } catch (Exception e) {
      throw new MetricsException(e);
    }
  }

  synchronized void removeObjectName(String name) {
    mBeanNames.map.remove(name);
  }

  synchronized void removeSource(String name) {
    sourceNames.map.remove(name);
  }

  synchronized String newSourceName(String name, boolean dupOK) {
    if (sourceNames.map.containsKey(name)) {
      if (dupOK) {
        return name;
      } else if (!miniClusterMode) {
        throw new MetricsException("Metrics source "+ name +" already exists!");
      }
    }
    return sourceNames.uniqueName(name);
  }
}

View on GitHub (pinned to 2add963021)

Solutions

Pass dupOK=true where exact duplicate names are acceptable (e.g., per-attempt records)
Remove the previous name via DefaultMetricsSystem.removeSourceName(name) before re-creating it
Enable DefaultMetricsSystem.setMiniClusterMode(true) in tests so unique suffixed names are generated

Example fix

// before
String name = DefaultMetricsSystem.sourceName("MyRecord", false);  // throws if exists

// after
String name = DefaultMetricsSystem.sourceName("MyRecord", true);  // dup OK

Defensive patterns

Strategy: validation

Validate before calling

boolean duplicateOk = lifecycleAllowsReuse;  // per-attempt records: true
String name = DefaultMetricsSystem.sourceName("MyRecord", duplicateOk);

Try / catch

try {
  String name = DefaultMetricsSystem.sourceName("MyRecord", false);
} catch (MetricsException e) {
  DefaultMetricsSystem.removeSourceName("MyRecord");  // clear stale name and retry once
}

Prevention

Pass dupOK=true when exact duplicate record names are semantically acceptable
Remove source names on teardown of components that create them
In tests, set miniClusterMode so duplicate names get unique suffixes instead of throwing

When it happens

Trigger: Creating a metrics source/record name that already exists in this JVM with dupOK=false — e.g., initializing metrics twice for the same component name without removing the old one.

Common situations: Repeated init in unit tests; two subsystems that both chose the same source name; component restart inside a long-lived process.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/ff618026f7bca893. Report an issue: GitHub.