apache/druid · error · ISE

Failed to merge existing aggregators when generating metrics

Error message

Failed to merge existing aggregators when generating metricsSpec; try providing explicit metricsSpec

What it means

CompactionTask analyzes the aggregators already present in existing segments and tries to merge them into a single metricsSpec for the compacted segments. If the AggregatorFactory.merge() of the per-segment aggregator lists returns null — meaning the aggregators across segments are incompatible or unmergeable — this ISE is thrown. The library throws it because it cannot infer a correct metricsSpec automatically and needs the user to supply one.

Source

Thrown at indexing-service/src/main/java/org/apache/druid/indexing/common/task/CompactionTask.java:1122

    public AggregatorFactory[] getMetricsSpec()
    {
      if (!needMetricsSpec) {
        throw new ISE("Not computing metricsSpec");
      }

      if (aggregatorFactoryLists.isEmpty()) {
        return new AggregatorFactory[0];
      }

      final AggregatorFactory[] mergedAggregators = AggregatorFactory.mergeAggregators(
          aggregatorFactoryLists.stream()
                                .map(xs -> xs.toArray(new AggregatorFactory[0]))
                                .collect(Collectors.toList())
      );

      if (mergedAggregators == null) {
        throw new ISE(
            "Failed to merge existing aggregators when generating metricsSpec; "
            + "try providing explicit metricsSpec"
        );
      }

      return mergedAggregators;
    }

    @Nullable
    public List<AggregateProjectionSpec> getProjections()
    {
      if (!needProjections) {
        throw new ISE("Not computing projections");
      }
      if (projections == null || projections.isEmpty()) {
        return null;
      }
      return ImmutableList.copyOf(projections.values());

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Provide an explicit metricsSpec in the compaction spec (or EngineConfig/tuning config) so no merging is attempted
2. Align historical ingestion specs so all segments for the datasource use merge-compatible aggregators
3. Re-ingest the affected intervals so all segments share one consistent metricsSpec
4. Check which column's aggregators conflict by comparing metricsSpec of the segments in question

Example fix

// before
{ "dataSource": "wikiticker" } // no metricsSpec; auto-merge fails
// after
{
  "dataSource": "wikiticker",
  "metricsSpec": [
    { "type": "count", "name": "count" },
    { "type": "longSum", "name": "added", "fieldName": "added" }
  ]
}

// Compare metricsSpec of existing segments before compaction
List<String> specs = coordinatorClient.getSegmentMetadata(ds, interval)
    .stream().map(m -> jsonOf(m.getAggregators())).distinct().collect(toList());
if (specs.size() > 1) {
  compactionSpec.setMetricsSpec(predefinedMetricsSpec); // supply explicit metricsSpec
}

boolean hasMergeableAggregators(List<AggregatorFactory[]> lists) {
  return AggregatorFactory.mergeAggregators(lists) != null;
}

try {
  submitCompaction(spec);
} catch (IllegalStateException e) {
  if (e.getMessage().contains("Failed to merge existing aggregators")) {
    spec.setMetricsSpec(explicitMetricsSpec); // resubmit with explicit metricsSpec
    submitCompaction(spec);
  } else throw e;
}

When it happens

Trigger: Running an automatic compaction or CompactionTask without an explicit metricsSpec when existing segments contain aggregators that cannot be merged (e.g., segments written with different aggregator types or different aggregator orderings for the same column, or aggregators whose merge() returns null such as mismatched sketch types).

Common situations: Datasources ingested over time with evolving ingestion specs (aggregator changed from longSum to doubleSum, or a sketch aggregator replaced by another type); segments written by different ingestion jobs with inconsistent metrics; upgrading Druid after changing aggregator implementations.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• No segments found for compaction. Please check that datasour
• Invalid operation for AveragerFactoryWrapper.
• Must have a valid, non-null aggregator name
• Parameter fieldName must be specified
• Not implemented

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/4b1b82f8f7358815. Report an issue: GitHub.