{"record":{"id":"4b1b82f8f7358815","repo":"apache/druid","slug":"failed-to-merge-existing-aggregators-when-generati","errorCode":null,"errorMessage":"Failed to merge existing aggregators when generating metricsSpec; try providing explicit metricsSpec","messagePattern":"Failed to merge existing aggregators when generating metricsSpec; try providing explicit metricsSpec","errorType":"exception","errorClass":"ISE","httpStatus":null,"severity":"error","filePath":"indexing-service/src/main/java/org/apache/druid/indexing/common/task/CompactionTask.java","lineNumber":1122,"sourceCode":"\n    public AggregatorFactory[] getMetricsSpec()\n    {\n      if (!needMetricsSpec) {\n        throw new ISE(\"Not computing metricsSpec\");\n      }\n\n      if (aggregatorFactoryLists.isEmpty()) {\n        return new AggregatorFactory[0];\n      }\n\n      final AggregatorFactory[] mergedAggregators = AggregatorFactory.mergeAggregators(\n          aggregatorFactoryLists.stream()\n                                .map(xs -> xs.toArray(new AggregatorFactory[0]))\n                                .collect(Collectors.toList())\n      );\n\n      if (mergedAggregators == null) {\n        throw new ISE(\n            \"Failed to merge existing aggregators when generating metricsSpec; \"\n            + \"try providing explicit metricsSpec\"\n        );\n      }\n\n      return mergedAggregators;\n    }\n\n    @Nullable\n    public List<AggregateProjectionSpec> getProjections()\n    {\n      if (!needProjections) {\n        throw new ISE(\"Not computing projections\");\n      }\n      if (projections == null || projections.isEmpty()) {\n        return null;\n      }\n      return ImmutableList.copyOf(projections.values());","sourceCodeStart":1104,"sourceCodeEnd":1140,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/indexing-service/src/main/java/org/apache/druid/indexing/common/task/CompactionTask.java#L1104-L1140","documentation":"CompactionTask analyzes the aggregators already present in existing segments and tries to merge them into a single metricsSpec for the compacted segments. If the AggregatorFactory.merge() of the per-segment aggregator lists returns null — meaning the aggregators across segments are incompatible or unmergeable — this ISE is thrown. The library throws it because it cannot infer a correct metricsSpec automatically and needs the user to supply one.","triggerScenarios":"Running an automatic compaction or CompactionTask without an explicit metricsSpec when existing segments contain aggregators that cannot be merged (e.g., segments written with different aggregator types or different aggregator orderings for the same column, or aggregators whose merge() returns null such as mismatched sketch types).","commonSituations":"Datasources ingested over time with evolving ingestion specs (aggregator changed from longSum to doubleSum, or a sketch aggregator replaced by another type); segments written by different ingestion jobs with inconsistent metrics; upgrading Druid after changing aggregator implementations.","solutions":["Provide an explicit metricsSpec in the compaction spec (or EngineConfig/tuning config) so no merging is attempted","Align historical ingestion specs so all segments for the datasource use merge-compatible aggregators","Re-ingest the affected intervals so all segments share one consistent metricsSpec","Check which column's aggregators conflict by comparing metricsSpec of the segments in question"],"exampleFix":"// before\n{ \"dataSource\": \"wikiticker\" } // no metricsSpec; auto-merge fails\n// after\n{\n  \"dataSource\": \"wikiticker\",\n  \"metricsSpec\": [\n    { \"type\": \"count\", \"name\": \"count\" },\n    { \"type\": \"longSum\", \"name\": \"added\", \"fieldName\": \"added\" }\n  ]\n}","handlingStrategy":"validation","validationCode":"// Compare metricsSpec of existing segments before compaction\nList<String> specs = coordinatorClient.getSegmentMetadata(ds, interval)\n    .stream().map(m -> jsonOf(m.getAggregators())).distinct().collect(toList());\nif (specs.size() > 1) {\n  compactionSpec.setMetricsSpec(predefinedMetricsSpec); // supply explicit metricsSpec\n}","typeGuard":"boolean hasMergeableAggregators(List<AggregatorFactory[]> lists) {\n  return AggregatorFactory.mergeAggregators(lists) != null;\n}","tryCatchPattern":"try {\n  submitCompaction(spec);\n} catch (IllegalStateException e) {\n  if (e.getMessage().contains(\"Failed to merge existing aggregators\")) {\n    spec.setMetricsSpec(explicitMetricsSpec); // resubmit with explicit metricsSpec\n    submitCompaction(spec);\n  } else throw e;\n}","preventionTips":["Keep ingestion specs for a datasource consistent over time","Always set an explicit metricsSpec in compaction specs for heterogeneous datasources","Audit segment metadata aggregators periodically","Avoid switching sketch/aggregator types without re-ingesting"],"tags":["druid","compaction","aggregator","configuration"],"backgroundTag":"unsupported-operation","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}