{"record":{"id":"10665a62307e3c76","repo":"apache/druid","slug":"mismatched-shardspecs-in-interval-s-for-segments","errorCode":null,"errorMessage":"Mismatched shardSpecs in interval[%s] for segments[%s]","messagePattern":"Mismatched shardSpecs in interval\\[(.+?)\\] for segments\\[(.+?)\\]","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"server/src/main/java/org/apache/druid/segment/realtime/appenderator/SegmentPublisherHelper.java","lineNumber":68,"sourceCode":"   * - When segment lock is used, the overwriting task should set the proper size of the atomic update group.\n   *   See {@link #annotateAtomicUpdateGroupFn}.\n   */\n  static Set<DataSegment> annotateShardSpec(Set<DataSegment> segments)\n  {\n    final Map<Interval, List<DataSegment>> intervalToSegments = new HashMap<>();\n    segments.forEach(\n        segment -> intervalToSegments.computeIfAbsent(segment.getInterval(), k -> new ArrayList<>()).add(segment)\n    );\n\n    for (Entry<Interval, List<DataSegment>> entry : intervalToSegments.entrySet()) {\n      final Interval interval = entry.getKey();\n      final List<DataSegment> segmentsPerInterval = entry.getValue();\n      final ShardSpec firstShardSpec = segmentsPerInterval.get(0).getShardSpec();\n      final boolean anyMismatch = segmentsPerInterval.stream().anyMatch(\n          segment -> segment.getShardSpec().getClass() != firstShardSpec.getClass()\n      );\n      if (anyMismatch) {\n        throw new ISE(\n            \"Mismatched shardSpecs in interval[%s] for segments[%s]\",\n            interval,\n            segmentsPerInterval\n        );\n      }\n      final Function<DataSegment, DataSegment> annotateFn;\n      if (firstShardSpec instanceof OverwriteShardSpec) {\n        annotateFn = annotateAtomicUpdateGroupFn(segmentsPerInterval.size());\n      } else if (firstShardSpec instanceof BuildingShardSpec) {\n        // sanity check\n        // BuildingShardSpec is used in non-appending mode. In this mode,\n        // the segments in each interval should have contiguous partitionIds,\n        // so that they can be queryable (see PartitionHolder.isComplete()).\n        int expectedCorePartitionSetSize = segmentsPerInterval.size();\n        int actualCorePartitionSetSize = Math.toIntExact(\n            segmentsPerInterval\n                .stream()\n                .filter(segment -> segment.getShardSpec().getPartitionNum() < expectedCorePartitionSetSize)","sourceCodeStart":50,"sourceCodeEnd":86,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/server/src/main/java/org/apache/druid/segment/realtime/appenderator/SegmentPublisherHelper.java#L50-L86","documentation":"SegmentPublisherHelper.annotateShardSpec groups segments per interval and computes shard-spec annotations for publication. If the segments in one interval do not all use the same ShardSpec class, the helper cannot derive a consistent annotation and throws this ISE, since mixed shard-spec types within an interval indicate a corrupted or inconsistent segment set.","triggerScenarios":"Publishing segments where an interval contains segments with different ShardSpec implementations (e.g. some NumberedShardSpec and some HashBasedNumberedShardSpec) — typically after a failed/aborted partial publish mixed with a re-generated set.","commonSituations":"Corrupted or hand-edited segment metadata in metadata storage; partially published segments left from a previous failed run with different partitioning; task retries reusing stale segment identifiers with new partitioning config.","solutions":["Kill and re-run the affected ingestion so all segments for the interval are generated with the same partitioning scheme","Clean up orphaned/partial segments for the interval (e.g. via kill tasks or metadata cleanup) before republishing","Ensure partitioning configuration (partitionsSpec) is not changed between retries of the same interval"],"exampleFix":"// before\n// republishing with changed partitionsSpec over leftover segments -> mixed shard specs\n// after\n// drop existing segments for the interval, then re-run ingestion with consistent partitionsSpec\n\"partitionsSpec\": {\"type\": \"dynamic\"} // consistent across all retries","handlingStrategy":"retry","validationCode":"Map<String, Set<String>> byInterval = segments.values().stream().collect(Collectors.groupingBy(s -> s.getInterval().toString(), Collectors.mapping(s -> s.getShardSpec().getClass().getName(), Collectors.toSet())));\nif (byInterval.values().stream().anyMatch(s -> s.size() > 1)) { /* abort and re-run */ }","typeGuard":null,"tryCatchPattern":"try { publish(); } catch (ISE e) { log.error(\"mixed shard specs; kill segments and re-run task\", e); killSegmentsForInterval(interval); retryIngestion(); }","preventionTips":["Keep partitionsSpec stable across task retries","Clean up orphaned segments after failed publishes","Avoid manual edits to segment metadata tables"],"tags":["java","segments","publishing"],"backgroundTag":"schema-validation-failed","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}