apache/druid · error · ISE

Can't compact segments of non-consecutive rootPartition rang

Error message

Can't compact segments of non-consecutive rootPartition range. Missing partitionIds between [%s] and [%s]

What it means

Thrown when, after sorting by rootPartitionId, two adjacent segments in the same interval neither share a rootPartitionRange nor have consecutive ranges: curSegment's endRootPartitionId is not nextSegment's startRootPartitionId minus gap-free continuity (they are not adjacent). Compaction/overwrite requires the input segments to cover a gap-free root-partition space so output segments can overshadow the entire range; holes mean some partitions are unaccounted for.

Source

Thrown at indexing-service/src/main/java/org/apache/druid/indexing/common/task/TaskLockHelper.java:289

    // sanity check
    for (int i = 0; i < sortedSegments.size() - 1; i++) {
      final DataSegment curSegment = sortedSegments.get(i);
      final DataSegment nextSegment = sortedSegments.get(i + 1);
      if (curSegment.getStartRootPartitionId() == nextSegment.getStartRootPartitionId()
          && curSegment.getEndRootPartitionId() == nextSegment.getEndRootPartitionId()) {
        // Input segments should have the same or consecutive rootPartition range
        if (curSegment.getMinorVersion() != nextSegment.getMinorVersion()
            || curSegment.getAtomicUpdateGroupSize() != nextSegment.getAtomicUpdateGroupSize()) {
          throw new ISE(
              "segment[%s] and segment[%s] have the same rootPartitionRange, but different minorVersion or atomicUpdateGroupSize",
              curSegment,
              nextSegment
          );
        }
        atomicUpdateGroupSize++;
      } else {
        if (curSegment.getEndRootPartitionId() != nextSegment.getStartRootPartitionId()) {
          throw new ISE(
              "Can't compact segments of non-consecutive rootPartition range. Missing partitionIds between [%s] and [%s]",
              curSegment.getEndRootPartitionId(),
              nextSegment.getStartRootPartitionId()
          );
        }
        if (atomicUpdateGroupSize != curSegment.getAtomicUpdateGroupSize()) {
          throw new ISE(
              "All atomicUpdateGroup must be compacted together. Expected size[%s] but current size[%s]",
              curSegment.getAtomicUpdateGroupSize(),
              atomicUpdateGroupSize
          );
        }
        atomicUpdateGroupSize = 1;
      }
    }
    if (atomicUpdateGroupSize != sortedSegments.get(sortedSegments.size() - 1).getAtomicUpdateGroupSize()) {
      throw new ISE(
          "All atomicUpdateGroup must be compacted together. Expected size[%s] but current size[%s]",

View on GitHub (pinned to 9b90983fd2)

Solutions

Include every used segment of the interval in the input list so the rootPartition ranges are consecutive with no gaps.
Check the segments table/segment metadata for missing partitionIds in the interval and restore them (unmark as overshadowed/tombstoned) or re-run ingestion to recreate them.
If partitions were intentionally dropped, run a full re-index (replace) of the interval instead of segment-level compaction.
Verify no filtering predicate (version, shardSpec, datasource version) is excluding segments before the helper call.

Example fix

// before: filtered subset leaves a partition hole
List<DataSegment> input = all.stream().filter(s -> !s.getVersion().equals(oldVersion)).collect(toList());
verifyRootPartitionIsAdjacentAndAtomicUpdateGroupIsFull(sort(input)); // ISE: gap

// after: use all used segments for the interval
List<DataSegment> input = timeline.getUsedSegmentsForInterval(ds, interval);
verifyRootPartitionIsAdjacentAndAtomicUpdateGroupIsFull(sort(input));

Defensive patterns

Strategy: validation

Validate before calling

// Java: ensure rootPartition ranges are gap-free before invoking compaction
List<DataSegment> sorted = segments.stream()
    .sorted(Comparator.comparingInt(DataSegment::getStartRootPartitionId))
    .collect(Collectors.toList());
for (int i = 0; i < sorted.size() - 1; i++) {
  if (sorted.get(i).getEndRootPartitionId() != sorted.get(i + 1).getStartRootPartitionId()
      && sorted.get(i).getStartRootPartitionId() != sorted.get(i + 1).getStartRootPartitionId()) {
    throw new IllegalStateException("Gap in rootPartition range between index " + i + " and " + (i + 1));
  }
}

Try / catch

try {
  TaskLockHelper.verifyRootPartitionIsAdjacentAndAtomicUpdateGroupIsFull(sorted);
} catch (IllegalStateException e) {
  if (e.getMessage().contains("non-consecutive rootPartition range")) {
    // fetch the full used-segment set for the interval and retry
  } else { throw e; }
}

Prevention

Always build compaction input from the full used-segment timeline for the interval, not from filtered lists.
Avoid dropping/tombstoning individual partitions before compaction.
After failed replaces, verify segment continuity with a segments-table query before compacting.

When it happens

Trigger: verifyAndLockExistingSegments -> tryLockSegments -> verifyAndFindRootPartitionRangeAndMinorVersion with a segment list for one interval where some root partitions are missing (e.g. segments [0,3) and [5,8) supplied but [3,5) omitted).

Common situations: Manually filtered segment lists (excluding segments by version or by metadata predicate) that accidentally drop partitions; killing/tombstoning some partitions before compaction; partial failures in prior replace operations; querying only a subset of used segments.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/aa95a9556d440379. Report an issue: GitHub.