apache/druid · error · ISE

Unused segment[%s] has version[%s] > task version[%s]

Error message

Unused segment[%s] has version[%s] > task version[%s]

What it means

RestoreTask verifies that every unused segment it is about to restore within its lock interval has a segment version less than or equal to the version of the task's time-partition lock. A segment with a newer version would be overwritten by restoring older data, which would violate Druid's versioning-based concurrency model (newer data must never be shadowed by older restores). The task throws ISE to abort the restore rather than silently clobber newer segments.

Source

Thrown at indexing-service/src/main/java/org/apache/druid/indexing/common/task/RestoreTask.java:89

  public Set<ResourceAction> getInputSourceResources()
  {
    return ImmutableSet.of();
  }

  @Override
  public TaskStatus runTask(TaskToolbox toolbox) throws Exception
  {
    final TaskLock myLock = getAndCheckLock(toolbox);

    // List unused segments
    final List<DataSegment> unusedSegments = toolbox
        .getTaskActionClient()
        .submit(new RetrieveUnusedSegmentsAction(myLock.getDataSource(), myLock.getInterval(), null, null, null));

    // Verify none of these segments have versions > lock version
    for (final DataSegment unusedSegment : unusedSegments) {
      if (unusedSegment.getVersion().compareTo(myLock.getVersion()) > 0) {
        throw new ISE(
            "Unused segment[%s] has version[%s] > task version[%s]",
            unusedSegment.getId(),
            unusedSegment.getVersion(),
            myLock.getVersion()
        );
      }

      log.info("OK to restore segment: %s", unusedSegment.getId());
    }

    final List<DataSegment> restoredSegments = new ArrayList<>();

    // Move segments
    for (DataSegment segment : unusedSegments) {
      final DataSegment restored = toolbox.getDataSegmentArchiver().restore(segment);
      if (restored != null) {
        restoredSegments.add(restored);
      } else {

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Inspect the unused segments (sys.segments table or coordinator API) and identify which have versions newer than the RestoreTask lock version; decide whether the restore is still intended.
2. If the newer unused segments are not needed, verify their creation history and re-run RestoreTask only for intervals where no newer-version unused segments exist.
3. If the restore is stale, drop/skip it — restoring would overwrite newer data; re-ingest the data instead to produce segments with a version newer than the conflicting ones.
4. Upgrade the lock version of the restore task (e.g. via a new task with a higher version / useLineageBasedSegmentAllocation or explicit version) so it dominates the unused segments' versions.

Example fix

// before
TaskLock myLock = ...; // old lock version
submitRestoreTask(lock); // ISE if unused segments have newer versions
// after
// first check versions via coordinator API, then restore only safe intervals
if (unusedSegments.stream().allMatch(s -> s.getVersion().compareTo(myLock.getVersion()) <= 0)) {
  submitRestoreTask(myLock);
}

List<DataSegment> unused = taskActionClient.submit(new RetrieveUnusedSegmentsAction(dataSource, interval, null, null, null));
boolean safe = unused.stream().allMatch(s -> s.getVersion().compareTo(lock.getVersion()) <= 0);
if (!safe) { throw new IllegalStateException("Restore aborted: newer-version unused segments present"); }

boolean isRestorable(DataSegment s, TaskLock lock) { return s.getVersion().compareTo(lock.getVersion()) <= 0; }

try {
  runRestoreTask();
} catch (IllegalStateException e) {
  if (e.getMessage().contains("has version")) {
    log.error("Conflicting newer unused segments; skipping restore", e);
  } else { throw e; }
}

When it happens

Trigger: Running RestoreTask on a datasource/interval where, after the segments were marked unused, another task (e.g. a compaction or replacement task) has written segments with a higher version into the same interval that were themselves later marked unused, so RetrieveUnusedSegmentsAction returns segments whose version exceeds the lock version held by RestoreTask.

Common situations: Compaction or re-indexing ran between the original mark-unused and the restore attempt; manual segment version bumps; replaying old restore tasks after other ingestion tasks upgraded the segment version in the same interval.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• The lock for interval[%s] is preempted and no longer valid
• Some locks for task[%s] are already revoked
• Some locks for task[%s] are already revoked
• Lock interval[%s] != task interval[%s]
• Unused segment[%s] has version[%s] > task version[%s]

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/904098a7eacfeaa4. Report an issue: GitHub.