apache/druid · error · ISE

Unused segment[ ] has version[ ] > task version[ ]

Error message

Unused segment[%s] has version[%s] > task version[%s]

What it means

RestoreTask verifies that every unused segment it is about to restore within its lock interval has a segment version less than or equal to the version of the task's time-partition lock. A segment with a newer version would be overwritten by restoring older data, which would violate Druid's versioning-based concurrency model (newer data must never be shadowed by older restores). The task throws ISE to abort the restore rather than silently clobber newer segments.

Solutions

1. Inspect the unused segments (sys.segments table or coordinator API) and identify which have versions newer than the RestoreTask lock version; decide whether the restore is still intended.
2. If the newer unused segments are not needed, verify their creation history and re-run RestoreTask only for intervals where no newer-version unused segments exist.
3. If the restore is stale, drop/skip it — restoring would overwrite newer data; re-ingest the data instead to produce segments with a version newer than the conflicting ones.
4. Upgrade the lock version of the restore task (e.g. via a new task with a higher version / useLineageBasedSegmentAllocation or explicit version) so it dominates the unused segments' versions.

Example fix

// before
TaskLock myLock = ...; // old lock version
submitRestoreTask(lock); // ISE if unused segments have newer versions
// after
// first check versions via coordinator API, then restore only safe intervals
if (unusedSegments.stream().allMatch(s -> s.getVersion().compareTo(myLock.getVersion()) <= 0)) {
  submitRestoreTask(myLock);
}

List<DataSegment> unused = taskActionClient.submit(new RetrieveUnusedSegmentsAction(dataSource, interval, null, null, null));
boolean safe = unused.stream().allMatch(s -> s.getVersion().compareTo(lock.getVersion()) <= 0);
if (!safe) { throw new IllegalStateException("Restore aborted: newer-version unused segments present"); }

boolean isRestorable(DataSegment s, TaskLock lock) { return s.getVersion().compareTo(lock.getVersion()) <= 0; }

try {
  runRestoreTask();
} catch (IllegalStateException e) {
  if (e.getMessage().contains("has version")) {
    log.error("Conflicting newer unused segments; skipping restore", e);
  } else { throw e; }
}

When it happens

Trigger: Running RestoreTask on a datasource/interval where, after the segments were marked unused, another task (e.g. a compaction or replacement task) has written segments with a higher version into the same interval that were themselves later marked unused, so RetrieveUnusedSegmentsAction returns segments whose version exceeds the lock version held by RestoreTask.

Common situations: Compaction or re-indexing ran between the original mark-unused and the restore attempt; manual segment version bumps; replaying old restore tasks after other ingestion tasks upgraded the segment version in the same interval.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• Can't find overwritingSegmentMeta for interval
• Failed to get locks for intervals
• Found a different granularity from knownSegmentGranularity
• Lock interval[ ] != task interval[ ]
• Locks[ ] for task[ ] can't cover segments[ ]

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/904098a7eacfeaa4. Report an issue: GitHub.

Appendix: source

Thrown at indexing-service/src/main/java/org/apache/druid/indexing/common/task/RestoreTask.java:89

  public Set<ResourceAction> getInputSourceResources()
  {
    return ImmutableSet.of();
  }

  @Override
  public TaskStatus runTask(TaskToolbox toolbox) throws Exception
  {
    final TaskLock myLock = getAndCheckLock(toolbox);

    // List unused segments
    final List<DataSegment> unusedSegments = toolbox
        .getTaskActionClient()
        .submit(new RetrieveUnusedSegmentsAction(myLock.getDataSource(), myLock.getInterval(), null, null, null));

    // Verify none of these segments have versions > lock version
    for (final DataSegment unusedSegment : unusedSegments) {
      if (unusedSegment.getVersion().compareTo(myLock.getVersion()) > 0) {
        throw new ISE(
            "Unused segment[%s] has version[%s] > task version[%s]",
            unusedSegment.getId(),
            unusedSegment.getVersion(),
            myLock.getVersion()
        );
      }

      log.info("OK to restore segment: %s", unusedSegment.getId());
    }

    final List<DataSegment> restoredSegments = new ArrayList<>();

    // Move segments
    for (DataSegment segment : unusedSegments) {
      final DataSegment restored = toolbox.getDataSegmentArchiver().restore(segment);
      if (restored != null) {
        restoredSegments.add(restored);
      } else {

View on GitHub (pinned to 9b90983fd2)