apache/druid · error · ISE

Unused segment[%s] has version[%s] > task version[%s]

Error message

Unused segment[%s] has version[%s] > task version[%s]

What it means

MoveTask relocates segments between deep-storage locations, but it may only move segments whose version is <= the version of the task's lock (segments with newer versions must not be overwritten/moved by an older-generation task). If any retrieved unused segment has a higher version, the task aborts with ISE.

Source

Thrown at indexing-service/src/main/java/org/apache/druid/indexing/common/task/MoveTask.java:96

  public Set<ResourceAction> getInputSourceResources()
  {
    return ImmutableSet.of();
  }

  @Override
  public TaskStatus runTask(TaskToolbox toolbox) throws Exception
  {
    final TaskLock myLock = getAndCheckLock(toolbox);

    // List unused segments
    final List<DataSegment> unusedSegments = toolbox
        .getTaskActionClient()
        .submit(new RetrieveUnusedSegmentsAction(myLock.getDataSource(), myLock.getInterval(), null, null, null));

    // Verify none of these segments have versions > lock version
    for (final DataSegment unusedSegment : unusedSegments) {
      if (unusedSegment.getVersion().compareTo(myLock.getVersion()) > 0) {
        throw new ISE(
            "Unused segment[%s] has version[%s] > task version[%s]",
            unusedSegment.getId(),
            unusedSegment.getVersion(),
            myLock.getVersion()
        );
      }

      log.info("OK to move segment: %s", unusedSegment.getId());
    }

    // Move segments
    for (DataSegment segment : unusedSegments) {
      final DataSegment movedSegment = toolbox.getDataSegmentMover().move(segment, targetLoadSpec);
      toolbox.getTaskActionClient().submit(new SegmentMetadataUpdateAction(ImmutableSet.of(movedSegment)));
    }

    return TaskStatus.success(getId());
  }

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Re-submit the move task after the concurrent replace/compaction finishes so the lock acquires the latest version
2. Restrict the move interval to segments whose versions match the move task's lock version
3. Verify with metadata queries which segments have newer versions and either kill them first or align versions

Example fix

// before
// move task created before a compaction ran
// after
// resubmit the move task; it acquires a new lock version covering the newer segments

List<DataSegment> unused = retrieveUnused(dataSource, interval);
if (unused.stream().anyMatch(s -> s.getVersion().compareTo(lockVersion) > 0)) { throw new IllegalStateException("newer segments present; resubmit move later"); }

try { moveTask.runTask(actionClient); } catch (ISE e) { if (e.getMessage().contains("has version")) { resubmitAfterCompaction(); } else { throw e; } }

When it happens

Trigger: A move task holding a lock with version V encounters an unused segment in the same interval whose version is newer than V — typically because a later replace/compaction task wrote segments with a new version into the same interval before the move ran.

Common situations: Submitting a move task while concurrent compaction has replaced segments with newer versions; stale move spec created before a re-ingestion; time-versioned segments after manual inserts or speculative retries.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• Locks[%s] for task[%s] can't cover segments[%s]
• The lock for interval[%s] is preempted and no longer valid
• Some locks for task[%s] are already revoked
• Some locks for task[%s] are already revoked
• Lock interval[%s] != task interval[%s]

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/4c2b59b7124ac2c6. Report an issue: GitHub.