apache/druid · error · IllegalStateException

Unused segment[%s] has version[%s] > task version[%s]

Error message

Unused segment[%s] has version[%s] > task version[%s]

What it means

Thrown by ArchiveTask.runTask after retrieving unused segments: a segment in the target interval has a version newer than the task's lock version. Archiving such segments would hide data written by a more recent (shadowing) writer, so Druid aborts to avoid archiving segments that newer versions are supposed to supersede.

Source

Thrown at indexing-service/src/main/java/org/apache/druid/indexing/common/task/ArchiveTask.java:88

  public Set<ResourceAction> getInputSourceResources()
  {
    return ImmutableSet.of();
  }

  @Override
  public TaskStatus runTask(TaskToolbox toolbox) throws Exception
  {
    final TaskLock myLock = getAndCheckLock(toolbox);

    // List unused segments
    final List<DataSegment> unusedSegments = toolbox
        .getTaskActionClient()
        .submit(new RetrieveUnusedSegmentsAction(myLock.getDataSource(), myLock.getInterval(), null, null, null));

    // Verify none of these segments have versions > lock version
    for (final DataSegment unusedSegment : unusedSegments) {
      if (unusedSegment.getVersion().compareTo(myLock.getVersion()) > 0) {
        throw new ISE(
            "Unused segment[%s] has version[%s] > task version[%s]",
            unusedSegment.getId(),
            unusedSegment.getVersion(),
            myLock.getVersion()
        );
      }

      log.info("OK to archive segment: %s", unusedSegment.getId());
    }

    // Move segments
    for (DataSegment segment : unusedSegments) {
      final DataSegment archivedSegment = toolbox.getDataSegmentArchiver().archive(segment);
      if (archivedSegment != null) {
        toolbox.getTaskActionClient().submit(new SegmentMetadataUpdateAction(ImmutableSet.of(archivedSegment)));
      } else {
        log.info("No action was taken for [%s]", segment.getId());
      }

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Retry the archive task so it acquires a fresh, current-version lock after newer writes finish.
2. Exclude or first kill the higher-version segments, then archive the remaining ones.
3. Set the archive task's version equal to or greater than the max segment version in the interval.
4. Inspect segment versions (sys.segments or coordinator API) for the interval before launching the archive task.

Example fix

// before: fixed old version
ArchiveTask task = new ArchiveTask(id, ds, new Interval("2023/2024"), null, "old-version");
// after: bump/retry so lock version >= segment versions
ArchiveTask task = new ArchiveTask(id, ds, new Interval("2023/2024"), null, null); // derive from current lock

final List<DataSegment> unused = client.submit(new RetrieveUnusedSegmentsAction(ds, interval, null, null, null));
final String maxVersion = unused.stream().map(s -> s.getVersion()).max(Comparator.naturalOrder()).orElse("");
if (lockVersion.compareTo(maxVersion) < 0) { /* postpone or raise task version */ }

try { archiveTask.run(...); } catch (ISE e) { if (e.getMessage().contains("has version")) { retry after newer tasks complete; } else throw e; }

When it happens

Trigger: Running an archive (or kill-style) task over an interval where a higher-version segment was written after the archive task acquired its lock — e.g., overlapping re-ingestion with a later version, or a manually chosen task version older than existing unused segments.

Common situations: Re-running failed compaction/replacement tasks that leave behind mixed-version unused segments; hand-crafted archive tasks with an explicit version string; race between a replacement task committing and an archive task scanning the interval.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• Could not create group mapping [%s] due to concurrent update
• Could not delete group mapping [%s] due to concurrent update
• Could not create role [%s] due to concurrent update contenti
• Could not delete role [%s] due to concurrent update contenti
• Could not assign role [%s] to user [%s] due to concurrent up

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/9da53903f5553d08. Report an issue: GitHub.