apache/druid · error · IllegalStateException

Some locks for task[%s] are already revoked

Error message

Some locks for task[%s] are already revoked

What it means

SegmentMetadataUpdateAction.perform() wraps metadata updates in a CriticalAction whose onInvalidLocks callback fires when the task's locks have been revoked (preempted by another task) before the update runs. Since updating segment metadata under invalid locks would corrupt state, Druid throws this ISE. The calling task is expected to fail and retry with fresh locks.

Source

Thrown at indexing-service/src/main/java/org/apache/druid/indexing/common/actions/SegmentMetadataUpdateAction.java:80

  @Override
  public Void perform(Task task, TaskActionToolbox toolbox)
  {
    TaskLocks.checkLockCoversSegments(task, toolbox.getTaskLockbox(), segments);

    try {
      toolbox.getTaskLockbox().doInCriticalSection(
          task,
          segments.stream().map(DataSegment::getInterval).collect(Collectors.toSet()),
          CriticalAction.builder()
                        .onValidLocks(
                            () -> {
                              toolbox.getIndexerMetadataStorageCoordinator().updateSegmentMetadata(segments);
                              return null;
                            }
                        )
                        .onInvalidLocks(
                            () -> {
                              throw new ISE("Some locks for task[%s] are already revoked", task.getId());
                            }
                        )
                        .build()
      );
    }
    catch (Exception e) {
      throw new RuntimeException(e);
    }

    // Emit metrics
    final ServiceMetricEvent.Builder metricBuilder = new ServiceMetricEvent.Builder();
    IndexTaskUtils.setTaskDimensions(metricBuilder, task);

    for (DataSegment segment : segments) {
      metricBuilder.setDimension(DruidMetrics.INTERVAL, segment.getInterval().toString());
      toolbox.getEmitter().emit(metricBuilder.setMetric("segment/moved/bytes", segment.getSize()));
    }

View on GitHub (pinned to 9b90983fd2)

Solutions

  1. Let the task fail and retry; on retry it will reacquire valid locks.
  2. Reschedule the higher-priority (compaction/overwrite) task to avoid overlapping the running task's intervals.
  3. Check task lock dashboards/logs to identify which task revoked the locks and adjust lock priorities.
  4. Reduce task runtimes so locks are used promptly after acquisition.
Defensive patterns

Strategy: try-catch

Validate before calling

// Pre-check lock validity before issuing the action where possible:
// if (toolbox.getTaskLockbox().isLockRevokedOrMissing(task)) fail fast;

Try / catch

try { result = action.perform(task, toolbox); } catch (ISE e) { if (e.getMessage().contains("already revoked")) { /* task fails; retry with fresh locks */ } else throw e; }

Prevention

When it happens

Trigger: A task calls SEGMENT_METADATA_UPDATE (e.g. during publish/cleanup) after another higher-priority task revoked its time-sharing locks for the overlapping intervals.

Common situations: Compaction/overwrite tasks preempting a still-running ingestion task; long-running tasks whose locks expired or were revoked mid-flight; manual kill/append tasks with higher priority overlapping the same interval.

Related errors


AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/87f73b63b0090d11. Report an issue: GitHub.