apache/druid · error · ISE

Locks[%s] for task[%s] can't cover segments[%s]

Error message

Locks[%s] for task[%s] can't cover segments[%s]

What it means

KillUnusedSegmentsTask verifies, before deleting unused segments, that the task's locks actually cover every segment interval it intends to kill. If some unused segments fall outside the locked intervals, the run aborts to prevent deleting data the task has no write-authority over.

Source

Thrown at indexing-service/src/main/java/org/apache/druid/indexing/common/task/KillUnusedSegmentsTask.java:274

        // No more segments eligible for kill, do not proceed further
        break;
      }

      // Fetch locks each time as a revokal could have occurred in between batches
      final NavigableMap<DateTime, List<TaskLock>> taskLockMap
              = getNonRevokedTaskLockMap(toolbox.getTaskActionClient());

      final Set<DataSegment> unusedSegments = unusedSegmentsPlus.stream()
                                                                .map(DataSegmentPlus::getDataSegment)
                                                                .collect(Collectors.toSet());
      final Map<String, DataSegmentPlus> unusedIdToSegmentPlus = CollectionUtils.toMap(
          unusedSegmentsPlus,
          segment -> segment.getDataSegment().getId().toString(),
          Function.identity()
      );

      if (!TaskLocks.isLockCoversSegments(taskLockMap, unusedSegments)) {
        throw new ISE(
                "Locks[%s] for task[%s] can't cover segments[%s]",
                taskLockMap.values().stream().flatMap(List::stream).collect(Collectors.toList()),
                getId(),
                unusedSegments
        );
      }

      // Kill segments - order of steps 1, 2, 3, 4 must remain the same

      // 1. Determine parent segment ids of killable unused segments
      final Map<String, String> upgradedFromSegmentIds
          = fetchParentIdsForSegments(toolbox, unusedIdToSegmentPlus);

      // 2. Identify killable segments whose load specs are not shared with any other segment
      final List<DataSegment> segmentsToKillFromDeepStore = getKillableSegments(
          unusedIdToSegmentPlus,
          upgradedFromSegmentIds,
          usedSegmentLoadSpecs,

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Re-submit the kill task with an interval that fully covers (and is aligned to) the segments to be killed, or enable 'markAsUsed' null + limit filters so locks cover candidates
2. Check the Overlord lock table for the task's acquired locks and compare with the segments in the error message
3. Retry after any concurrent compaction/kill finishes so locks are granted over the full interval

Example fix

// before
KillUnusedSegmentsTask kill = new KillUnusedSegmentsTask(null, null, Ids,
    Intervals.of("2024-01-05/2024-01-07"), null, null, null);
// after (interval widened to cover full segment intervals)
KillUnusedSegmentsTask kill = new KillUnusedSegmentsTask(null, null, Ids,
    Intervals.of("2024-01-01/2024-01-15"), null, null, null);

// before submitting kill: ensure interval covers all candidate segments
Interval aligned = Intervals.utc(granularity.bucketStart(raw.getStart()).getMillis(), granularity.bucketEnd(raw.getEnd().minus(1)).getMillis());

try { killTask.runTask(actionClient); } catch (ISE e) { if (e.getMessage().contains("can't cover segments")) { resubmitWithWiderInterval(); } else { throw e; } }

When it happens

Trigger: Calling kill with an interval/partitions filter whose matching unused segments extend beyond the intervals acquired by the task's locks (e.g. segment interval wider than the kill interval, or lock revoked/shrunk during the run).

Common situations: Kill tasks submitted with intervals that don't exactly match segment intervals (misaligned segmentGranularity); concurrent compaction/replacement changing segment intervals; automatic lock downgrades while the kill task runs.

Understand the failure class

Background: "You do not have permission" / 403 Forbidden errors: authenticated but not allowed — causes and fixes across open-source libraries — this error's family across 31 libraries.

Related errors

• Unused segment[%s] has version[%s] > task version[%s]
• The lock for interval[%s] is preempted and no longer valid
• Some locks for task[%s] are already revoked
• Some locks for task[%s] are already revoked
• Lock interval[%s] != task interval[%s]

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/b601d18681877e7b. Report an issue: GitHub.