apache/druid · error · IllegalStateException

Can't find segmentsForSequence for sequence[%s]

Error message

Can't find segmentsForSequence for sequence[%s]

What it means

Thrown by BatchAppenderatorDriver.pushAndClear when it iterates the sequence names used for pushing and finds no SegmentsForSequence entry in the driver's internal segments map. Every sequence that had pending pushes must have bookkeeping state; a missing entry means the sequence was never registered or was already cleaned up.

Source

Thrown at server/src/main/java/org/apache/druid/segment/realtime/appenderator/BatchAppenderatorDriver.java:168

    // Sanity check
    final Map<SegmentIdWithShardSpec, DataSegment> pushedSegmentIdToSegmentMap = segmentsAndCommitMetadata
        .getSegments()
        .stream()
        .collect(Collectors.toMap(SegmentIdWithShardSpec::fromDataSegment, Function.identity()));

    if (!pushedSegmentIdToSegmentMap.keySet().equals(requestedSegmentIdsForSequences)) {
      throw new ISE(
          "Pushed segments[%s] are different from the requested ones[%s]",
          pushedSegmentIdToSegmentMap.keySet(),
          requestedSegmentIdsForSequences
      );
    }

    synchronized (segments) {
      for (String sequenceName : sequenceNames) {
        final SegmentsForSequence segmentsForSequence = segments.get(sequenceName);
        if (segmentsForSequence == null) {
          throw new ISE("Can't find segmentsForSequence for sequence[%s]", sequenceName);
        }

        segmentsForSequence.getAllSegmentsOfInterval().forEach(segmentsOfInterval -> {
          final SegmentWithState appendingSegment = segmentsOfInterval.getAppendingSegment();
          if (appendingSegment != null) {
            final DataSegment pushedSegment = pushedSegmentIdToSegmentMap.get(appendingSegment.getSegmentIdentifier());
            if (pushedSegment == null) {
              throw new ISE("Can't find pushedSegments for segment[%s]", appendingSegment.getSegmentIdentifier());
            }

            segmentsOfInterval.finishAppendingToCurrentActiveSegment(
                segmentWithState -> segmentWithState.pushAndDrop(pushedSegment)
            );
          }
        });
      }
    }

View on GitHub (pinned to 9b90983fd2)

Solutions

  1. Ensure pushAllAndClear is called exactly once per batch task lifecycle, before completing the publish
  2. Verify no code path calls pushAndClear twice or after segments were cleared
  3. Check task logs for earlier driver calls that reset sequence state
  4. Retry the task if the state desync resulted from a transient failure
Defensive patterns

Strategy: validation

Validate before calling

// ensure each sequence still has bookkeeping state before pushing
for (String seq : sequenceNames) {
  if (!driver.hasSequence(seq)) {
    throw new IllegalStateException("Sequence already cleared: " + seq);
  }
}

Try / catch

try {
  driver.pushAllAndClear(publisher, commitFn);
} catch (ISE e) {
  if (e.getMessage().startsWith("Can't find segmentsForSequence")) {
    // sequence state already cleared; restart the task rather than re-pushing
    throw new TaskAbortedException(e);
  }
  throw e;
}

Prevention

When it happens

Trigger: pushAndClear iterating sequenceNames after a push and encountering a sequenceName with no entry in the synchronized segments map — e.g. pushAllAndClear called with sequences whose state was already cleared by a previous pushAndClear call.

Common situations: Double-invocation of pushAllAndClear in the same task lifecycle; calling the driver after commit/cleanup; task restart logic reusing stale sequence names; bugs in task orchestration code around the batch driver.

Understand the failure class

Background: "Not found" and "does not exist" errors: why "Task not found", "No such folder", and "Can't find" fire when a lookup comes back empty — this error's family across 14 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/a957914bb96210b7. Report an issue: GitHub.