apache/druid · error · IllegalStateException

Can't find previous segmentIds for sequence[%s]

Error message

Can't find previous segmentIds for sequence[%s]

What it means

Thrown by SinglePhaseParallelIndexTaskRunner.allocateNewSegment when allocating the next segment for a sequence: the runner looks up the previously allocated segment IDs list in the current partition map but finds no entry for the previous segment's sequence. This is an internal invariant of the parallel batch ingestion state machine — the sequence must already have a recorded list of segment IDs before a subsequent allocation. Druid throws ISE because reaching this state indicates corrupted or out-of-sync task state, not user input error.

Source

Thrown at indexing-service/src/main/java/org/apache/druid/indexing/common/task/batch/parallel/SinglePhaseParallelIndexTaskRunner.java:232

      String dataSource,
      DateTime timestamp,
      String sequenceName,
      @Nullable String prevSegmentId
  ) throws IOException
  {
    NonnullPair<Interval, String> intervalAndVersion = findIntervalAndVersion(timestamp, LockGranularity.TIME_CHUNK);

    MutableObject<SegmentIdWithShardSpec> segmentIdHolder = new MutableObject<>();
    sequenceToSegmentIds.compute(sequenceName, (k, v) -> {
      final int prevSegmentIdIndex;
      final List<String> segmentIds;
      if (prevSegmentId == null) {
        prevSegmentIdIndex = -1;
        segmentIds = v == null ? new ArrayList<>() : v;
      } else {
        segmentIds = v;
        if (segmentIds == null) {
          throw new ISE("Can't find previous segmentIds for sequence[%s]", sequenceName);
        }
        prevSegmentIdIndex = segmentIds.indexOf(prevSegmentId);
        if (prevSegmentIdIndex == -1) {
          throw new ISE("Can't find previously allocated segmentId[%s] for sequence[%s]", prevSegmentId, sequenceName);
        }
      }
      final int nextSegmentIdIndex = prevSegmentIdIndex + 1;
      final SegmentIdWithShardSpec newSegmentId;
      if (nextSegmentIdIndex < segmentIds.size()) {
        SegmentId segmentId = SegmentId.tryParse(dataSource, segmentIds.get(nextSegmentIdIndex));
        if (segmentId == null) {
          throw new ISE("Illegal segmentId format [%s]", segmentIds.get(nextSegmentIdIndex));
        }
        newSegmentId = new SegmentIdWithShardSpec(
            segmentId.getDataSource(),
            segmentId.getInterval(),
            segmentId.getVersion(),
            new BuildingNumberedShardSpec(segmentId.getPartitionNum())

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Retry the supervisor task from scratch so all subtasks reallocate segments and rebuild the sequence map
2. Check overlord task logs to find which subtask returned the stale prevSegmentId and whether its attempt number is older than the current state
3. Verify the metadata store (derby/mysql) used for task/segment state is consistent and not shared/rolled back between clusters
4. Upgrade Druid — several segment-allocation state bugs in SinglePhaseParallelIndexTaskRunner were fixed in later releases

Example fix

// before (subtask state referencing stale sequence)
final List<String> segmentIds = partitionMap.get(sequenceName); // null
// after: restart the supervisor task so subtasks reallocate
// or guard allocation:
if (segmentIds == null) { segmentIds = allocateFreshSequence(sequenceName); }

// before retrying/resubmitting, verify task state is fresh
TaskStatus status = overlordClient.status(supervisorTaskId).get();
if (!"RUNNING".equals(status.getStatusCode().toString())) { throw new IllegalStateException("resubmit supervisor before subtask allocation"); }

catch (ISE e) {
  if (e.getMessage().startsWith("Can't find previous segmentIds")) {
    log.warn("stale allocation state; resubmitting supervisor task");
    resubmitSupervisor(taskSpec);
  } else throw e;
}

When it happens

Trigger: A subtask reports a prevSegmentId for a sequenceName that has no entry in the allocateAction-returned segment ID map; typically after partial supervisor-task state loss, task retry against a different overlord state, or concurrent overwrites/append allocations evicting the sequence entry.

Common situations: Killing and resubmitting a parallel batch index task mid-run while subtasks still report old segment IDs; overlord metadata store restored from backup; running append-to-existing jobs where segment allocation state was reset between attempts.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• Unexpected state: Two versions([%s], [%s]) for the same inte
• Can't find previously allocated segmentId[%s] for sequence[%
• Failed to add a row with timestamp[%s]
• Pushed segments[%s] are different from the requested ones[%s
• Can't find segmentsForSequence for sequence[%s]

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/171c839c56bf04d0. Report an issue: GitHub.