{"record":{"id":"171c839c56bf04d0","repo":"apache/druid","slug":"can-t-find-previous-segmentids-for-sequence-s","errorCode":null,"errorMessage":"Can't find previous segmentIds for sequence[%s]","messagePattern":"Can't find previous segmentIds for sequence\\[(.+?)\\]","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"indexing-service/src/main/java/org/apache/druid/indexing/common/task/batch/parallel/SinglePhaseParallelIndexTaskRunner.java","lineNumber":232,"sourceCode":"      String dataSource,\n      DateTime timestamp,\n      String sequenceName,\n      @Nullable String prevSegmentId\n  ) throws IOException\n  {\n    NonnullPair<Interval, String> intervalAndVersion = findIntervalAndVersion(timestamp, LockGranularity.TIME_CHUNK);\n\n    MutableObject<SegmentIdWithShardSpec> segmentIdHolder = new MutableObject<>();\n    sequenceToSegmentIds.compute(sequenceName, (k, v) -> {\n      final int prevSegmentIdIndex;\n      final List<String> segmentIds;\n      if (prevSegmentId == null) {\n        prevSegmentIdIndex = -1;\n        segmentIds = v == null ? new ArrayList<>() : v;\n      } else {\n        segmentIds = v;\n        if (segmentIds == null) {\n          throw new ISE(\"Can't find previous segmentIds for sequence[%s]\", sequenceName);\n        }\n        prevSegmentIdIndex = segmentIds.indexOf(prevSegmentId);\n        if (prevSegmentIdIndex == -1) {\n          throw new ISE(\"Can't find previously allocated segmentId[%s] for sequence[%s]\", prevSegmentId, sequenceName);\n        }\n      }\n      final int nextSegmentIdIndex = prevSegmentIdIndex + 1;\n      final SegmentIdWithShardSpec newSegmentId;\n      if (nextSegmentIdIndex < segmentIds.size()) {\n        SegmentId segmentId = SegmentId.tryParse(dataSource, segmentIds.get(nextSegmentIdIndex));\n        if (segmentId == null) {\n          throw new ISE(\"Illegal segmentId format [%s]\", segmentIds.get(nextSegmentIdIndex));\n        }\n        newSegmentId = new SegmentIdWithShardSpec(\n            segmentId.getDataSource(),\n            segmentId.getInterval(),\n            segmentId.getVersion(),\n            new BuildingNumberedShardSpec(segmentId.getPartitionNum())","sourceCodeStart":214,"sourceCodeEnd":250,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/indexing-service/src/main/java/org/apache/druid/indexing/common/task/batch/parallel/SinglePhaseParallelIndexTaskRunner.java#L214-L250","documentation":"Thrown by SinglePhaseParallelIndexTaskRunner.allocateNewSegment when allocating the next segment for a sequence: the runner looks up the previously allocated segment IDs list in the current partition map but finds no entry for the previous segment's sequence. This is an internal invariant of the parallel batch ingestion state machine — the sequence must already have a recorded list of segment IDs before a subsequent allocation. Druid throws ISE because reaching this state indicates corrupted or out-of-sync task state, not user input error.","triggerScenarios":"A subtask reports a prevSegmentId for a sequenceName that has no entry in the allocateAction-returned segment ID map; typically after partial supervisor-task state loss, task retry against a different overlord state, or concurrent overwrites/append allocations evicting the sequence entry.","commonSituations":"Killing and resubmitting a parallel batch index task mid-run while subtasks still report old segment IDs; overlord metadata store restored from backup; running append-to-existing jobs where segment allocation state was reset between attempts.","solutions":["Retry the supervisor task from scratch so all subtasks reallocate segments and rebuild the sequence map","Check overlord task logs to find which subtask returned the stale prevSegmentId and whether its attempt number is older than the current state","Verify the metadata store (derby/mysql) used for task/segment state is consistent and not shared/rolled back between clusters","Upgrade Druid — several segment-allocation state bugs in SinglePhaseParallelIndexTaskRunner were fixed in later releases"],"exampleFix":"// before (subtask state referencing stale sequence)\nfinal List<String> segmentIds = partitionMap.get(sequenceName); // null\n// after: restart the supervisor task so subtasks reallocate\n// or guard allocation:\nif (segmentIds == null) { segmentIds = allocateFreshSequence(sequenceName); }","handlingStrategy":"retry","validationCode":"// before retrying/resubmitting, verify task state is fresh\nTaskStatus status = overlordClient.status(supervisorTaskId).get();\nif (!\"RUNNING\".equals(status.getStatusCode().toString())) { throw new IllegalStateException(\"resubmit supervisor before subtask allocation\"); }","typeGuard":null,"tryCatchPattern":"catch (ISE e) {\n  if (e.getMessage().startsWith(\"Can't find previous segmentIds\")) {\n    log.warn(\"stale allocation state; resubmitting supervisor task\");\n    resubmitSupervisor(taskSpec);\n  } else throw e;\n}","preventionTips":["Don't reuse subtask specs from an earlier attempt against a restarted supervisor","Avoid restoring task/segment state from backups mid-ingestion","Keep all Druid nodes on the same version","Monitor overlord logs for allocation inconsistencies during parallel batch runs"],"tags":["ingestion","batch-ingestion","internal-state"],"backgroundTag":"internal-invariant-violation","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}