{"record":{"id":"3e4303b52c914f5f","repo":"apache/druid","slug":"last-status-of-complete-task-is-missing","errorCode":null,"errorMessage":"Last status of complete task is missing!","messagePattern":"Last status of complete task is missing!","errorType":"exception","errorClass":"ISE","httpStatus":null,"severity":"error","filePath":"indexing-service/src/main/java/org/apache/druid/indexing/common/task/batch/parallel/ParallelIndexPhaseRunner.java","lineNumber":155,"sourceCode":"      while (isRunning() && subTaskSpecIterator.hasNext() && taskMonitor.getNumRunningTasks() < maxNumConcurrentSubTasks) {\n        submitNewTask(taskMonitor, subTaskSpecIterator.next());\n      }\n\n      LOG.info(\"Waiting for subTasks to be completed\");\n      while (isRunning()) {\n        final SubTaskCompleteEvent<SubTaskType> taskCompleteEvent = taskCompleteEvents.poll(\n            taskStatusCheckingPeriod,\n            TimeUnit.MILLISECONDS\n        );\n\n        if (taskCompleteEvent != null) {\n          final TaskState completeState = taskCompleteEvent.getLastState();\n          getSubtaskCompletionCallback(taskCompleteEvent).run();\n          switch (completeState) {\n            case SUCCESS:\n              final TaskStatusPlus completeStatus = taskCompleteEvent.getLastStatus();\n              if (completeStatus == null) {\n                throw new ISE(\"Last status of complete task is missing!\");\n              }\n              if (!subTaskSpecIterator.hasNext()) {\n                // We have no more subTasks to run\n                if (taskMonitor.getNumRunningTasks() == 0 && taskCompleteEvents.isEmpty()) {\n                  subTaskScheduleAndMonitorStopped = true;\n                  if (subTaskSpecIterator.count == taskMonitor.getNumSucceededTasks()) {\n                    // Succeeded\n                    state = TaskState.SUCCESS;\n                  } else {\n                    // Failed\n                    final ParallelIndexingPhaseProgress monitorStatus = taskMonitor.getProgress();\n                    throw new ISE(\n                        \"Expected [%d] tasks to succeed, but we got [%d] succeeded tasks and [%d] failed tasks\",\n                        subTaskSpecIterator.count,\n                        monitorStatus.getSucceeded(),\n                        monitorStatus.getFailed()\n                    );\n                  }","sourceCodeStart":137,"sourceCodeEnd":173,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/indexing-service/src/main/java/org/apache/druid/indexing/common/task/batch/parallel/ParallelIndexPhaseRunner.java#L137-L173","documentation":"Thrown by ParallelIndexPhaseRunner.run when a subtask completes with TaskState.SUCCESS but its TaskStatusPlus payload (taskCompleteEvent.getLastStatus()) is null. The runner needs the status to inspect report data (e.g. multi-phase partitioning reports); without it, the phase cannot proceed. This indicates an inconsistent completion event rather than a task failure.","triggerScenarios":"A successful subtask completion event is delivered with a null lastStatus — typically when the overlord/task runner reports SUCCESS without attaching the TaskStatusPlus, or when completion bookkeeping (taskCompleteEvent) is constructed incompletely during coordinator/overlord failover.","commonSituations":"Overlord restart or HA failover while subtasks complete; custom or downgraded overlord/task runner clients that drop the status payload; bugs in task completion reporting plugins.","solutions":["Check overlord logs for how the completion event for the subtask was recorded; restart the supervisor task to rerun the phase.","Upgrade Druid — completion-event reporting bugs have been fixed in later versions.","Verify no custom extensions replace the task runner/overlord completion path and drop lastStatus.","If failover-induced, ensure the overlord's task storage (metadata DB) is healthy so completion events are fully persisted."],"exampleFix":"// caller-side guard before trusting a completion event\nif (event.getLastState() == TaskState.SUCCESS && event.getLastStatus() == null) {\n  // resubmit/refresh the task status instead of feeding the event to the phase runner\n  taskStatus = overlordClient.getTaskStatus(event.getId()).get();\n}","handlingStrategy":"try-catch","validationCode":"if (event.getLastState() == TaskState.SUCCESS && event.getLastStatus() == null) {\n  throw new IllegalStateException(\"completion event for \" + event.getId() + \" lacks status; refetch from overlord\");\n}","typeGuard":null,"tryCatchPattern":"try {\n  runner.run(toolbox);\n} catch (ISE e) {\n  if (e.getMessage().contains(\"Last status of complete task is missing\")) {\n    // refetch status from overlord and retry the phase\n  } else throw e;\n}","preventionTips":["Keep overlord/middle manager versions aligned","Ensure task completion events are fully persisted in metadata storage","Avoid custom extensions that strip TaskStatusPlus from completion events"],"tags":["parallel-ingestion","subtask-completion","internal-state"],"backgroundTag":"internal-invariant-violation","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}