apache/druid · error · ISE

spec[%s] is in an invalid state[%s]

Error message

spec[%s] is in an invalid state[%s]

What it means

Thrown from the default branch of the switch over the subtask's final TaskState in ParallelIndexPhaseRunner.run. Only SUCCESS and FAILED are expected; any other state (e.g. RUNNING, WAITING) on a completion event is treated as a bug in completion reporting.

Source

Thrown at indexing-service/src/main/java/org/apache/druid/indexing/common/task/batch/parallel/ParallelIndexPhaseRunner.java:196

              } else {
                // We have more subTasks to run, but don't have enough available task slots
                // do nothing
              }
              break;
            case FAILED:
              // TaskMonitor already tried everything it can do for failed tasks. We failed.
              state = TaskState.FAILED;
              subTaskScheduleAndMonitorStopped = true;
              final TaskStatusPlus lastStatus = taskCompleteEvent.getLastStatus();
              if (lastStatus != null) {
                LOG.error("Failed because of the failed sub task[%s]", lastStatus.getId());
              } else {
                final SubTaskSpec<?> spec = taskCompleteEvent.getSpec();
                LOG.error("Failed to process spec[%s] with an unknown last status", spec.getId());
              }
              break;
            default:
              throw new ISE("spec[%s] is in an invalid state[%s]", taskCompleteEvent.getSpec().getId(), completeState);
          }
        }
      }
    }
    finally {
      stopInternal();
      if (!state.isComplete()) {
        state = TaskState.FAILED;
      }
    }

    return state;
  }

  private class CountingSubTaskSpecIterator implements Iterator<SubTaskSpec<SubTaskType>>
  {
    private final Iterator<SubTaskSpec<SubTaskType>> delegate;
    private int count;

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Query the overlord for the subtask's real final status to understand what state was reported.
2. Restart/resubmit the parallel index task; this is normally a one-off inconsistent event.
3. Align Druid versions across the cluster (overlord, middle managers, indexers).
4. Report/fix any extension that emits completion events with non-terminal states.

catch (ISE e) {
  if (e.getMessage().contains("is in an invalid state")) {
    // query overlord for real final task status and resubmit
  }
}

When it happens

Trigger: A task completion event is delivered whose lastState is not SUCCESS or FAILED — an inconsistent TaskStatusPlus from the overlord, or an enum value added later being routed through old runner logic.

Common situations: Corrupted or partially-written task status records after a crash; mixing Druid versions between overlord and middle managers where TaskState semantics differ; custom task runner extensions emitting non-terminal states.

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

• Last status of complete task is missing!
• Expected [%d] tasks to succeed, but we got [%d] succeeded ta
• Failed to publish segments
• Expected [%s], but [%s] is in use
• interval %s, bucketId %s mismatched shard specs: %s and %s

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/ff43713571a47816. Report an issue: GitHub.