apache/druid · error · IllegalStateException

Work order not present for stage[%s]

Error message

Work order not present for stage[%s]

What it means

Thrown by ControllerQueryKernel.startStage as an IllegalStateException when the kernel's stageWorkOrders map contains no entry for the given StageId. The kernel requires createWorkOrders(...) to have populated work orders for the stage before startStage is called, since starting a stage means handing those work orders to workers.

Source

Thrown at multi-stage-query/src/main/java/org/apache/druid/msq/kernel/controller/ControllerQueryKernel.java:590

  }

  /**
   * Delegates call to {@link ControllerStageTracker#getResultObject()}
   */
  public Object getResultObjectForStage(final StageId stageId)
  {
    return getStageTrackerOrThrow(stageId).getResultObject();
  }

  /**
   * Checks if the stage can be started, delegates call to {@link ControllerStageTracker#start()} for internal phase
   * transition and registers the transition in this queryKernel. Work orders need to be created via
   * {@link ControllerQueryKernel#createWorkOrders(int, int, Int2ObjectMap)} before calling this method.
   */
  public void startStage(final StageId stageId)
  {
    if (stageWorkOrders.get(stageId) == null) {
      throw new ISE("Work order not present for stage[%s]", stageId);
    }

    doWithStageTracker(stageId, stageTracker -> {
      if (stageTracker.getPhase() != ControllerStagePhase.NEW) {
        throw new ISE("Cannot start the stage: [%s]", stageId);
      }

      stageTracker.start();
    });
  }

  /**
   * Checks if the stage can be finished, delegates call to {@link ControllerStageTracker#finish()} for internal phase
   * transition and registers the transition in this query kernel
   * <p>
   * If the method is called with strict = true, we confirm if the stage can be marked as finished or else
   * throw illegal argument exception
   */

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Ensure createWorkOrders(...) is called for the stage number before startStage(stageId)
2. Check the controller's state-machine ordering (startWorkForStage) to confirm the stage is transitioning from a phase where work orders exist
3. If this follows a controller restart, re-run the query or verify the kernel snapshot persisted in the task includes work orders
4. Guard with a kernel API check (e.g. isStageKnown / getStagePhase) before calling startStage

Example fix

// before
kernel.startStage(stageId);

// after
if (kernel.getStagePhase(stageId) == ControllerStagePhase.NEW) {
  kernel.startStage(stageId);
}

if (queryKernel.getStagePhase(stageId) == ControllerStagePhase.NEW) {
  queryKernel.startStage(stageId);
}

boolean canStartStage(StageId stageId) {
  try {
    return queryKernel.getStagePhase(stageId) == ControllerStagePhase.NEW;
  } catch (IllegalArgumentException e) {
    return false;
  }
}

try {
  queryKernel.startStage(stageId);
} catch (IllegalStateException e) {
  LOG.warn(e, "Stage %s has no work orders; skipping start", stageId);
}

When it happens

Trigger: Calling startStage(stageId) before createWorkOrders(stageNumber, maxWorkerCount, workerOrders) has run for that stage, or after the work orders were removed (e.g. by a prior finishStage). Also reachable via startWorkForStage when the controller state machine is out of order or the kernel was reloaded/restored from a snapshot that lacks the work orders.

Common situations: Controller failover/restart replaying the state machine with a kernel snapshot that predates work-order creation; custom or patched controller logic invoking startStage manually; race conditions where the stage finished and finishStage removed the work orders before a retry path tried to start it again.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• Cannot start the stage: [%s]
• Cannot mark the stage: [%s] finished
• Stage[%d] work orders not found
• Work order for worker[%d] not found for stage[%d]
• Result partition information is not ready yet

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/557cca90729fa900. Report an issue: GitHub.