apache/druid · error · IllegalStateException

Worker[ ] cannot transistion from state[ ] to state[ ]…

Error message

Worker[%d] cannot transistion from state[%s] to state[%s] while sending work order

What it means

Thrown when workOrderSentForWorker is called for a worker whose current phase cannot legally transition to READING_INPUT. The worker state machine forbids re-sending a work order once the worker has advanced past a phase from which READING_INPUT is reachable, indicating out-of-order or duplicated work-order handling in the controller.

Solutions

1. Only send the work order once per worker and per stage; track which workers already received one.
2. Check the worker's current phase before sending and skip if it has already advanced past READING_INPUT.
3. Add idempotency/ordering to the code path that triggers workOrderSentForWorker (e.g. dedupe on task id).

Example fix

// before
tracker.workOrderSentForWorker(worker); // may be called twice
// after
if (!workOrdersSent.contains(worker)) {
  tracker.workOrderSentForWorker(worker);
  workOrdersSent.add(worker);
}

if (sentWorkOrders.add(stageAndWorkerKey)) {
  tracker.workOrderSentForWorker(worker);
}

try {
  tracker.workOrderSentForWorker(worker);
} catch (IllegalStateException e) {
  // worker already past READING_INPUT; safe to skip duplicate
}

When it happens

Trigger: Calling workOrderSentForWorker twice for the same worker, or after the worker already progressed to POST_SHUFFLE / finished phases, so ControllerWorkerStagePhase.READING_INPUT.canTransitionFrom(state) returns false.

Common situations: Duplicate work-order send due to controller retries; race between task-status callbacks and work-order issuance; test harnesses invoking controller methods out of order.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• BroadcastTablesTooLarge
• Cannot handle dataSource
• Cannot mark the stage
• Cannot mix sortable and unsortable key columns
• Cannot start the stage

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/8e5df94c455dd5cb. Report an issue: GitHub.

Appendix: source

Thrown at multi-stage-query/src/main/java/org/apache/druid/msq/kernel/controller/ControllerStageTracker.java:295

    }
    return workers;
  }

  /**
   * Indicates that the work order for worker has been sent. Transitions the state to {@link ControllerWorkerStagePhase#READING_INPUT}
   * if no more work orders need to be sent.
   *
   * @param worker
   */
  void workOrderSentForWorker(int worker)
  {

    workerToPhase.compute(worker, (wk, state) -> {
      if (state == null) {
        throw new ISE("Worker[%d] not found for stage[%s]", wk, stageDef.getStageNumber());
      }
      if (!ControllerWorkerStagePhase.READING_INPUT.canTransitionFrom(state)) {
        throw new ISE(
            "Worker[%d] cannot transistion from state[%s] to state[%s] while sending work order",
            worker,
            state,
            ControllerWorkerStagePhase.READING_INPUT
        );
      }
      return ControllerWorkerStagePhase.READING_INPUT;
    });
    if (phase != ControllerStagePhase.READING_INPUT) {
      if (allWorkOrdersSent()) {
        // if no more work orders need to be sent, change state to reading input from retrying.
        transitionTo(ControllerStagePhase.READING_INPUT);
      }
    }

  }

  /**

View on GitHub (pinned to 9b90983fd2)