apache/druid · error · IllegalArgumentException

Cannot find kernel corresponding to stage [%s] in query [%s]

Error message

Cannot find kernel corresponding to stage [%s] in query [%s]

What it means

Thrown by ControllerQueryKernel.getStageTrackerOrThrow as an IllegalArgumentException when stageTrackers has no ControllerStageTracker for the requested StageId. The trackers are fixed at kernel creation from the QueryDefinition, so this indicates the stage id does not belong to this query.

Source

Thrown at multi-stage-query/src/main/java/org/apache/druid/msq/kernel/controller/ControllerQueryKernel.java:702

  public IntSet getAllParticipatingWorkers()
  {
    final IntSet retVal = new IntAVLTreeSet();

    for (final ControllerStageTracker tracker : stageTrackers.values()) {
      retVal.addAll(tracker.getWorkerInputs().workers());
    }

    return retVal;
  }

  /**
   * Fetches and returns the stage kernel corresponding to the provided stage id, else throws {@link IAE}
   */
  private ControllerStageTracker getStageTrackerOrThrow(StageId stageId)
  {
    ControllerStageTracker stageTracker = stageTrackers.get(stageId);
    if (stageTracker == null) {
      throw new IAE("Cannot find kernel corresponding to stage [%s] in query [%s]", stageId, queryDef.getQueryId());
    }
    return stageTracker;
  }

  private WorkOrder getWorkOrder(int workerNumber, StageId stageId)
  {
    Int2ObjectMap<WorkOrder> stageWorkOrder = stageWorkOrders.get(stageId);

    if (stageWorkOrder == null) {
      throw new ISE("Stage[%d] work orders not found", stageId.getStageNumber());
    }

    WorkOrder workOrder = stageWorkOrder.get(workerNumber);
    if (workOrder == null) {
      throw new ISE("Work order for worker[%d] not found for stage[%d]", workerNumber, stageId.getStageNumber());
    }
    return workOrder;
  }

View on GitHub (pinned to 9b90983fd2)

Solutions

Confirm the StageId's queryId matches the kernel's query (queryDef.getQueryId()) before lookup
Verify the stage number exists in the current QueryDefinition's stage count
After a controller restart or kernel reload, use the stage ids from the reloaded kernel, not cached ones from before
Handle IAE in the caller and re-fetch or re-register the kernel for the correct query id

Example fix

// before
ControllerStageKernel kernel = queryKernel.stageKernel(stageId);

// after
if (!queryKernel.isStageKnown(stageId) || !stageId.getQueryId().equals(queryDef.getQueryId())) {
  throw new ISE("Stage %s does not belong to query %s", stageId, queryDef.getQueryId());
}
ControllerStageKernel kernel = queryKernel.stageKernel(stageId);

Defensive patterns

Strategy: type-guard

Validate before calling

if (queryKernel.isStageKnown(stageId) && stageId.getQueryId().equals(queryDef.getQueryId())) {
  Object phase = queryKernel.getStagePhase(stageId);
}

Type guard

boolean stageBelongsToQuery(StageId stageId, ControllerQueryKernel kernel) {
  try {
    kernel.getStagePhase(stageId);
    return true;
  } catch (IllegalArgumentException e) {
    return false;
  }
}

Try / catch

try {
  return queryKernel.getStagePhase(stageId);
} catch (IllegalArgumentException e) {
  LOG.warn(e, "No tracker for stage %s in query %s", stageId, queryId);
  return null;
}

Prevention

Build StageIds only from the current QueryDefinition's stage list
Never cache StageIds across controller restarts or kernel reloads
Validate stage number against the query's stage count before lookups
Route all kernel access through a single helper that checks query id first

When it happens

Trigger: Querying a StageId from a different query (mismatched queryId) or a stage number >= the query's stage count; calling getStagePhase, stageKernel, doesStageHaveResultPartitions, getResultPartitionsForStage, getWorkersToSendPartitionBoundaries, or getResultPartitionBoundariesForStage with a stale or fabricated stage id after kernel recreation from a new QueryDefinition.

Common situations: Worker/controller messages arriving for a query whose kernel was rebuilt with a different stage layout; version changes to the query definition that renumber stages; bugs where a StageId is built from the wrong queryId and stage number pair.

Understand the failure class

Background: Record Not Found Errors: "not found", RecordNotFound, and "was not found" — what they mean and how to fix them — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/d11ade865a397e1b. Report an issue: GitHub.