apache/druid · error · IllegalStateException

Stage [%s] is missing a definition

Error message

Stage [%s] is missing a definition

What it means

After collecting all stages referenced as inputs (non-final stages), QueryDefinition.create verifies that every referenced input stage actually has a definition in the stage map. A dangling reference — a stage listed as an input of another stage but never defined — throws ISE, since the query would deadlock waiting for a stage that never runs.

Source

Thrown at multi-stage-query/src/main/java/org/apache/druid/msq/kernel/QueryDefinition.java:96

    final Map<StageId, StageDefinition> stageMap = new HashMap<>();
    final Set<StageId> nonFinalStages = new HashSet<>();
    final IntSet stageNumbers = new IntOpenHashSet();

    for (final StageDefinition stage : stageDefinitions) {
      if (!stageNumbers.add(stage.getStageNumber())) {
        throw new ISE("Cannot accept duplicate stage numbers");
      }

      stageMap.put(stage.getId(), stage);

      for (int stageNumber : stage.getInputStageNumbers()) {
        nonFinalStages.add(new StageId(stage.getId().getQueryId(), stageNumber));
      }
    }

    for (final StageId nonFinalStageId : nonFinalStages) {
      if (!stageMap.containsKey(nonFinalStageId)) {
        throw new ISE("Stage [%s] is missing a definition", nonFinalStageId);
      }
    }

    final int finalStageCandidates = stageMap.size() - nonFinalStages.size();

    if (finalStageCandidates == 1) {

      return new QueryDefinition(
          stageMap,
          Iterables.getOnlyElement(Sets.difference(stageMap.keySet(), nonFinalStages)),
          context == null ? QueryContext.empty() : context
      );
    } else {
      throw new IAE("Must have a single final stage, but found [%d] candidates", finalStageCandidates);
    }
  }

  public static QueryDefinitionBuilder builder(final String queryId)

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Ensure every input stage number has a corresponding StageDefinition added to the builder before building
2. Fix plan-rewrite logic to update/remove dangling input references when stages are dropped or renumbered
3. Validate the plan (all input numbers < stage count and defined) before calling create

Example fix

// before: input referenced but never added
builder.add(StageDefinition...withInputStage(1)); // stage 1 never added
// after
builder.add(stage1Def); // add the dependency first
builder.add(StageDefinition...withInputStage(1));

Set<Integer> defined = stageDefinitions.stream().map(StageDefinition::getStageNumber).collect(toSet());
for (StageDefinition stage : stageDefinitions) {
  for (int input : stage.getInputStageNumbers()) {
    if (!defined.contains(input)) {
      throw new IllegalArgumentException("Stage " + stage.getStageNumber() + " references undefined input " + input);
    }
  }
}

try {
  return QueryDefinition.create(stageDefinitions, context);
} catch (IllegalStateException e) {
  throw new PlanValidationException("Dangling input stage reference", e);
}

When it happens

Trigger: QueryDefinition.create with stages whose getInputStageNumbers() reference a stage number not present in stageDefinitions (e.g. a plan built with addInput referencing a stage never added, or a stage removed during plan rewriting without updating inputs).

Common situations: Plan-rewriting/optimization bugs that drop or renumber stages; hand-assembled multi-stage plans in tests or embedded engines; merging partial plans without their dependency stages.

Understand the failure class

Background: "Not found" and "does not exist" errors: why "Task not found", "No such folder", and "Can't find" fire when a lookup comes back empty — this error's family across 14 libraries.

Related errors

• Cannot accept duplicate stage numbers
• Must have a single final stage, but found [%d] candidates
• No such stage [%s]
• Unknown
• Unknown

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/7446004e8dd65386. Report an issue: GitHub.