apache/druid · error · IllegalStateException

column[ ] doesn't exist, but window function FIRST wants it…

Error message

column[%s] doesn't exist, but window function FIRST wants it to

What it means

Window value processors (FIRST/LAST/etc.) project a new output column from an existing input column via processInternal. If the configured input column is absent from the incoming RowsAndColumns, Druid throws an illegal-state error because the window function cannot be evaluated without its input. This almost always means the column referenced in the window spec doesn't exist at this stage of the query.

Solutions

  1. Verify the column name in the window function matches the actual column in the input (check with EXPLAIN PLAN)
  2. Add the missing column to the projection/scan before the window stage
  3. Fix aliasing so the source column survives to the window operator

Example fix

// before
FIRST_VALUE(nonexistent_col) OVER (ORDER BY ts)
// after
FIRST_VALUE(existing_col) OVER (ORDER BY ts)
Defensive patterns

Strategy: validation

Validate before calling

// verify the column exists in the input signature before the window stage
if (!inputSignature.getColumnNames().contains("my_col")) {
  throw new IllegalStateException("my_col missing before window function");
}

Try / catch

try {
  runQuery(sql);
} catch (IllegalStateException e) {
  if (e.getMessage().contains("doesn't exist, but window function")) {
    throw new QueryValidationException("check window function column name", e);
  }
  throw e;
}

Prevention

When it happens

Trigger: WindowOperatorFrame with a FIRST (or subclass) processor whose inputColumn is not present in the rows flowing through the window stage; typically after a typo in column name or a projection that dropped the column.

Common situations: SQL query references a column alias that doesn't exist in the underlying scan; aggregation renamed the column before window processing; case-sensitivity mismatch between column names; schema drift in the data source.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/bc27b433a246b799. Report an issue: GitHub.

Appendix: source

Thrown at processing/src/main/java/org/apache/druid/query/operator/window/value/WindowValueProcessorBase.java:78

    return outputColumn;
  }

  /**
   * This implements the common logic between the various value processors.  It looks like it could be static, but if
   * it is static then the lambda becomes polymorphic.  We keep it as a member method of the base class so taht the
   * JVM can inline it and specialize the lambda
   *
   * @param input incoming RowsAndColumns, as in Processor.process
   * @param fn    function that converts the input column into the output column
   * @return RowsAndColumns, as in Processor.process
   */
  public RowsAndColumns processInternal(RowsAndColumns input, Function<Column, Column> fn)
  {
    final AppendableRowsAndColumns retVal = RowsAndColumns.expectAppendable(input);

    final Column column = input.findColumn(inputColumn);
    if (column == null) {
      throw new ISE("column[%s] doesn't exist, but window function FIRST wants it to", inputColumn);
    }

    retVal.addColumn(outputColumn, fn.apply(column));
    return retVal;
  }

  @Override
  public boolean validateEquivalent(Processor otherProcessor)
  {
    return getClass() == otherProcessor.getClass()
           && intervalValidation((WindowValueProcessorBase) otherProcessor);
  }

  protected boolean intervalValidation(WindowValueProcessorBase other)
  {
    // Only input needs to be the same for the processors to produce equivalent results
    return inputColumn.equals(other.inputColumn);
  }

View on GitHub (pinned to 9b90983fd2)