What causes "Cannot get NaN value for type " in apache/iceberg?

Apply isNaN()/isNotNaN() only to FLOAT or DOUBLE columns; cast or change the column type.

apache/iceberg · error · IllegalArgumentException

Cannot get NaN value for type

Error message

Cannot get NaN value for type

What it means

ExpressionToSearchArgument.getNaNForType() produces the NaN sentinel for NaN predicate pushdown and only supports FLOAT and DOUBLE ORC columns. When a NaN filter is applied to any other column type, IllegalArgumentException("Cannot get NaN value for type X") is thrown since NaN is meaningless there.

Source

Thrown at orc/src/main/java/org/apache/iceberg/orc/ExpressionToSearchArgument.java:148

  }

  @Override
  public <T> Action isNaN(Bound<T> expr) {
    return () ->
        this.builder.equals(
            idToColumnName.get(expr.ref().fieldId()),
            type(expr.ref().type()),
            literal(expr.ref().type(), getNaNForType(expr.ref().type())));
  }

  private Object getNaNForType(Type type) {
    switch (type.typeId()) {
      case FLOAT:
        return Float.NaN;
      case DOUBLE:
        return Double.NaN;
      default:
        throw new IllegalArgumentException("Cannot get NaN value for type " + type.typeId());
    }
  }

  @Override
  public <T> Action notNaN(Bound<T> expr) {
    return () -> {
      this.builder.startOr();
      isNull(expr).invoke();
      this.builder.startNot();
      isNaN(expr).invoke();
      this.builder.end(); // end NOT
      this.builder.end(); // end OR
    };
  }

  @Override
  public <T> Action lt(Bound<T> expr, Literal<T> lit) {
    return () ->

View on GitHub (pinned to 86d9c8fc54)

Solutions

Apply isNaN()/isNotNaN() only to FLOAT or DOUBLE columns; cast or change the column type.
Rewrite the filter as a null/equality check suitable for the column's actual type.
Disable the NaN pushdown (turn off ORC filter pushdown) if the query cannot change.
Cast the column to double in the query so the NaN predicate has a valid type.

Example fix

// before
df.filter("isnan(dec_col)") // dec_col DECIMAL -> Cannot get NaN value for type DECIMAL
// after
df.filter("dec_col IS NULL OR cast(dec_col AS DOUBLE) = cast(dec_col AS DOUBLE) == false OR isnan(cast(dec_col AS DOUBLE))")
// or simply restrict isnan to float/double columns

Defensive patterns

Strategy: validation

Validate before calling

// only apply NaN filters to float/double columns
if (!(exprType instanceof Types.FloatType || exprType instanceof Types.DoubleType)) {
  throw new IllegalArgumentException("isNaN requires float/double, got " + exprType);
}

Type guard

boolean nanCapable = t instanceof Types.FloatType || t instanceof Types.DoubleType;

Try / catch

try { scan.filter(isNaN(col)); } catch (IllegalArgumentException e) { if (e.getMessage().startsWith("Cannot get NaN value")) { /* use type-appropriate filter */ } throw e; }

Prevention

Use isNan only on float/double columns
Cast decimals to double before NaN checks
Validate filter columns' types in query builders
Keep pushdown types in mind when writing SQL

When it happens

Trigger: Applying an isNotNaN()/isNaN() filter (e.g., Spark's isNan(col)) to a non-float/double column (decimal, int, string) that gets pushed down to ORC search arguments.

Common situations: Spark SQL isNan() on decimal or integer columns with ORC tables and filter pushdown enabled; schemas changed from float to decimal after queries were written.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/5aedb86c4a94392c. Report an issue: GitHub.