apache/iceberg · error · IllegalArgumentException

Cannot parse predicates in where option:

Error message

Cannot parse predicates in where option:

What it means

BaseProcedure.filterExpression converts the procedure's WHERE option into an Iceberg expression by first collecting a resolved Spark expression. If parsing/analysis of the predicate fails (AnalysisException), it rethrows as IllegalArgumentException because the predicate string is unparseable in the table's context.

Solutions

  1. Test the predicate in a plain SELECT ... WHERE query first to confirm it resolves
  2. Fully qualify column names and check spelling against the table schema (DESCRIBE TABLE)
  3. Fix quoting: escape single quotes properly in the where option string
  4. Simplify the predicate — use only deterministic, resolvable expressions

Example fix

// before
CALL cat.sys.remove_orphan_files(table => 'cat.db.t', where => "ts < '2024-01-01'");
// after
CALL cat.sys.remove_orphan_files(table => 'cat.db.t', where => "ts < TIMESTAMP '2024-01-01 00:00:00'");
Defensive patterns

Strategy: validation

Validate before calling

// test the predicate resolves before the procedure
spark.sql(s"SELECT 1 FROM cat.db.t WHERE $whereClause LIMIT 1").collect()

Try / catch

try { spark.sql(call) } catch { case e: IllegalArgumentException if e.getMessage.startsWith("Cannot parse predicates in where option") => /* fix predicate */ }

Prevention

When it happens

Trigger: Calling a procedure with where => 'some invalid predicate' — unresolved columns, wrong function names, non-deterministic expressions, or syntax errors in the WHERE string.

Common situations: Referencing columns that don't exist (or after a rename); using Spark SQL functions not resolvable at that point; quoting issues inside the Scala/SQL string literal; predicates on partition columns with wrong types.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/ab5c0294160eba4f. Report an issue: GitHub.

Appendix: source

Thrown at spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/procedures/BaseProcedure.java:198

    String tableName = Spark3Util.quotedFullIdentifier(tableCatalog().name(), tableIdent);
    return spark().read().options(options).table(tableName);
  }

  protected void refreshSparkCache(Identifier ident, Table table) {
    CacheManager cacheManager = spark.sharedState().cacheManager();
    DataSourceV2Relation relation =
        DataSourceV2Relation.create(table, Option.apply(tableCatalog), Option.apply(ident));
    cacheManager.recacheByPlan(spark, relation);
  }

  protected Expression filterExpression(Identifier ident, String where) {
    try {
      String name = Spark3Util.quotedFullIdentifier(tableCatalog.name(), ident);
      org.apache.spark.sql.catalyst.expressions.Expression expression =
          SparkExpressionConverter.collectResolvedSparkExpression(spark, name, where);
      return SparkExpressionConverter.convertToIcebergExpression(expression);
    } catch (AnalysisException e) {
      throw new IllegalArgumentException("Cannot parse predicates in where option: " + where, e);
    }
  }

  protected InternalRow newInternalRow(Object... values) {
    return new GenericInternalRow(values);
  }

  protected static class Result implements LocalScan {
    private final StructType readSchema;
    private final InternalRow[] rows;

    public Result(StructType readSchema, InternalRow[] rows) {
      this.readSchema = readSchema;
      this.rows = rows;
    }

    @Override
    public StructType readSchema() {

View on GitHub (pinned to 86d9c8fc54)