{"record":{"id":"ab5c0294160eba4f","repo":"apache/iceberg","slug":"cannot-parse-predicates-in-where-option","errorCode":null,"errorMessage":"Cannot parse predicates in where option: ","messagePattern":"Cannot parse predicates in where option: ","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/procedures/BaseProcedure.java","lineNumber":198,"sourceCode":"    String tableName = Spark3Util.quotedFullIdentifier(tableCatalog().name(), tableIdent);\n    return spark().read().options(options).table(tableName);\n  }\n\n  protected void refreshSparkCache(Identifier ident, Table table) {\n    CacheManager cacheManager = spark.sharedState().cacheManager();\n    DataSourceV2Relation relation =\n        DataSourceV2Relation.create(table, Option.apply(tableCatalog), Option.apply(ident));\n    cacheManager.recacheByPlan(spark, relation);\n  }\n\n  protected Expression filterExpression(Identifier ident, String where) {\n    try {\n      String name = Spark3Util.quotedFullIdentifier(tableCatalog.name(), ident);\n      org.apache.spark.sql.catalyst.expressions.Expression expression =\n          SparkExpressionConverter.collectResolvedSparkExpression(spark, name, where);\n      return SparkExpressionConverter.convertToIcebergExpression(expression);\n    } catch (AnalysisException e) {\n      throw new IllegalArgumentException(\"Cannot parse predicates in where option: \" + where, e);\n    }\n  }\n\n  protected InternalRow newInternalRow(Object... values) {\n    return new GenericInternalRow(values);\n  }\n\n  protected static class Result implements LocalScan {\n    private final StructType readSchema;\n    private final InternalRow[] rows;\n\n    public Result(StructType readSchema, InternalRow[] rows) {\n      this.readSchema = readSchema;\n      this.rows = rows;\n    }\n\n    @Override\n    public StructType readSchema() {","sourceCodeStart":180,"sourceCodeEnd":216,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/procedures/BaseProcedure.java#L180-L216","documentation":"BaseProcedure.filterExpression converts the procedure's WHERE option into an Iceberg expression by first collecting a resolved Spark expression. If parsing/analysis of the predicate fails (AnalysisException), it rethrows as IllegalArgumentException because the predicate string is unparseable in the table's context.","triggerScenarios":"Calling a procedure with where => 'some invalid predicate' — unresolved columns, wrong function names, non-deterministic expressions, or syntax errors in the WHERE string.","commonSituations":"Referencing columns that don't exist (or after a rename); using Spark SQL functions not resolvable at that point; quoting issues inside the Scala/SQL string literal; predicates on partition columns with wrong types.","solutions":["Test the predicate in a plain SELECT ... WHERE query first to confirm it resolves","Fully qualify column names and check spelling against the table schema (DESCRIBE TABLE)","Fix quoting: escape single quotes properly in the where option string","Simplify the predicate — use only deterministic, resolvable expressions"],"exampleFix":"// before\nCALL cat.sys.remove_orphan_files(table => 'cat.db.t', where => \"ts < '2024-01-01'\");\n// after\nCALL cat.sys.remove_orphan_files(table => 'cat.db.t', where => \"ts < TIMESTAMP '2024-01-01 00:00:00'\");","handlingStrategy":"validation","validationCode":"// test the predicate resolves before the procedure\nspark.sql(s\"SELECT 1 FROM cat.db.t WHERE $whereClause LIMIT 1\").collect()","typeGuard":null,"tryCatchPattern":"try { spark.sql(call) } catch { case e: IllegalArgumentException if e.getMessage.startsWith(\"Cannot parse predicates in where option\") => /* fix predicate */ }","preventionTips":["Validate the WHERE string with a plain SELECT first","Qualify column names and use typed literals (TIMESTAMP '...', DATE '...')","Beware quoting/escaping when building the predicate string"],"tags":["spark","procedures","predicate","sql-parse"],"backgroundTag":"invalid-argument-format","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}