{"record":{"id":"af9e1b473f6e6bf5","repo":"apache/iceberg","slug":"cannot-parse-predicates-in-where-option-af9e1b","errorCode":null,"errorMessage":"Cannot parse predicates in where option: ","messagePattern":"Cannot parse predicates in where option: ","errorType":"exception","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/procedures/BaseProcedure.java","lineNumber":198,"sourceCode":"    String tableName = Spark3Util.quotedFullIdentifier(tableCatalog().name(), tableIdent);\n    return spark().read().options(options).table(tableName);\n  }\n\n  protected void refreshSparkCache(Identifier ident, Table table) {\n    CacheManager cacheManager = spark.sharedState().cacheManager();\n    DataSourceV2Relation relation =\n        DataSourceV2Relation.create(table, Option.apply(tableCatalog), Option.apply(ident));\n    cacheManager.recacheByPlan(spark, relation);\n  }\n\n  protected Expression filterExpression(Identifier ident, String where) {\n    try {\n      String name = Spark3Util.quotedFullIdentifier(tableCatalog.name(), ident);\n      org.apache.spark.sql.catalyst.expressions.Expression expression =\n          SparkExpressionConverter.collectResolvedSparkExpression(spark, name, where);\n      return SparkExpressionConverter.convertToIcebergExpression(expression);\n    } catch (AnalysisException e) {\n      throw new IllegalArgumentException(\"Cannot parse predicates in where option: \" + where, e);\n    }\n  }\n\n  protected InternalRow newInternalRow(Object... values) {\n    return new GenericInternalRow(values);\n  }\n\n  protected static class Result implements LocalScan {\n    private final StructType readSchema;\n    private final InternalRow[] rows;\n\n    public Result(StructType readSchema, InternalRow[] rows) {\n      this.readSchema = readSchema;\n      this.rows = rows;\n    }\n\n    @Override\n    public StructType readSchema() {","sourceCodeStart":180,"sourceCodeEnd":216,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/procedures/BaseProcedure.java#L180-L216","documentation":"Iceberg procedures accepting a 'where' option parse the predicate via Spark (collecting a resolved expression from a query against the table). If parsing raises AnalysisException — the predicate is invalid SQL or references unknown columns — it is rethrown as IllegalArgumentException with the offending where string.","triggerScenarios":"Calling a procedure (remove_orphan_files, rewrite_data_files, expire_snapshots) with where => '<expr>' where expr fails Spark analysis: syntax error, non-existent column, or type-incompatible comparison.","commonSituations":"Referencing partition columns by wrong names; using Spark SQL functions unsupported in the expression; quoting issues in SQL strings embedded in procedure calls.","solutions":["Test the predicate in a Spark SQL SELECT first (SELECT * FROM table WHERE <expr>) to confirm it resolves.","Fix column names/types in the where clause to match the table schema (df.printSchema()).","Escape single quotes correctly in the where string within the CALL statement."],"exampleFix":"// before\ncall remove_orphan_files(table => 'db.t', where => \"event_time < '2024-01-01'\") -- no such column\n// after\ncall remove_orphan_files(table => 'db.t', where => \"ts < '2024-01-01'\")","handlingStrategy":"validation","validationCode":"spark.sql(s\"SELECT * FROM $table WHERE $where\").limit(1).collect() // probe before CALL","typeGuard":null,"tryCatchPattern":"try {\n  spark.sql(s\"CALL cat.system.remove_orphan_files(table => '$t', where => '$w')\")\n} catch {\n  case e: IllegalArgumentException if e.getMessage.startsWith(\"Cannot parse predicates\") =>\n    logger.error(s\"Bad where clause '$w': ${e.getCause}\")\n}","preventionTips":["Test predicates in a plain SELECT first","Match column names/types against the table schema","Beware quote escaping inside CALL statements"],"tags":["spark","sql","predicate"],"backgroundTag":"invalid-argument-format","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}