{"record":{"id":"2990841aebe9116b","repo":"apache/iceberg","slug":"cannot-parse-predicates-in-where-option-where","errorCode":null,"errorMessage":"Cannot parse predicates in where option: ${where}","messagePattern":"Cannot parse predicates in where option: (.+?)","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/procedures/BaseProcedure.java","lineNumber":182,"sourceCode":"    String tableName = Spark3Util.quotedFullIdentifier(tableCatalog().name(), tableIdent);\n    return spark().read().options(options).table(tableName);\n  }\n\n  protected void refreshSparkCache(Identifier ident, Table table) {\n    CacheManager cacheManager = spark.sharedState().cacheManager();\n    DataSourceV2Relation relation =\n        DataSourceV2Relation.create(table, Option.apply(tableCatalog), Option.apply(ident));\n    cacheManager.recacheByPlan(spark, relation);\n  }\n\n  protected Expression filterExpression(Identifier ident, String where) {\n    try {\n      String name = Spark3Util.quotedFullIdentifier(tableCatalog.name(), ident);\n      org.apache.spark.sql.catalyst.expressions.Expression expression =\n          SparkExpressionConverter.collectResolvedSparkExpression(spark, name, where);\n      return SparkExpressionConverter.convertToIcebergExpression(expression);\n    } catch (AnalysisException e) {\n      throw new IllegalArgumentException(\"Cannot parse predicates in where option: \" + where, e);\n    }\n  }\n\n  protected InternalRow newInternalRow(Object... values) {\n    return new GenericInternalRow(values);\n  }\n\n  protected abstract static class Builder<T extends BaseProcedure> implements ProcedureBuilder {\n    private TableCatalog tableCatalog;\n\n    @Override\n    public Builder<T> withTableCatalog(TableCatalog newTableCatalog) {\n      this.tableCatalog = newTableCatalog;\n      return this;\n    }\n\n    @Override\n    public T build() {","sourceCodeStart":164,"sourceCodeEnd":200,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/procedures/BaseProcedure.java#L164-L200","documentation":"Procedures accepting a `where` option resolve it by collecting a resolved Spark expression through SQL analysis. If analysis throws AnalysisException, BaseProcedure rethrows it as an IllegalArgumentException saying the predicates in the where option cannot be parsed, including the offending `where` string.","triggerScenarios":"Calling a procedure with `where => \"...\"` containing invalid SQL, unknown columns, unresolved functions, or dialect-mismatched expressions.","commonSituations":"Referencing columns not in the table; using non-Spark functions; double-quoted string literals; typos in column names; predicates copied from other SQL engines.","solutions":["Test the predicate standalone: `SELECT count(*) FROM tbl WHERE <predicate>` to confirm it analyzes.","Use single quotes for string literals and valid Spark SQL syntax.","Verify column names/types with DESCRIBE TABLE before building the predicate.","Catch IllegalArgumentException around the CALL and surface the where string for correction."],"exampleFix":"// before\nCALL cat.sys.expire_snapshots(table => 't', where => \"ts > \\\"2024-01-01\\\"\")\n// after\nCALL cat.sys.expire_snapshots(table => 't', where => \"ts > '2024-01-01'\")","handlingStrategy":"validation","validationCode":"// pre-flight: ensure the predicate analyzes against the table\nspark.sql(\"SELECT 1 FROM \" + fullTableName + \" WHERE \" + where).limit(1).collect();","typeGuard":null,"tryCatchPattern":"try { spark.sql(\"CALL cat.sys.expire_snapshots(table => 't', where => \\\"ts > '2024-01-01'\\\")\"); } catch (IllegalArgumentException e) { if (e.getMessage().startsWith(\"Cannot parse predicates in where option\")) { /* fix the where SQL */ } throw e; }","preventionTips":["Test the predicate in a standalone SELECT before using it in a procedure.","Use single quotes for string literals in Spark SQL.","Validate column names/types with DESCRIBE TABLE first.","Avoid dialect-specific syntax from other SQL engines."],"tags":["spark","procedures","sql-parsing","predicates"],"backgroundTag":"invalid-argument-format","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}