{"record":{"id":"8e3ce9701c2c5284","repo":"apache/iceberg","slug":"failed-to-find-filter-expression-8e3ce9","errorCode":null,"errorMessage":"Failed to find filter expression","messagePattern":"Failed to find filter expression","errorType":"exception","errorClass":"IcebergAnalysisException","httpStatus":null,"severity":"error","filePath":"spark/v4.1/spark/src/main/scala/org/apache/spark/sql/execution/datasources/SparkExpressionConverter.scala","lineNumber":69,"sourceCode":"    }\n  }\n\n  @throws[IcebergAnalysisException]\n  def collectResolvedSparkExpression(\n      session: SparkSession,\n      tableName: String,\n      where: String): Expression = {\n    val tableAttrs = session.table(tableName).queryExecution.analyzed.output\n    val unresolvedExpression = session.sessionState.sqlParser.parseExpression(where)\n    val filter = Filter(unresolvedExpression, DummyRelation(tableAttrs))\n    val optimizedLogicalPlan = session.sessionState.executePlan(filter).optimizedPlan\n    optimizedLogicalPlan\n      .collectFirst {\n        case filter: Filter => filter.condition\n        case _: DummyRelation => Literal.TrueLiteral\n        case _: LocalRelation => Literal.FalseLiteral\n      }\n      .getOrElse(throw new IcebergAnalysisException(\"Failed to find filter expression\"))\n  }\n\n  case class DummyRelation(output: Seq[Attribute]) extends LeafNode\n}\n","sourceCodeStart":51,"sourceCodeEnd":74,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.1/spark/src/main/scala/org/apache/spark/sql/execution/datasources/SparkExpressionConverter.scala#L51-L74","documentation":"collectResolvedSparkExpression scans the optimized logical plan for the Filter condition that Spark wants Iceberg to explain/convert, and throws IcebergAnalysisException when no Filter node is found. The helper is only valid for plans that contain exactly one filter; a plan without a Filter (or DummyRelation/LocalRelation stand-in) means there is no resolvable filter expression to collect.","triggerScenarios":"Invoking collectResolvedSparkExpression (used by procedures like the extended filter-resolution path, e.g. rewrite/expire procedures that take a filter) with an optimized plan that has no Filter operator — e.g. the input DataFrame has no WHERE clause, or the optimizer rewrote/eliminated the filter before this point.","commonSituations":"Passing a DataFrame with no predicate to a procedure expecting a filtered plan; the filter being pushed below/away by Catalyst optimization (constant-folding a FALSE predicate); wiring the helper against a plan produced from a different query.","solutions":["Ensure the DataFrame passed in actually has a WHERE/Filter condition that survives to the optimized plan.","Verify the plan via df.queryExecution.optimizedPlan before calling; if the filter was optimized away (e.g. Literal.TrueLiteral), skip the call instead of invoking the resolver.","If you intentionally want a trivially-true/false result, model your input with the corresponding case (DummyRelation / LocalRelation) rather than a plan with no filter.","Update to a newer Iceberg release if your Spark optimizer legitimately produces a Filter shape this collector doesn't recognize."],"exampleFix":"// before\nval filtered = spark.table(\"t\") // no filter -> no Filter node in plan\nprocedure.collectResolvedSparkExpression(filtered.queryExecution)\n// after\nval filtered = spark.table(\"t\").where($\"ts\" < lit(\"2024-01-01\"))","handlingStrategy":"validation","validationCode":"val hasFilter = df.queryExecution.optimizedPlan.exists {\n  case _: org.apache.spark.sql.catalyst.plans.logical.Filter => true\n  case _ => false\n}\nrequire(hasFilter, \"Plan must contain a Filter node before collecting the resolved expression\")","typeGuard":"def planHasFilter(plan: LogicalPlan): Boolean = plan.exists {\n  case _: Filter => true\n  case _: DummyRelation | _: LocalRelation => true\n  case _ => false\n}","tryCatchPattern":null,"preventionTips":["Always add a WHERE clause to DataFrames passed to filter-resolving procedures","Check optimizedPlan before invoking helpers that require a Filter","Remember constant-folded filters may disappear from the optimized plan","Guard procedure calls with a check that the predicate survives optimization"],"tags":["spark","logical-plan","missing-filter"],"backgroundTag":"empty-result-set","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}