apache/iceberg · warning

Failed to check if can be pushed down

Error message

Failed to check if {} can be pushed down: {}

What it means

A WARN log in BaseSparkScanBuilder.pushPredicates emitted when evaluating whether a Spark filter can be converted/pushed to Iceberg throws an exception. The predicate is conservatively kept as a post-scan (post-download) filter, so correctness is preserved but pushdown is lost and the scan reads more data than necessary.

Solutions

1. Inspect the full exception via DEBUG logging or the code path and identify which predicate fails conversion
2. Simplify or rewrite the offending WHERE clause into supported Iceberg predicate forms (comparisons on primitive columns)
3. Upgrade Iceberg to a version with broader Spark filter conversion support
4. If acceptable, leave it: the filter is still applied post-scan, only performance is affected

Example fix

// before
SELECT * FROM t WHERE struct_col.field > 10
// after
SELECT * FROM t WHERE struct_col.field > 10 /* accept post-scan eval, or rewrite */
SELECT * FROM t WHERE plain_col > 10

// check predicate columns are simple primitives before pushing
sparkSession.sql("SELECT * FROM t WHERE col > 10").explain(true); // verify PushedFilters in plan

When it happens

Trigger: Calling SparkScanBuilder.pushFilters/pushPredicates with a filter whose Spark-to-Iceberg conversion or evaluation (e.g. SparkFilters.convert, Binder.bind, or supported-type checks) throws at runtime, such as exotic expressions, non-literal bounds, or unsupported types like nested struct fields in the filter.

Common situations: Queries with complex filters (UDF-derived literals, struct/array types, NaN or malformed literals) pushed into a Spark Iceberg read; newer Spark filter shapes not yet handled by the converter.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• Failed to check if can be pushed down
• Cannot convert Spark filter: $filter to Iceberg expression
• Cannot translate Spark expression: $sparkExpression to data…
• Failed to check if can be pushed down
• Failed to close changelog scan:

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/640f9ea29fea617c. Report an issue: GitHub.

Appendix: source

Thrown at spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/source/BaseSparkScanBuilder.java:176

    for (Predicate predicate : predicates) {
      try {
        Expression expr = SparkV2Filters.convert(predicate);

        if (expr != null) {
          // try binding the expression to ensure it can be pushed down
          Binder.bind(projection.asStruct(), expr, caseSensitive);
          expressions.add(expr);
          pushablePredicates.add(predicate);
        }

        if (expr == null || !ExpressionUtil.selectsPartitions(expr, table, caseSensitive)) {
          postScanPredicates.add(predicate);
        } else {
          LOG.info("Evaluating completely on Iceberg side: {}", predicate);
        }

      } catch (Exception e) {
        LOG.warn("Failed to check if {} can be pushed down: {}", predicate, e.getMessage());
        postScanPredicates.add(predicate);
      }
    }

    this.filters = expressions;
    this.pushedPredicates = pushablePredicates.toArray(new Predicate[0]);

    return postScanPredicates.toArray(new Predicate[0]);
  }

  // logic necessary for SupportsPushDownV2Filters
  public Predicate[] pushedPredicates() {
    return pushedPredicates;
  }

  // logic necessary for SupportsPushDownLimit
  public boolean pushLimit(int newLimit) {
    this.limit = newLimit;

View on GitHub (pinned to 86d9c8fc54)