apache/iceberg · warning

Failed to check if can be pushed down

Error message

Failed to check if {} can be pushed down: {}

What it means

During scan planning SparkScanBuilder.pushPredicates evaluates each Spark filter to decide whether it can be translated to an Iceberg Predicate and pushed into the scan. If translation/evaluation throws for any reason, this warning is logged and the filter is kept as a post-scan (residual) filter instead — the query still runs correctly, just without pushdown.

Solutions

1. Read the logged exception message to identify which predicate failed and why.
2. Rewrite the offending filter using supported expressions/operators that Iceberg can push down.
3. Check Iceberg version — upgrade if your Spark functions were added after the connector's converter support.
4. Verify data source is still correct: the filter is applied post-scan, so results are right but may read more data; check scanned-file metrics to assess impact.

Example fix

// before: unpushable filter keeps whole-table scan
df.filter(callUDF("myfunc", col("ts")).gt(lit(0)))

// after: pushable native predicate
long ts = ...;
df.filter(functions.col("ts").gt(functions.lit(ts)));

// Pre-check whether a filter is pushable
Expression expr = SparkExpressionConverter.convertToExpression(table.schema(), sparkPredicate); // throws if unsupported
if (expr == null) {
  LOG.info("Filter {} will not be pushed down; consider rewriting", sparkPredicate);
}

try {
  converter.convertToExpression(schema, sparkPredicate);
} catch (Exception e) {
  LOG.warn("Unpushable filter, keeping as post-scan: {}", sparkPredicate, e);
}

When it happens

Trigger: Calling a DataFrame/SQL query with a filter expression that SparkExpressionConverter cannot map (e.g. unusual UDF-derived functions, non-deterministic expressions, struct/map edge cases, incompatible type casts) while Iceberg attempts pushdown in pushPredicates.

Common situations: Queries with custom Catalyst expressions or third-party functions on Iceberg tables; version mismatches between Spark and Iceberg where expression converters lag new Spark functions; filters over columns whose types don't match the schema after evolution.

Related errors

• Failed to check if can be pushed down
• Cannot convert Spark filter: $filter to Iceberg expression
• Cannot translate Spark expression: $sparkExpression to data…
• Failed to check if can be pushed down
• Failed to close task iterable

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/c3274f4b43287230. Report an issue: GitHub.

Appendix: source

Thrown at spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkScanBuilder.java:186

        Expression expr = SparkV2Filters.convert(predicate);

        if (expr != null) {
          // try binding the expression to ensure it can be pushed down
          Binder.bind(schema.asStruct(), expr, caseSensitive);
          expressions.add(expr);
          pushableFilters.add(predicate);
        }

        if (expr == null
            || unpartitioned()
            || !ExpressionUtil.selectsPartitions(expr, table, caseSensitive)) {
          postScanFilters.add(predicate);
        } else {
          LOG.info("Evaluating completely on Iceberg side: {}", predicate);
        }

      } catch (Exception e) {
        LOG.warn("Failed to check if {} can be pushed down: {}", predicate, e.getMessage());
        postScanFilters.add(predicate);
      }
    }

    this.filterExpressions = expressions;
    this.pushedPredicates = pushableFilters.toArray(new Predicate[0]);

    return postScanFilters.toArray(new Predicate[0]);
  }

  private boolean unpartitioned() {
    return table.specs().values().stream().noneMatch(PartitionSpec::isPartitioned);
  }

  @Override
  public Predicate[] pushedPredicates() {
    return pushedPredicates;
  }

View on GitHub (pinned to 86d9c8fc54)