apache/iceberg · warning

Failed to bind to expected schema, skipping runtime filter

Error message

Failed to bind {} to expected schema, skipping runtime filter

What it means

During scan planning Spark may pass runtime (dynamic) filters down; SparkBatchQueryScan attempts to convert each Spark V2 predicate to an Iceberg expression and bind it to the expected schema. If binding fails with ValidationException, the filter cannot be applied safely on Iceberg's side, so it is logged and skipped — the query still runs, just without that runtime filter optimization.

Solutions

1. No action strictly required — the filter is skipped and correctness is preserved.
2. Check the referenced expression against the table schema; align column names/types if you want the filter applied.
3. Verify case sensitivity settings (spark.sql.caseSensitive) match the filter's column casing.

// ensure the runtime filter column exists and type-compatible before relying on pushdown
boolean columnInSchema = expectedSchema.columns().stream()
    .anyMatch(c -> c.name().equalsIgnoreCase(filterColumn));

boolean bindable = expr != null && expectedSchema.caseInsensitiveFindField(filterColumn) != null;

try { Binder.bind(expectedSchema.asStruct(), expr, caseSensitive()); } catch (ValidationException e) { /* skip filter, rely on post-scan evaluation */ }

When it happens

Trigger: A Spark runtime filter (e.g. from a bloom filter or dynamic pruning) references a column or type not present/compatible with the scan's expectedSchema, causing Binder.bind to throw ValidationException.

Common situations: Runtime bloom filters built on partition or metadata columns absent from the projected schema; schema evolution making a runtime filter's type incompatible; case-sensitivity mismatches.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Failed to bind to expected schema, skipping runtime filter
• Unsupported runtime filter
• Cannot add column since setting default values in Spark is…
• Cannot add column since setting default values in Spark is…
• Cannot convert unsupported type to Spark

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/2fc9a823ffa19636. Report an issue: GitHub.

Appendix: source

Thrown at spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/source/SparkBatchQueryScan.java:205

      return deleteFile.content() != FileContent.EQUALITY_DELETES;
    }

    return ContentFileUtil.isFileScoped(deleteFile);
  }

  // at this moment, Spark can only pass IN filters for a single attribute
  // if there are multiple filter attributes, Spark will pass two separate IN filters
  private Expression convertRuntimeFilters(Predicate[] predicates) {
    Expression runtimeFilterExpr = Expressions.alwaysTrue();

    for (Predicate predicate : predicates) {
      Expression expr = SparkV2Filters.convert(predicate);
      if (expr != null) {
        try {
          Binder.bind(expectedSchema().asStruct(), expr, caseSensitive());
          runtimeFilterExpr = Expressions.and(runtimeFilterExpr, expr);
        } catch (ValidationException e) {
          LOG.warn("Failed to bind {} to expected schema, skipping runtime filter", expr, e);
        }
      } else {
        LOG.warn("Unsupported runtime filter {}", predicate);
      }
    }

    return runtimeFilterExpr;
  }

  @Override
  public Statistics estimateStatistics() {
    if (scan() == null) {
      return estimateStatistics(null);

    } else if (snapshotId != null) {
      Snapshot snapshot = table().snapshot(snapshotId);
      return estimateStatistics(snapshot);

View on GitHub (pinned to 86d9c8fc54)