apache/beam · error · UnsupportedOperationException

%s does not support predicate/project push-down, yet non-emp

Error message

%s does not support predicate/project push-down, yet non-empty %s is passed.

What it means

DeltaTable.buildIOReader does not support predicate or projection push-down because DeltaIO/Managed Delta Lake does not expose them. If the planner passes a non-default table filter (indicating actual predicates) the table throws UnsupportedOperationException rather than silently ignoring the push-down.

Source

Thrown at sdks/java/extensions/sql/delta/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/delta/DeltaTable.java:155

    }
  }

  @Override
  public PCollection<Row> buildIOReader(PBegin begin) {
    return begin
        .apply(Managed.read(Managed.DELTA_LAKE).withConfig(getBaseConfig()))
        .getSinglePCollection();
  }

  @Override
  public PCollection<Row> buildIOReader(
      PBegin begin, BeamSqlTableFilter filters, List<String> fieldNames) {
    // TODO: Support predicate pushdown and column pruning when supported by DeltaIO
    // / Managed Delta
    // Lake source.
    String error = "%s does not support predicate/project push-down, yet non-empty %s is passed.";
    if (!(filters instanceof DefaultTableFilter)) {
      throw new UnsupportedOperationException(
          String.format(error, this.getClass().getName(), "'filters'"));
    }
    if (!fieldNames.isEmpty()) {
      throw new UnsupportedOperationException(
          String.format(error, this.getClass().getName(), "'fieldNames'"));
    }
    return buildIOReader(begin);
  }

  @Override
  public POutput buildIOWriter(PCollection<Row> input) {
    // TODO: Support writing to Delta Lake tables once a Delta Lake sink is
    // available.
    throw new UnsupportedOperationException(
        "Writing to Delta Lake tables is currently not supported.");
  }

  @Override

View on GitHub (pinned to 12126d8942)

Solutions

Rewrite the query so the unsupported predicate is evaluated after the scan, or simplify the WHERE clause.
Upgrade Beam — the TODO indicates push-down may land in a later DeltaIO/Managed version.
Wrap or materialize the filtering in a subsequent transform rather than relying on push-down.

Example fix

// before
SELECT * FROM delta_t WHERE custom_udf(x) = 1 -- non-default filter reaches reader
// after
SELECT * FROM delta_t -- then filter the resulting PCollection in your pipeline

Defensive patterns

Strategy: validation

Validate before calling

// ensure filters are only default (non-pushdown) filters before reading
if (!(filters instanceof DefaultTableFilter)) {
  throw new IllegalArgumentException("Delta provider supports only default (no-op) filters");
}

Type guard

boolean supportsPushdown(BeamSqlTableFilter filters) { return filters instanceof DefaultTableFilter; }

Try / catch

try {
  result = deltaTable.buildIOReader(begin, filters, fieldNames);
} catch (UnsupportedOperationException e) {
  // fall back to reading full table and filtering in the pipeline
  result = deltaTable.buildIOReader(begin);
}

Prevention

Avoid predicates the Delta provider cannot handle; filter after the scan.
Do not rely on push-down optimizations for Delta tables in Beam SQL.
Keep WHERE clauses simple and verify no custom filter chains reach the reader.

When it happens

Trigger: Querying a Delta table through Beam SQL with WHERE clauses that the planner could not fully evaluate locally, producing a filter object that is not a DefaultTableFilter, passed into buildIOReader.

Common situations: Complex predicates or user-defined functions in WHERE clauses on Delta tables; SQL queries whose filter chains retain non-default filter objects.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/e29a1f6e29689a29. Report an issue: GitHub.