apache/beam · error · UnsupportedOperationException

does not support projection pushdown.

Error message

%s does not support projection pushdown.

What it means

Beam SQL's SchemaIO table wrapper throws this UnsupportedOperationException when a query tries to push a column projection (SELECT of specific fields) down into an IO connector whose underlying transform cannot consume a FieldAccessDescriptor. The library only pushes projections when the provider's projectionProducer can actuate them; otherwise it falls into the else branch and refuses rather than silently ignoring the pushdown request.

Solutions

  1. Use SELECT * or select all columns for this table so no projection pushdown is attempted
  2. Check the SchemaIO implementation's supportsProjectionPushdown/actuateProjectionPushdown support before relying on column pruning
  3. Read only the needed fields after the PCollection is produced (project downstream in Java) instead of at the SQL level
  4. Implement/enable projection pushdown in the custom SchemaIO's projectionProducer if pruning matters for performance

Example fix

// before
SELECT col_a FROM schema:`path`.table
// after (provider lacks pushdown support)
SELECT * FROM schema:`path`.table  // project fields downstream in pipeline code
Defensive patterns

Strategy: validation

Validate before calling

// before querying
if (!schemaIO.projectionProducer().map(p -> p.supportsProjectionPushdown()).orElse(false)
    && !selectsAllColumns(sql)) {
  throw new IllegalStateException("provider does not support projection pushdown; use SELECT * or project downstream");
}

Try / catch

try { table.eval(...) } catch (UnsupportedOperationException e) { // fall back to SELECT * / full read
  return fullReadAndProject(sql);
}

Prevention

When it happens

Trigger: Running a Beam SQL query against a table backed by SchemaIOTableProviderWrapper (e.g. via schema:// or other SchemaIO providers) where the SELECT statement yields field names that get wrapped in FieldAccessDescriptor.withFieldNames, and the resolved transform's projectionProducer reports it cannot actuate projection pushdown, so buildIOReader throws.

Common situations: Querying a SchemaIO-backed table with a column-selecting SQL statement (not SELECT *) against a connector (some file-based or third-party SchemaIO implementations) that does not implement projection pushdown; upgrading Beam or switching providers and expecting pushdown that the target IO never supported.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/c84861ead3ebffbc. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/SchemaIOTableProviderWrapper.java:148

      if (!(filters instanceof DefaultTableFilter)) {
        throw new UnsupportedOperationException(
            String.format(
                "Filter pushdown is not yet supported in %s. https://github.com/apache/beam/issues/21001",
                SchemaIOTableWrapper.class));
      }
      if (!fieldNames.isEmpty()) {
        if (readerTransform instanceof ProjectionProducer) {
          // The pushdown must return a PTransform that can be applied to a PBegin, or this cast
          // will fail.
          ProjectionProducer<PTransform<PBegin, PCollection<Row>>> projectionProducer =
              (ProjectionProducer<PTransform<PBegin, PCollection<Row>>>) readerTransform;
          FieldAccessDescriptor fieldAccessDescriptor =
              FieldAccessDescriptor.withFieldNames(fieldNames);
          readerTransform =
              projectionProducer.actuateProjectionPushdown(
                  ImmutableMap.of(new TupleTag<PCollection<Row>>("output"), fieldAccessDescriptor));
        } else {
          throw new UnsupportedOperationException(
              String.format("%s does not support projection pushdown.", this.getClass()));
        }
      }
      return begin.apply(readerTransform);
    }

    @Override
    public ProjectSupport supportsProjects() {
      PTransform<PBegin, PCollection<Row>> readerTransform = schemaIO.buildReader();
      if (readerTransform instanceof ProjectionProducer) {
        if (((ProjectionProducer<?>) readerTransform).supportsProjectionPushdown()) {
          // For ProjectionProducer, supportsProjectionPushdown implies field reordering support.
          return ProjectSupport.WITH_FIELD_REORDERING;
        }
      }
      return ProjectSupport.NONE;
    }

View on GitHub (pinned to 12126d8942)