{"record":{"id":"c84861ead3ebffbc","repo":"apache/beam","slug":"s-does-not-support-projection-pushdown","errorCode":null,"errorMessage":"%s does not support projection pushdown.","messagePattern":"(.+?) does not support projection pushdown\\.","errorType":"exception","errorClass":"UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/SchemaIOTableProviderWrapper.java","lineNumber":148,"sourceCode":"      if (!(filters instanceof DefaultTableFilter)) {\n        throw new UnsupportedOperationException(\n            String.format(\n                \"Filter pushdown is not yet supported in %s. https://github.com/apache/beam/issues/21001\",\n                SchemaIOTableWrapper.class));\n      }\n      if (!fieldNames.isEmpty()) {\n        if (readerTransform instanceof ProjectionProducer) {\n          // The pushdown must return a PTransform that can be applied to a PBegin, or this cast\n          // will fail.\n          ProjectionProducer<PTransform<PBegin, PCollection<Row>>> projectionProducer =\n              (ProjectionProducer<PTransform<PBegin, PCollection<Row>>>) readerTransform;\n          FieldAccessDescriptor fieldAccessDescriptor =\n              FieldAccessDescriptor.withFieldNames(fieldNames);\n          readerTransform =\n              projectionProducer.actuateProjectionPushdown(\n                  ImmutableMap.of(new TupleTag<PCollection<Row>>(\"output\"), fieldAccessDescriptor));\n        } else {\n          throw new UnsupportedOperationException(\n              String.format(\"%s does not support projection pushdown.\", this.getClass()));\n        }\n      }\n      return begin.apply(readerTransform);\n    }\n\n    @Override\n    public ProjectSupport supportsProjects() {\n      PTransform<PBegin, PCollection<Row>> readerTransform = schemaIO.buildReader();\n      if (readerTransform instanceof ProjectionProducer) {\n        if (((ProjectionProducer<?>) readerTransform).supportsProjectionPushdown()) {\n          // For ProjectionProducer, supportsProjectionPushdown implies field reordering support.\n          return ProjectSupport.WITH_FIELD_REORDERING;\n        }\n      }\n      return ProjectSupport.NONE;\n    }\n","sourceCodeStart":130,"sourceCodeEnd":166,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/SchemaIOTableProviderWrapper.java#L130-L166","documentation":"Beam SQL's SchemaIO table wrapper throws this UnsupportedOperationException when a query tries to push a column projection (SELECT of specific fields) down into an IO connector whose underlying transform cannot consume a FieldAccessDescriptor. The library only pushes projections when the provider's projectionProducer can actuate them; otherwise it falls into the else branch and refuses rather than silently ignoring the pushdown request.","triggerScenarios":"Running a Beam SQL query against a table backed by SchemaIOTableProviderWrapper (e.g. via schema:// or other SchemaIO providers) where the SELECT statement yields field names that get wrapped in FieldAccessDescriptor.withFieldNames, and the resolved transform's projectionProducer reports it cannot actuate projection pushdown, so buildIOReader throws.","commonSituations":"Querying a SchemaIO-backed table with a column-selecting SQL statement (not SELECT *) against a connector (some file-based or third-party SchemaIO implementations) that does not implement projection pushdown; upgrading Beam or switching providers and expecting pushdown that the target IO never supported.","solutions":["Use SELECT * or select all columns for this table so no projection pushdown is attempted","Check the SchemaIO implementation's supportsProjectionPushdown/actuateProjectionPushdown support before relying on column pruning","Read only the needed fields after the PCollection is produced (project downstream in Java) instead of at the SQL level","Implement/enable projection pushdown in the custom SchemaIO's projectionProducer if pruning matters for performance"],"exampleFix":"// before\nSELECT col_a FROM schema:`path`.table\n// after (provider lacks pushdown support)\nSELECT * FROM schema:`path`.table  // project fields downstream in pipeline code","handlingStrategy":"validation","validationCode":"// before querying\nif (!schemaIO.projectionProducer().map(p -> p.supportsProjectionPushdown()).orElse(false)\n    && !selectsAllColumns(sql)) {\n  throw new IllegalStateException(\"provider does not support projection pushdown; use SELECT * or project downstream\");\n}","typeGuard":null,"tryCatchPattern":"try { table.eval(...) } catch (UnsupportedOperationException e) { // fall back to SELECT * / full read\n  return fullReadAndProject(sql);\n}","preventionTips":["Verify the SchemaIO implementation supports projection pushdown before writing column-pruning queries","Prefer projecting fields in pipeline code (Select/apply) for connectors without pushdown","Add an integration test per provider that exercises the SQL shape you ship"],"tags":["beam-sql","unsupported-operation","projection-pushdown"],"backgroundTag":"unsupported-operation","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}