apache/beam · error · UnsupportedOperationException

Writing to Delta Lake tables is currently not supported.

Error message

Writing to Delta Lake tables is currently not supported.

What it means

DeltaTable.buildIOWriter is unimplemented because no Delta Lake sink exists in Beam. Any attempt to write pipeline output into a Delta table via Beam SQL (INSERT INTO a Delta table) throws UnsupportedOperationException unconditionally.

Solutions

Do not write via Beam SQL to Delta Lake; write to an intermediate sink and publish to Delta with Delta Lake tooling (e.g. Spark, delta-rs).
Use a provider with sink support (e.g. Iceberg, BigQuery) if writes from Beam are required.
Track Beam's Delta Lake sink work and upgrade when the TODO is resolved.

Example fix

// before
INSERT INTO delta_t SELECT * FROM csv_t; -- unsupported
// after
-- write csv_t to a supported sink, then append to delta_t via Delta Lake tooling

Defensive patterns

Strategy: validation

Validate before calling

if (ioTarget instanceof DeltaTable) {
  throw new IllegalArgumentException("Delta Lake tables cannot be written to from Beam SQL; use a supported sink");
}

Try / catch

try {
  output = sqlTransform.apply(input); // INSERT INTO delta table
} catch (UnsupportedOperationException e) {
  if (e.getMessage().contains("Writing to Delta Lake")) {
    // reroute writes to a supported sink and publish to Delta externally
  }
}

Prevention

Treat Delta tables in Beam SQL as read-only sources.
Route writes to Iceberg/BigQuery or other sink-capable providers.
Publish to Delta Lake with external tooling (Spark, delta-rs) after Beam writes.

When it happens

Trigger: Executing INSERT/CREATE TABLE AS SELECT against a Delta table in Beam SQL, or otherwise invoking buildIOWriter on a DeltaTable instance.

Common situations: Assuming Delta tables are writable like Iceberg/BigQuery tables in Beam SQL; migrating pipelines from other providers to Delta Lake.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/00cdd52f9253417e. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/extensions/sql/delta/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/delta/DeltaTable.java:169

    // / Managed Delta
    // Lake source.
    String error = "%s does not support predicate/project push-down, yet non-empty %s is passed.";
    if (!(filters instanceof DefaultTableFilter)) {
      throw new UnsupportedOperationException(
          String.format(error, this.getClass().getName(), "'filters'"));
    }
    if (!fieldNames.isEmpty()) {
      throw new UnsupportedOperationException(
          String.format(error, this.getClass().getName(), "'fieldNames'"));
    }
    return buildIOReader(begin);
  }

  @Override
  public POutput buildIOWriter(PCollection<Row> input) {
    // TODO: Support writing to Delta Lake tables once a Delta Lake sink is
    // available.
    throw new UnsupportedOperationException(
        "Writing to Delta Lake tables is currently not supported.");
  }

  @Override
  public PCollection.IsBounded isBounded() {
    return PCollection.IsBounded.BOUNDED;
  }

  @Override
  public BeamSqlTableFilter constructFilter(List<RexNode> filter) {
    // TODO: Support predicate pushdown when supported by DeltaIO / Managed Delta
    // Lake source.
    return new DefaultTableFilter(filter);
  }

  private Map<String, Object> getBaseConfig() {
    ImmutableMap.Builder<String, Object> managedConfigBuilder = ImmutableMap.builder();
    managedConfigBuilder.put("table", tableLocation);

View on GitHub (pinned to 12126d8942)