What causes "Writing to HCatalog is not supported in Beam SQL" in apache/beam?

Write results with the HCatalog/Hive IO connector (HCatalogIO.write) instead of Beam SQL sinks.

apache/beam · error · UnsupportedOperationException

Writing to HCatalog is not supported in Beam SQL

Error message

Writing to HCatalog is not supported in Beam SQL

What it means

HCatalogTable implements the Beam SQL table interface as read-only: buildIOWriter, which would produce an output PTransform for writing rows, unconditionally throws UnsupportedOperationException. Beam SQL can read from HCatalog but not sink to it.

Solutions

Write results with the HCatalog/Hive IO connector (HCatalogIO.write) instead of Beam SQL sinks.
Write to a supported sink (e.g. text, BigQuery, parquet files) from Beam SQL.
Materialize the PCollection<Row> and apply HCatalogIO manually.

Example fix

// before
rows.apply(SqlTransform.query(
    "INSERT INTO hcatalog.db.tbl SELECT * FROM PCOLLECTION"));
// after
rows.apply(HCatalogIO.write()
    .withConfigProperties(hiveProps)
    .withDatabase("db")
    .withTable("tbl"));

Defensive patterns

Strategy: try-catch

Validate before calling

if (isSink && "hcatalog".equals(tableType)) {
  throw new IllegalArgumentException("Use HCatalogIO.write for HCatalog sinks");
}

Try / catch

try {
  beamSqlTable.buildIOWriter(rows);
} catch (UnsupportedOperationException e) {
  rows.apply(HCatalogIO.write().withDatabase(db).withTable(tbl));
}

Prevention

Never use hcatalog tables as INSERT INTO sinks in Beam SQL.
Sink results via HCatalogIO or another supported sink.
Restrict hcatalog usage in Beam SQL to read-only SELECTs.

When it happens

Trigger: Using an hcatalog table as a sink, e.g. INSERT INTO hcatalog.tbl SELECT ... or calling buildIOWriter directly on the BeamSqlTable.

Common situations: Trying to persist query results back into a Hive table; migrating pipelines that wrote to Hive via other mechanisms into Beam SQL.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/ce5273733136a596. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/extensions/sql/hcatalog/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/hcatalog/HCatalogTable.java:60

  public abstract String database();

  public abstract String table();

  @Override
  public PCollection<Row> buildIOReader(PBegin begin) {
    return begin.apply(
        "HCatalog-Read-" + database() + "-" + table(),
        HCatToRow.fromSpec(
            HCatalogIO.read()
                .withConfigProperties(config())
                .withDatabase(database())
                .withTable(table())));
  }

  @Override
  public POutput buildIOWriter(PCollection<Row> input) {
    throw new UnsupportedOperationException("Writing to HCatalog is not supported in Beam SQL");
  }

  @Override
  public PCollection.IsBounded isBounded() {
    return PCollection.IsBounded.BOUNDED;
  }

  @Override
  public Schema getSchema() {
    return schema();
  }

  static Builder builder() {
    return new AutoValue_HCatalogTable.Builder();
  }

  @AutoValue.Builder
  abstract static class Builder {

View on GitHub (pinned to 12126d8942)