apache/beam · error · RuntimeException

Table is not empty. Aborting COPY with disposition EMPTY

Error message

Table is not empty. Aborting COPY with disposition EMPTY

What it means

SnowflakeBatchServiceImpl checks a table's row count before running a COPY when the createDisposition is EMPTY; if the query indicates the table already contains rows, the pipeline aborts with this RuntimeException. The EMPTY disposition promises that data lands only in a fresh table, and Beam refuses to silently append to a populated one.

Solutions

1. Truncate or drop the target table before the run: DROP TABLE <table> (or TRUNCATE TABLE <table>) in Snowflake.
2. Change createDisposition to AUTO or FAIL so Beam does not require an empty table.
3. Point the write at a different, empty table name for this run.
4. If no data exists but the error still fires, verify the table-existence query's result set shape matches what checkIfTableIsEmpty expects (first column = row count).

Example fix

// before
DisposalConfig: .withCreateDisposition(SnowflakeIO.Write.CreateDisposition.EMPTY)
// after
// ensure the table is empty first, or:
.withCreateDisposition(SnowflakeIO.Write.CreateDisposition.AUTO)

// Run before the pipeline: confirm the target table is empty when using EMPTY disposition
// SELECT COUNT(*) AS cnt FROM <db>.<schema>.<table>;
// if (cnt > 0) throw new IllegalStateException("Target table must be empty for CreateDisposition.EMPTY");

When it happens

Trigger: Using SnowflakeIO.write with createDisposition=EMPTY while the target Snowflake table already exists and has at least one row; the result set of the table-existence/row-count query returns a non-empty table or no row at all (resultSet.next() false also triggers it).

Common situations: Re-running a pipeline against a table populated by a previous run; forgetting to drop/truncate the staging table between runs; pointing EMPTY disposition at an existing production table by mistake.

Related errors

• Unable run pipeline with CREATE IF NEEDED - no response.
• Unable run pipeline with EMPTY disposition.
• Unable to create table.
• Attempting to create database
• Attempting to create database

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/7ae1ef3fa2da5dc5. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/snowflake/src/main/java/org/apache/beam/sdk/io/snowflake/services/SnowflakeBatchServiceImpl.java:179

  }

  private static void checkIfTableIsEmpty(DataSource dataSource, String tablePath)
      throws SQLException {
    String selectQuery = String.format("SELECT count(*) FROM %s LIMIT 1;", tablePath);
    runConnectionWithStatement(
        dataSource,
        selectQuery,
        resultSet -> {
          assert resultSet != null;
          checkIfTableIsEmpty(resultSet);
        });
  }

  private static void checkIfTableIsEmpty(ResultSet resultSet) {
    int columnId = 1;
    try {
      if (!resultSet.next() || !checkIfTableIsEmpty(resultSet, columnId)) {
        throw new RuntimeException("Table is not empty. Aborting COPY with disposition EMPTY");
      }
    } catch (SQLException e) {
      throw new RuntimeException("Unable run pipeline with EMPTY disposition.", e);
    }
  }

  private static boolean checkIfTableIsEmpty(ResultSet resultSet, int columnId)
      throws SQLException {
    int rowCount = resultSet.getInt(columnId);
    if (rowCount >= 1) {
      return false;
    }
    return true;
  }

  private void prepareTableAccordingCreateDisposition(
      DataSource dataSource,
      String database,

View on GitHub (pinned to 12126d8942)