apache/beam · error · RuntimeException

Unable run pipeline with EMPTY disposition.

Error message

Unable run pipeline with EMPTY disposition.

What it means

This wraps a SQLException raised while evaluating whether the target table is empty for the EMPTY create-disposition check in SnowflakeBatchServiceImpl. It means the row-count query itself failed (connection problem, bad query/permissions), not that the table is non-empty.

Source

Thrown at sdks/java/io/snowflake/src/main/java/org/apache/beam/sdk/io/snowflake/services/SnowflakeBatchServiceImpl.java:182

      throws SQLException {
    String selectQuery = String.format("SELECT count(*) FROM %s LIMIT 1;", tablePath);
    runConnectionWithStatement(
        dataSource,
        selectQuery,
        resultSet -> {
          assert resultSet != null;
          checkIfTableIsEmpty(resultSet);
        });
  }

  private static void checkIfTableIsEmpty(ResultSet resultSet) {
    int columnId = 1;
    try {
      if (!resultSet.next() || !checkIfTableIsEmpty(resultSet, columnId)) {
        throw new RuntimeException("Table is not empty. Aborting COPY with disposition EMPTY");
      }
    } catch (SQLException e) {
      throw new RuntimeException("Unable run pipeline with EMPTY disposition.", e);
    }
  }

  private static boolean checkIfTableIsEmpty(ResultSet resultSet, int columnId)
      throws SQLException {
    int rowCount = resultSet.getInt(columnId);
    if (rowCount >= 1) {
      return false;
    }
    return true;
  }

  private void prepareTableAccordingCreateDisposition(
      DataSource dataSource,
      String database,
      String schema,
      String table,
      SnowflakeTableSchema tableSchema,

View on GitHub (pinned to 12126d8942)

Solutions

1. Inspect the wrapped SQLException cause for the actual Snowflake error (auth, network, SQL).
2. Retry the pipeline; Snowflake transient connection errors (codes 390100/250001 style) often resolve on retry.
3. Verify Snowflake credentials, warehouse, and network egress from the worker (VPC/firewall/proxy).
4. Update the Snowflake JDBC driver to a current version.

Example fix

// before: unguarded pipeline run
pipeline.run();
// after: retry on transient failures
// wrap the pipeline execution in a retry loop for transient Snowflake errors
// and check e.getCause() instanceof java.sql.SQLException for the root cause

// Pre-flight connectivity check
// try (Connection c = dataSource.getConnection()) { c.createStatement().execute("SELECT 1"); }

// catch RuntimeException, inspect cause
try { pipeline.run().waitUntilFinish(); }
catch (RuntimeException e) {
  if (e.getCause() instanceof java.sql.SQLException sqlEx && isTransient(sqlEx.getErrorCode())) retry();
  else throw e;
}

When it happens

Trigger: resultSet.next() or the recursive checkIfTableIsEmpty(resultSet, columnId) throws SQLException — e.g. connectivity loss to Snowflake, invalid result metadata, or a closed result set during the EMPTY-disposition preflight.

Common situations: Transient Snowflake connection drops; expired credentials or session timeouts mid-check; warehouse suspend/resume races; driver version issues.

Understand the failure class

Background: "query failed", "%w: SQL error" — wrapped database query errors in Go libraries explained — this error's family across 3 libraries.

Related errors

• Unable run pipeline with CREATE IF NEEDED disposition.
• Beam JDBC connection has not been initialized
• Unable to parse statement
• Failed to infer Beam schema
• Error while determining columns from table: ${getTable()}

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/0e975e3f8f71cd54. Report an issue: GitHub.