{"record":{"id":"7ae1ef3fa2da5dc5","repo":"apache/beam","slug":"table-is-not-empty-aborting-copy-with-disposition-empty","errorCode":null,"errorMessage":"Table is not empty. Aborting COPY with disposition EMPTY","messagePattern":"Table is not empty\\. Aborting COPY with disposition EMPTY","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/snowflake/src/main/java/org/apache/beam/sdk/io/snowflake/services/SnowflakeBatchServiceImpl.java","lineNumber":179,"sourceCode":"  }\n\n  private static void checkIfTableIsEmpty(DataSource dataSource, String tablePath)\n      throws SQLException {\n    String selectQuery = String.format(\"SELECT count(*) FROM %s LIMIT 1;\", tablePath);\n    runConnectionWithStatement(\n        dataSource,\n        selectQuery,\n        resultSet -> {\n          assert resultSet != null;\n          checkIfTableIsEmpty(resultSet);\n        });\n  }\n\n  private static void checkIfTableIsEmpty(ResultSet resultSet) {\n    int columnId = 1;\n    try {\n      if (!resultSet.next() || !checkIfTableIsEmpty(resultSet, columnId)) {\n        throw new RuntimeException(\"Table is not empty. Aborting COPY with disposition EMPTY\");\n      }\n    } catch (SQLException e) {\n      throw new RuntimeException(\"Unable run pipeline with EMPTY disposition.\", e);\n    }\n  }\n\n  private static boolean checkIfTableIsEmpty(ResultSet resultSet, int columnId)\n      throws SQLException {\n    int rowCount = resultSet.getInt(columnId);\n    if (rowCount >= 1) {\n      return false;\n    }\n    return true;\n  }\n\n  private void prepareTableAccordingCreateDisposition(\n      DataSource dataSource,\n      String database,","sourceCodeStart":161,"sourceCodeEnd":197,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/snowflake/src/main/java/org/apache/beam/sdk/io/snowflake/services/SnowflakeBatchServiceImpl.java#L161-L197","documentation":"SnowflakeBatchServiceImpl checks a table's row count before running a COPY when the createDisposition is EMPTY; if the query indicates the table already contains rows, the pipeline aborts with this RuntimeException. The EMPTY disposition promises that data lands only in a fresh table, and Beam refuses to silently append to a populated one.","triggerScenarios":"Using SnowflakeIO.write with createDisposition=EMPTY while the target Snowflake table already exists and has at least one row; the result set of the table-existence/row-count query returns a non-empty table or no row at all (resultSet.next() false also triggers it).","commonSituations":"Re-running a pipeline against a table populated by a previous run; forgetting to drop/truncate the staging table between runs; pointing EMPTY disposition at an existing production table by mistake.","solutions":["Truncate or drop the target table before the run: DROP TABLE <table> (or TRUNCATE TABLE <table>) in Snowflake.","Change createDisposition to AUTO or FAIL so Beam does not require an empty table.","Point the write at a different, empty table name for this run.","If no data exists but the error still fires, verify the table-existence query's result set shape matches what checkIfTableIsEmpty expects (first column = row count)."],"exampleFix":"// before\nDisposalConfig: .withCreateDisposition(SnowflakeIO.Write.CreateDisposition.EMPTY)\n// after\n// ensure the table is empty first, or:\n.withCreateDisposition(SnowflakeIO.Write.CreateDisposition.AUTO)","handlingStrategy":"validation","validationCode":"// Run before the pipeline: confirm the target table is empty when using EMPTY disposition\n// SELECT COUNT(*) AS cnt FROM <db>.<schema>.<table>;\n// if (cnt > 0) throw new IllegalStateException(\"Target table must be empty for CreateDisposition.EMPTY\");","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Prefer CreateDisposition.AUTO unless you truly need a guaranteed-fresh table.","Add a pre-run TRUNCATE/DROP step in your workflow orchestration.","Use unique table names per run (e.g. suffix with date/time) to avoid collisions."],"tags":["snowflake","database","pipeline","precondition-failed"],"backgroundTag":"database-write-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}