apache/beam · error · IllegalStateException

Table ' ' already exists. It should have been cleaned up by…

Error message

Table '${tableReference}' already exists. It should have been cleaned up by the test rule.

What it means

TestBigQuery.createTable verifies that the target table does not already exist before creating it, because the rule guarantees automatic cleanup after each test. A pre-existing table means cleanup from a previous run failed, so it throws IllegalStateException naming the table reference.

Solutions

Delete the leftover table (or dataset) in BigQuery before re-running the test
Make table names unique per test (random suffix) to avoid collisions
Check why the previous test's cleanup failed (exception, killed process) and fix it
Run against an ephemeral dataset that the rule fully owns and deletes afterward
Avoid running such tests in parallel against the same project/dataset

Example fix

// before
String table = "my_table"; // fixed name, collides across runs
// after
String table = "my_table_" + UUID.randomUUID().toString().replace("-", "");

Defensive patterns

Strategy: validation

Validate before calling

if (datasetService.getTable(tableReference) != null) {
  datasetService.deleteTable(tableReference); // or fail fast before the rule does
}

Try / catch

try {
  bigquery.createTable(tableReference, schema);
} catch (IllegalStateException e) {
  if (e.getMessage() != null && e.getMessage().contains("already exists")) {
    datasetService.deleteTable(tableReference); // clean orphan then retry once
  } else throw e;
}

Prevention

Use unique table names per test (UUID suffix)
Run tests against ephemeral datasets the rule owns
Fix root causes of failed prior-test cleanup (killed workers, swallowed exceptions)
Avoid parallel runs sharing one dataset

When it happens

Trigger: Calling initializeBigQuery/createTable when the dataset already contains the requested table — leftover from a previous test whose cleanup failed, a crashed JVM, or a fixed table name colliding across tests.

Common situations: Reused table names across tests, orphaned tables after a killed CI worker, running tests against a persistent dataset where cleanup was skipped, or parallel test executions sharing a dataset.

Understand the failure class

Background: "already exists" / EEXIST / FileAlreadyExistsException: what the 'file already exists' error means and how to fix it — this error's family across 37 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/9d912c0f171bdd5b. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/TestBigQuery.java:149

            .setProjectId(
                pipelineOptions.getBigQueryProject() == null
                    ? pipelineOptions.getProject()
                    : pipelineOptions.getBigQueryProject())
            .setDatasetId(pipelineOptions.getTargetDataset())
            .setTableId(createRandomizedName(description));

    Table newTable =
        new Table()
            .setTableReference(tableReference)
            .setSchema(BigQueryUtils.toTableSchema(schema))
            .setDescription(
                "Table created for "
                    + description.getDisplayName()
                    + " by TestBigQueryRule. "
                    + "Should be automatically cleaned up after test completion.");

    if (datasetService.getTable(tableReference) != null) {
      throw new IllegalStateException(
          "Table '"
              + tableReference
              + "' already exists. "
              + "It should have been cleaned up by the test rule.");
    }

    datasetService.createTable(newTable);
    table = newTable;
    return table;
  }

  private void tearDown() throws IOException, InterruptedException {
    if (this.datasetService == null) {
      return;
    }

    try {
      if (table != null) {

View on GitHub (pinned to 12126d8942)