apache/beam · error · IOException

Error checking whether table %s exists

Error message

Error checking whether table %s exists

What it means

BigtableServiceFactory.checkTableExists probes the Cloud Bigtable admin API to see if a table exists. If the lookup itself fails with a gRPC ApiException whose status code is not NOT_FOUND, the API error is wrapped in an IOException with this message and rethrown, since the pipeline can no longer tell whether the table exists.

Source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableServiceFactory.java:205

    BigtableOptions effectiveOptions = getEffectiveOptions(config);
    if (effectiveOptions != null) {
      config = BigtableConfigTranslator.translateToBigtableConfig(config, effectiveOptions);
    }

    if (config.isDataAccessible()) {
      BigtableDataSettings settings =
          BigtableConfigTranslator.translateToVeneerSettings(config, pipelineOptions);

      try (BigtableDataClient client = BigtableDataClient.create(settings)) {
        try {
          client.readRow(tableId, "non-exist-row");
        } catch (ApiException e) {
          if (e.getStatusCode().getCode() == GrpcStatusCode.Code.NOT_FOUND) {
            return false;
          }
          String message = String.format("Error checking whether table %s exists", tableId);
          LOG.error("Error checking whether table {} exists", tableId, e);
          throw new IOException(message, e);
        }
      }
    }
    return true;
  }

  @VisibleForTesting
  static boolean isEmpty() {
    synchronized (lock) {
      return entries.isEmpty();
    }
  }

  synchronized ConfigId newId() {
    return ConfigId.create();
  }

  private BigtableOptions getEffectiveOptions(BigtableConfig config) {

View on GitHub (pinned to 12126d8942)

Solutions

1. Verify the projectId, instanceId, and tableId are correct in your BigtableIO configuration
2. Grant the pipeline's service account the Bigtable Reader/Admin roles so it can query table metadata
3. Inspect the wrapped cause (ApiException) for the gRPC status code and address it (permissions, quotas, network)
4. Retry the job if the underlying ApiException was transient (UNAVAILABLE, DEADLINE_EXCEEDED)

Example fix

// before
new BigtableServiceFactory(options, instanceId, tableId); // throws if admin call fails
// after
// check permissions/config first
try {
  factory.tableExists(tableId);
} catch (IOException e) {
  LOG.error("Check cause ApiException status", e.getCause());
  throw new RuntimeException("Verify Bigtable project/instance config and IAM roles", e);
}

// before the pipeline
try (var admin = BigtableTableAdminClient.of(projectId, instanceId)) {
  admin.getTable(tableId); // surfaces permission/config errors early
}

try {
  factory.tableExists(tableId);
} catch (IOException e) {
  if (e.getCause() instanceof ApiException) {
    Code code = ((ApiException) e.getCause()).getStatusCode().getCode();
    // handle PERMISSION_DENIED / UNAVAILABLE distinctly
  }
}

When it happens

Trigger: Calling BigtableIO read/write path with a project/instance whose admin client cannot serve the table lookup for a reason other than NOT_FOUND — e.g. permission-denied on the instance, invalid project/instance id, or transient gRPC/network failures.

Common situations: Misconfigured project or instance IDs in pipeline options; the service account lacks bigtable.tables.get / bigtable.instances.list permissions; transient Cloud Bigtable outages or network issues during pipeline startup.

Related errors

• BulkMutation took too long to close
• Failed to close batch
• could not close data operations client: %v
• Failed to retrieve or create secret bytes
• Failed to retrieve secret bytes

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/516abba1421253e0. Report an issue: GitHub.