apache/beam · error · RuntimeException

Error creating Data Catalog client

Error message

Error creating Data Catalog client

What it means

createDataCatalogClient builds a DataCatalogClient with custom retry settings; if client construction throws IOException (credentials unavailable, transport failure, bad endpoint), it is rethrown as RuntimeException("Error creating Data Catalog client", e). This happens at provider creation time, before any table lookup.

Source

Thrown at sdks/java/extensions/sql/datacatalog/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/datacatalog/DataCatalogTableProvider.java:185

          .lookupEntrySettings()
          .setRetryableCodes(
              ImmutableSet.of(Code.PERMISSION_DENIED, Code.DEADLINE_EXCEEDED, Code.UNAVAILABLE))
          .setRetrySettings(
              builder.lookupEntrySettings().getRetrySettings().toBuilder()
                  .setMaxRetryDelay(Duration.ofMinutes(1L))
                  .build());
      builder
          .updateEntrySettings()
          .setRetryableCodes(
              ImmutableSet.of(Code.PERMISSION_DENIED, Code.DEADLINE_EXCEEDED, Code.UNAVAILABLE))
          .setRetrySettings(
              builder.updateEntrySettings().getRetrySettings().toBuilder()
                  .setMaxRetryDelay(Duration.ofMinutes(1L))
                  .build());

      return DataCatalogClient.create(builder.build());
    } catch (IOException e) {
      throw new RuntimeException("Error creating Data Catalog client", e);
    }
  }

  private Table toCalciteTable(String tableName, Entry entry) {
    if (entry.getSchema().getColumnsCount() == 0) {
      throw new UnsupportedOperationException(
          "Entry doesn't have a schema. Please attach a schema to '"
              + tableName
              + "' in Data Catalog: "
              + entry.toString());
    }
    Schema schema = SchemaUtils.fromDataCatalog(entry.getSchema());

    Optional<Table.Builder> tableBuilder = tableFactory.tableBuilder(entry);
    if (!tableBuilder.isPresent()) {
      throw new UnsupportedOperationException(
          String.format(
              "Unsupported Data Catalog entry: %s",

View on GitHub (pinned to 12126d8942)

Solutions

1. Set up valid Application Default Credentials (GOOGLE_APPLICATION_CREDENTIALS or gcloud auth application-default login).
2. Verify network access to datacatalog.googleapis.com and that the Data Catalog API is enabled on the project.
3. Inspect the wrapped IOException cause for the specific transport/credential failure; check any custom endpoint/proxy configuration.

Example fix

// before
// no credentials configured
DataCatalogTableProvider provider = DataCatalogTableProvider.create(options);

// after (shell)
// export GOOGLE_APPLICATION_CREDENTIALS=/path/to/sa.json
DataCatalogTableProvider provider = DataCatalogTableProvider.create(options);

// fail fast before creating the provider
if (System.getenv("GOOGLE_APPLICATION_CREDENTIALS") == null
    && !new File(
        System.getProperty("user.home") + "/.config/gcloud/application_default_credentials.json")
        .exists()) {
  throw new IllegalStateException("No Google Application Default Credentials found");
}

try {
  DataCatalogTableProvider p = DataCatalogTableProvider.create(options);
} catch (RuntimeException e) {
  if (e.getMessage().contains("Error creating Data Catalog client")) {
    // inspect e.getCause() (IOException) for credential/transport failure
  }
}

When it happens

Trigger: Calling DataCatalogTableProvider.create(...) with an environment lacking valid Google credentials, network/firewall issues reaching datacatalog.googleapis.com, or misconfigured endpoints/proxy settings.

Common situations: Running outside GCP without GOOGLE_APPLICATION_CREDENTIALS set, service accounts missing Data Catalog API enablement, blocked egress in CI, or an expired/invalid credentials file.

Related errors

• Could not resolve table in Data Catalog: {tableName}
• Unable to obtain credential
• Could not resolve GCP project ID%s. Please specify 'project'
• GCP Authentication Extension not configured properly: %s not
• Unsupported format for BigQuery table path: '{linkedResource

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/3aeb8b429c9ebc5b. Report an issue: GitHub.