apache/seatunnel · critical · BigQueryConnectorException

CLIENT_CREATE_FAILED

CLIENT_CREATE_FAILED

Error message

Failed to create BigQueryWriteClient

What it means

This BigQueryConnectorException with code CLIENT_CREATE_FAILED is thrown by BigQueryClientFactory.getWriteClient when BigQueryWriteClient.create(settings) fails with an IOException. The BigQueryWrite client requires gRPC channel setup and credential loading; failure typically means credentials could not be resolved or the transport settings are invalid. The resulting sink cannot obtain a write client, failing the whole task.

Source

Thrown at seatunnel-connectors-v2/connector-bigquery/src/main/java/org/apache/seatunnel/connectors/bigquery/client/BigQueryClientFactory.java:84

                BigQueryWriteClient bigQueryWriteClient = BigQueryWriteClient.create(settings);
                log.info("Created BigQueryWriteClient for emulator at {}", emulatorHost);

                return bigQueryWriteClient;
            }

            GoogleCredentials credentials = getCredentials(config);

            BigQueryWriteSettings.Builder settingsBuilder =
                    BigQueryWriteSettings.newBuilder()
                            .setCredentialsProvider(FixedCredentialsProvider.create(credentials));

            if (config.get(BigQuerySinkOptions.UNIVERSE_DOMAIN) != null) {
                settingsBuilder.setUniverseDomain(config.get(BigQuerySinkOptions.UNIVERSE_DOMAIN));
            }

            return BigQueryWriteClient.create(settingsBuilder.build());
        } catch (IOException e) {
            throw new BigQueryConnectorException(
                    BigQueryConnectorErrorCode.CLIENT_CREATE_FAILED,
                    "Failed to create BigQueryWriteClient",
                    e);
        }
    }

    public static BigQuery getBigQuery(ReadonlyConfig config) {
        String projectId = config.get(BigQuerySinkOptions.PROJECT_ID);
        if (config.get(BigQuerySinkOptions.EMULATOR_HOST) != null) {
            return BigQueryOptions.newBuilder()
                    .setHost("http://" + config.get(BigQuerySinkOptions.EMULATOR_HOST))
                    .setProjectId(projectId)
                    .setCredentials(NoCredentials.getInstance())
                    .build()
                    .getService();
        }

        GoogleCredentials credentials = getCredentials(config);

View on GitHub (pinned to cf67b549a7)

Solutions

1. Check the wrapped IOException cause — usually 'Your default credentials were not found' or a file-read error
2. Set/verify GOOGLE_APPLICATION_CREDENTIALS or inline credentials options in the sink config; ensure the key file exists on every worker node
3. Validate the UNIVERSE_DOMAIN option matches your GCP universe (default googleapis.com)
4. Confirm network egress/DNS to bigquerystorage.googleapis.com and required OAuth scopes

Example fix

// before
BigQuerySinkOptions.UNIVERSE_DOMAIN -> "mydomain.com"  // wrong custom domain
// after
// remove UNIVERSE_DOMAIN option (defaults to googleapis.com) or set the correct universe domain
BigQuerySinkOptions.UNIVERSE_DOMAIN -> "googleapis.com"

// before building the sink, ensure credentials resolve
String creds = System.getenv("GOOGLE_APPLICATION_CREDENTIALS");
if (creds == null || !new java.io.File(creds).canRead()) {
    throw new IllegalStateException("BigQuery credentials not set or unreadable: " + creds);
}

try {
    BigQueryWriteClient client = factory.getWriteClient();
} catch (BigQueryConnectorException e) {
    if ("CLIENT_CREATE_FAILED".equals(e.getErrorCode().getCode())) {
        LOG.error("Write client init failed; check credentials/network: " + e.getCause(), e);
    }
    throw e;
}

When it happens

Trigger: Building BigQueryWriteSettings with invalid/missing credentials, unreadable service-account key file, wrong universe domain or endpoint, or a network/DNS failure during client construction when getWriteClient is first invoked (lazy singleton init).

Common situations: GOOGLE_APPLICATION_CREDENTIALS not set in the worker environment; key file path typo or missing file in the container image; invalid 'universe-domain' option (e.g. non-Google domain); private connectivity misconfiguration; credentials lacking BigQuery scopes.

Related errors

• Failed to open BigQueryCatalog
• Failed to list BigQuery databases (datasets)
• Failed to list tables in dataset:
• Failed to create BigQuery table:
• Failed to drop BigQuery table:

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/60f027d10d91bcfb. Report an issue: GitHub.