apache/beam · error · RuntimeException

Exception getting credentials

Error message

Exception getting credentials 

What it means

buildBigtableDataSettings installs a FixedCredentialsProvider built from the configured credentialFactory. If obtaining the GoogleCredentials throws GeneralSecurityException, it is wrapped in this RuntimeException so pipeline translation fails loudly rather than continuing without credentials.

Solutions

1. Regenerate or re-download the service account key / credentials and confirm the file loads (e.g. ServiceAccountCredentials.fromStream).
2. Verify the credentialFactory configuration (path, key type, passphrase) used to build the BigtableConfig.
3. Consider relying on Application Default Credentials instead of explicit credential material to avoid key handling errors.

Example fix

// before
BigtableConfig config = BigtableConfig.newBuilder().setCredentialFactory(new FancyCredentialFactory("bad.pem")).build();
// after
BigtableConfig config = BigtableConfig.newBuilder()
    .setCredentialFactory(new ServiceAccountJsonCredentialFactory("/valid/sa.json")).build();

// pre-flight: load credentials once before submitting the job
GoogleCredentials creds = credentialFactory.getCredential(); // fails fast locally
if (creds == null) throw new IllegalStateException("No credentials configured");

try { settings = translator.buildBigtableDataSettings(config, options); } catch (RuntimeException e) { if (e.getMessage().startsWith("Exception getting credentials")) { log.error("Fix the Bigtable credentialFactory config", e); } throw e; }

When it happens

Trigger: Translating a BigtableConfig with explicit credentials where credentialFactory.getCredential() raises GeneralSecurityException (bad key material, corrupt service-account file, invalid key format).

Common situations: Malformed or expired service-account JSON/PEM; wrong key algorithm; credentials file edited or truncated; test fixtures with fake key data.

Related errors

• exception while retrieving credentials
• private key cannot be null
• AWS credential provider type
• Bigtable location must be in the following format…
• Builder method has to be explicitly allowed

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/c48bf1daca0c0f26. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableConfigTranslator.java:151

    // Skip resetting the credentials if it's connected to an emulator
    if (!emulator) {
      if (pipelineOptions.as(GcpOptions.class).getGcpCredential() != null) {
        dataBuilder
            .stubSettings()
            .setCredentialsProvider(
                FixedCredentialsProvider.create(
                    pipelineOptions.as(GcpOptions.class).getGcpCredential()));
      }

      if (config.getCredentialFactory() != null) {
        CredentialFactory credentialFactory = config.getCredentialFactory();
        try {
          dataBuilder
              .stubSettings()
              .setCredentialsProvider(
                  FixedCredentialsProvider.create(credentialFactory.getCredential()));
        } catch (GeneralSecurityException e) {
          throw new RuntimeException("Exception getting credentials ", e);
        }
      }
    }

    configureChannelPool(dataBuilder.stubSettings(), config);
    configureHeaderProvider(dataBuilder.stubSettings(), pipelineOptions);

    // Provide a way to override any BigtableDataSettings
    String overrideClassName =
        ExperimentalOptions.getExperimentValue(pipelineOptions, BIGTABLE_SETTINGS_OVERRIDE);

    return configureSettingsOverride(overrideClassName, dataBuilder, pipelineOptions);
  }

  private static BigtableDataSettings.Builder configureSettingsOverride(
      @Nullable String override,
      BigtableDataSettings.Builder dataBuilder,
      PipelineOptions pipelineOptions) {

View on GitHub (pinned to 12126d8942)