apache/beam · critical · RuntimeException

Unable to obtain credential

Error message

Unable to obtain credential

What it means

GcpOptions.getCredentialFactory() builds a credential via the configured CredentialFactoryClass and wraps any IOException or GeneralSecurityException from credential creation in a RuntimeException with this message. It means the library could not construct a valid Google Cloud credential from the pipeline options (e.g. service account key, metadata server, or installed-account flow).

Source

Thrown at sdks/java/extensions/google-cloud-platform-core/src/main/java/org/apache/beam/sdk/extensions/gcp/options/GcpOptions.java:342

  /**
   * Attempts to load the GCP credentials. See {@link CredentialFactory#getCredential()} for more
   * details.
   */
  class GcpUserCredentialsFactory implements DefaultValueFactory<Credentials> {
    @Override
    public Credentials create(PipelineOptions options) {
      GcpOptions gcpOptions = options.as(GcpOptions.class);
      try {
        CredentialFactory factory =
            InstanceBuilder.ofType(CredentialFactory.class)
                .fromClass(gcpOptions.getCredentialFactoryClass())
                .fromFactoryMethod("fromOptions")
                .withArg(PipelineOptions.class, options)
                .build();
        return factory.getCredential();
      } catch (IOException | GeneralSecurityException e) {
        throw new RuntimeException("Unable to obtain credential", e);
      }
    }
  }

  /** EnableStreamingEngine defaults to false unless one of the two experiments is set. */
  class EnableStreamingEngineFactory implements DefaultValueFactory<Boolean> {
    @Override
    public Boolean create(PipelineOptions options) {
      return ExperimentalOptions.hasExperiment(options, STREAMING_ENGINE_EXPERIMENT)
          || ExperimentalOptions.hasExperiment(options, WINDMILL_SERVICE_EXPERIMENT);
    }
  }

  /**
   * A GCS path for storing temporary files in GCP.
   *
   * <p>Its default to {@link PipelineOptions#getTempLocation}.
   */

View on GitHub (pinned to 12126d8942)

Solutions

1. Set --serviceAccountKeyFile (or GOOGLE_APPLICATION_CREDENTIALS) to a valid, downloadable JSON service account key
2. Run `gcloud auth application-default login` for local development so ADC succeeds
3. Validate the key JSON parses and the service account is enabled and has roles (e.g. roles/iam.serviceAccountUser, storage access)
4. If using a custom CredentialFactoryClass, verify fromOptions() does not throw for your options

Example fix

// before
PipelineOptions options = PipelineOptionsFactory.create(); // no credentials configured
// after
options.as(GcpOptions.class).setServiceAccountKeyFile("/path/to/valid-key.json");

import com.google.auth.oauth2.GoogleCredentials;
GoogleCredentials creds = GoogleCredentials.getApplicationDefault(); // throws if ADC unusable
if (options.as(GcpOptions.class).getServiceAccountKeyFile() != null)
  if (!new java.io.File(options.as(GcpOptions.class).getServiceAccountKeyFile()).exists())
    throw new IllegalStateException("service account key file missing");

boolean hasCredentialSource(GcpOptions o) {
  return o.getServiceAccountKeyFile() != null || System.getenv("GOOGLE_APPLICATION_CREDENTIALS") != null;
}

try {
  Credential c = GcpOptions.CredentialFactory.createFromOptions(options);
} catch (RuntimeException e) {
  if (e.getMessage() != null && e.getMessage().contains("Unable to obtain credential")) {
    throw new IllegalStateException("Configure GOOGLE_APPLICATION_CREDENTIALS or --serviceAccountKeyFile", e);
  }
  throw e;
}

When it happens

Trigger: Calling PipelineOptionsFactory-based runs where GcpOptions.getCredential() is invoked and the credential factory's fromOptions().getCredential() throws IOException or GeneralSecurityException: missing/invalid service account JSON key, unreadable key file path, corrupted key, or default-credential lookup failing on a machine without ADC.

Common situations: Running a Beam pipeline locally without GOOGLE_APPLICATION_CREDENTIALS set; pointing serviceAccountKeyFile to a missing or malformed JSON key; a key from the wrong project or a revoked service account; firewall blocking the metadata server on GCE.

Related errors

• Unable to get application default credentials. Please see ht
• Failed to get application default credential.
• Could not resolve GCP project ID%s. Please specify 'project'
• GCP Authentication Extension not configured properly: %s not
• Error creating Data Catalog client

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/5e776af76aaf6a4a. Report an issue: GitHub.