apache/beam · error · RuntimeException

Failed to get application default credential.

Error message

Failed to get application default credential.

What it means

BigqueryClient.getDefaultCredential obtains Google Application Default Credentials (ADC) for BigQuery test clients. If GoogleCredentials.getApplicationDefault() throws IOException, no ADC could be found in the environment, and a RuntimeException wrapping 'Failed to get application default credential.' is thrown.

Source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/testing/BigqueryClient.java:144

  // The initial backoff for executing a BigQuery RPC
  private static final Duration INITIAL_BACKOFF = Duration.standardSeconds(1L);

  // The backoff factory with initial configs
  static final FluentBackoff BACKOFF_FACTORY =
      FluentBackoff.DEFAULT.withMaxRetries(MAX_QUERY_RETRIES).withInitialBackoff(INITIAL_BACKOFF);

  private static final Collection<String> RESERVED_FIELD_NAMES =
      ClassInfo.of(TableRow.class).getNames();

  private Bigquery bqClient;

  private static Credentials getDefaultCredential() {
    GoogleCredentials credential;
    try {
      credential = GoogleCredentials.getApplicationDefault();
    } catch (IOException e) {
      throw new RuntimeException("Failed to get application default credential.", e);
    }

    if (credential.createScopedRequired()) {
      Collection<String> bigqueryScope = Lists.newArrayList(BigqueryScopes.all());
      credential = credential.createScoped(bigqueryScope);
    }
    return credential;
  }

  public static Bigquery getNewBigqueryClient(String applicationName) {
    HttpTransport transport = Transport.getTransport();
    JsonFactory jsonFactory = Transport.getJsonFactory();
    Credentials credential = getDefaultCredential();
    return new Bigquery.Builder(transport, jsonFactory, new HttpCredentialsAdapter(credential))
        .setApplicationName(applicationName)
        .build();
  }

View on GitHub (pinned to 12126d8942)

Solutions

Run `gcloud auth application-default login` on the machine executing the tests.
Set GOOGLE_APPLICATION_CREDENTIALS to a valid service-account JSON key path with BigQuery access.
Verify the file at GOOGLE_APPLICATION_CREDENTIALS exists and is readable (not a broken path).
In GCP environments (GCE/GKE/Cloud Run), attach the service account / workload identity with BigQuery scopes.
Grant the identity the `bigquery.jobs.create` / dataset roles needed.

Example fix

// before (shell)
mvn verify -Pit

// after (shell)
gcloud auth application-default login
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json
mvn verify -Pit

Defensive patterns

Strategy: validation

Validate before calling

// shell
if [ -z "$GOOGLE_APPLICATION_CREDENTIALS" ] && ! gcloud auth application-default print-access-token >/dev/null 2>&1; then
  echo "No application default credentials found"; exit 1
fi

Try / catch

try {
  Credentials c = GoogleCredentials.getApplicationDefault();
} catch (IOException e) {
  throw new IllegalStateException("Set GOOGLE_APPLICATION_CREDENTIALS or run gcloud auth application-default login", e);
}

Prevention

Run gcloud auth application-default login before local IT runs.
Export GOOGLE_APPLICATION_CREDENTIALS in CI setup steps and fail fast if unset.
Verify key files exist and are readable at job startup.
Use workload identity in GCP environments instead of key files.

When it happens

Trigger: Calling credential()/getDefaultCredential() in an environment where GOOGLE_APPLICATION_CREDENTIALS is unset and no gcloud ADC metadata (~/.config/gcloud/application_default_credentials.json), GCE/GKE/App Engine default identity, or workload identity is available.

Common situations: Running BigQuery ITs locally without `gcloud auth application-default login`; CI containers without the credentials env var; mispointed GOOGLE_APPLICATION_CREDENTIALS path; service account key file deleted or unreadable.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/5bc7e88c122be323. Report an issue: GitHub.