apache/beam · error · RuntimeException

Failed to retrieve secret bytes

Error message

Failed to retrieve secret bytes

What it means

GcpSecret.getSecretBytes opens a SecretManagerServiceClient and accesses the secret version; if client creation or the access call throws IOException it is wrapped in RuntimeException('Failed to retrieve secret bytes', cause). Note: API-level errors like NOT_FOUND surface as ApiException (runtime), so this wrap mostly covers transport/credential IO problems.

Source

Thrown at sdks/java/extensions/google-cloud-platform-core/src/main/java/org/apache/beam/sdk/extensions/gcp/util/GcpSecret.java:140

                + "Please specify 'project' in the secret spec, set GOOGLE_CLOUD_PROJECT environment variable, "
                + "or configure Application Default Credentials.",
            context != null ? " for " + context : ""));
  }

  /**
   * Returns the secret as a byte array. Assumes that the current active service account has
   * permissions to read the secret.
   *
   * @return The secret as a byte array.
   */
  @Override
  public byte[] getSecretBytes() {
    try (SecretManagerServiceClient client = SecretManagerServiceClient.create()) {
      SecretVersionName secretVersionName = SecretVersionName.parse(versionName);
      AccessSecretVersionResponse response = client.accessSecretVersion(secretVersionName);
      return response.getPayload().getData().toByteArray();
    } catch (IOException e) {
      throw new RuntimeException("Failed to retrieve secret bytes", e);
    }
  }

  /**
   * Returns the version name of the secret.
   *
   * @return The version name as a String.
   */
  public String getVersionName() {
    return versionName;
  }

  @Override
  public boolean equals(@Nullable Object obj) {
    if (this == obj) {
      return true;
    }
    if (!(obj instanceof GcpSecret)) {

View on GitHub (pinned to 12126d8942)

Solutions

1. Read the chained cause for the underlying IOException
2. Configure credentials: set GOOGLE_APPLICATION_CREDENTIALS or run gcloud auth application-default login
3. Verify network access to secretmanager.googleapis.com:443
4. Confirm the version name parses and exists (invalid names usually surface as ApiException, not this wrap)

if (System.getenv("GOOGLE_APPLICATION_CREDENTIALS") == null
    && GoogleCredentials.getApplicationDefault() == null) {
  throw new IllegalStateException("No ADC credentials available for Secret Manager");
}

try { byte[] b = secret.getSecretBytes(); }
catch (RuntimeException e) {
  LOG.error("Secret bytes unavailable: " + e.getCause(), e.getCause());
  throw new IOException("Secret retrieval failed; check credentials/network", e);
}

When it happens

Trigger: SecretManagerServiceClient.create() failing due to credentials/environment IO issues; network failure during accessSecretVersion; ADC cannot be loaded (IOException path).

Common situations: No Application Default Credentials configured (missing GOOGLE_APPLICATION_CREDENTIALS file); network/proxy blocking secretmanager.googleapis.com; quota or connection reset errors.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• Failed to retrieve or create secret bytes
• Unable to verify project with ID ${projectId}
• Unable to get project number
• Invalid secret parameter %s
• Invalid secret parameter %s

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/3dfb78f571c82455. Report an issue: GitHub.