{"record":{"id":"948a9fa5971d0894","repo":"apache/beam","slug":"failed-to-retrieve-or-create-secret-bytes","errorCode":null,"errorMessage":"Failed to retrieve or create secret bytes","messagePattern":"Failed to retrieve or create secret bytes","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"sdks/java/extensions/google-cloud-platform-core/src/main/java/org/apache/beam/sdk/extensions/gcp/util/GcpHsmGeneratedSecret.java","lineNumber":160,"sourceCode":"\n      try {\n        // Always retrieve remote secret as source-of-truth in case another thread created it\n        AccessSecretVersionResponse response = client.accessSecretVersion(secretVersionName);\n        return response.getPayload().getData().toByteArray();\n      } catch (NotFoundException e) {\n        LOG.info(\n            \"Secret version {} not found after re-check. Creating new secret and version.\",\n            secretVersionName.toString());\n      }\n\n      SecretPayload payload =\n          SecretPayload.newBuilder().setData(ByteString.copyFrom(newKey)).build();\n      client.addSecretVersion(secretName, payload);\n      AccessSecretVersionResponse response = client.accessSecretVersion(secretVersionName);\n      return response.getPayload().getData().toByteArray();\n\n    } catch (IOException | GeneralSecurityException e) {\n      throw new RuntimeException(\"Failed to retrieve or create secret bytes\", e);\n    }\n  }\n\n  private byte[] generateDek() throws IOException, GeneralSecurityException {\n    int dekSize = 32;\n    try (KeyManagementServiceClient client = KeyManagementServiceClient.create()) {\n      // 1. Generate nonce_one. This doesn't need to have baked in randomness since the\n      // actual randomness comes from KMS.\n      byte[] nonceOne = new byte[dekSize];\n      random.nextBytes(nonceOne);\n\n      // 2. Encrypt to get nonce_two\n      CryptoKeyName keyName = CryptoKeyName.of(projectId, locationId, keyRingId, keyId);\n      EncryptResponse response = client.encrypt(keyName, ByteString.copyFrom(nonceOne));\n      byte[] nonceTwo = response.getCiphertext().toByteArray();\n\n      // 3. Generate DK\n      byte[] dk = new byte[dekSize];","sourceCodeStart":142,"sourceCodeEnd":178,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/extensions/google-cloud-platform-core/src/main/java/org/apache/beam/sdk/extensions/gcp/util/GcpHsmGeneratedSecret.java#L142-L178","documentation":"GcpHsmGeneratedSecret.getSecretBytes either finds/creates a Secret Manager secret backed by Cloud KMS and reads its bytes. Any IOException or GeneralSecurityException during secret creation, addSecretVersion, or accessSecretVersion is wrapped into a RuntimeException('Failed to retrieve or create secret bytes', cause). The root cause is always attached.","triggerScenarios":"KMS key not found or caller lacks cryptoKeyEncrypterDecrypter permission; Secret Manager API not enabled; secret creation/access API call failing (network, quota); invalid service credentials.","commonSituations":"Missing roles/cloudkms.cryptoKeyEncrypterDecrypter on the service account; secret in a region different from the KMS key ring; google-cloud-secretmanager not enabled on the project.","solutions":["Read the chained cause exception for the real API error","Enable the Secret Manager API: gcloud services enable secretmanager.googleapis.com","Grant the service account roles/secretmanager.admin and roles/cloudkms.cryptoKeyEncrypterDecrypter","Verify location_id matches the KMS key ring's region and the key exists"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try { byte[] dek = hsmSecret.getSecretBytes(); }\ncatch (RuntimeException e) {\n  Throwable cause = e.getCause(); // IOException | GeneralSecurityException\n  LOG.error(\"Secret retrieval failed: \" + cause, cause);\n  throw new IllegalStateException(\"Check Secret Manager API, KMS permissions, and credentials\", e);\n}","preventionTips":["Enable secretmanager.googleapis.com and cloudkms.googleapis.com","Grant roles/secretmanager.admin and roles/cloudkms.cryptoKeyEncrypterDecrypter to the runtime service account","Verify location_id/key_ring_id/key_id exist in the same region with gcloud kms keys describe"],"tags":["gcp","kms","secret-manager","io","java"],"backgroundTag":"api-request-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}