apache/beam · error · ValueError

Unsupported secret manager

Error message

Unsupported secret manager: '{secret_manager_name}'. Currently supported options: 'GoogleCloudSecretManager', 'GoogleCloudHsmGeneratedSecretManager'.

What it means

Secret.from_json in apache_beam/utils/secret.py resolves a secret spec string to a registered Secret subclass. Only two secret manager names are accepted: 'GoogleCloudSecretManager' (mapped from 'gcpsecret') and 'GoogleCloudHsmGeneratedSecretManager' (mapped from 'gcphsmgeneratedsecret'). If the spec names any other manager, a ValueError is raised listing the supported options.

Solutions

Set secret_manager to exactly 'GoogleCloudSecretManager' for GCP Secret Manager secrets.
Use 'GoogleCloudHsmGeneratedSecretManager' for HSM-backed secrets (spec type 'gcphsmgeneratedsecret').
Check spelling and casing against the message's supported list; the lookup is by exact registered name.
If you need another provider, either pre-fetch the secret outside Beam or implement and register a custom Secret subclass.

Example fix

// before
Secret.from_json('{"secret_manager": "GoogleSecretManager", "config": {"name": "my-secret"}}')
// after
Secret.from_json('{"secret_manager": "GoogleCloudSecretManager", "config": {"name": "my-secret"}}')

Defensive patterns

Strategy: validation

Validate before calling

SUPPORTED = {'GoogleCloudSecretManager', 'GoogleCloudHsmGeneratedSecretManager'}
assert spec.get('secret_manager') in SUPPORTED, f"unsupported secret_manager: {spec.get('secret_manager')!r}"

Type guard

def is_supported_secret_manager(name) -> bool:
    return isinstance(name, str) and name in {'GoogleCloudSecretManager', 'GoogleCloudHsmGeneratedSecretManager'}

Try / catch

try:
    secret = Secret.from_json(spec)
except ValueError as e:
    logging.error('Bad secret manager spec %s: %s', spec, e)
    raise

Prevention

Copy the manager name exactly from the supported list
Centralize secret specs in one module with constants
Validate pipeline options at startup before submitting the job

When it happens

Trigger: Calling Secret.from_json (directly or via parse_secret_option) with a spec string whose secret_manager name is misspelled, has different casing, or is a manager type Beam does not implement, e.g. Secret.from_json('{"secret_manager": "AWSParameterStore", ...}') or 'HashiCorpVault'.

Common situations: Typo in the secret manager name in pipeline options (e.g. 'GoogleCloudsecretManager', 'gcp_secret'); copy-pasting a secret spec from another framework; assuming other cloud providers' secret managers are supported.

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/7d7b386825bd73cb. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/utils/secret.py:163

          spec_dict = None
      except Exception:
        pass

    if secret_manager_name:
      secret_cls_entry = _SECRET_CLASSES.get(secret_manager_name.lower())
      if secret_cls_entry:
        if isinstance(secret_cls_entry, str):
          secret_cls = globals().get(secret_cls_entry, secret_cls_entry)
        else:
          secret_cls = secret_cls_entry
        if isinstance(spec_dict, dict) and hasattr(secret_cls, 'from_dict'):
          return secret_cls.from_dict(spec_dict)
        elif isinstance(spec_dict, dict):
          return secret_cls(**spec_dict)
        else:
          return secret_cls(spec)
      else:
        raise ValueError(
            f"Unsupported secret manager: '{secret_manager_name}'. Currently supported options: 'GoogleCloudSecretManager', 'GoogleCloudHsmGeneratedSecretManager'."
        )

    # If secret_manager is not set or empty, check if spec is a JSON specification dict
    if spec_dict is not None:
      msg = (
          "The 'spec' parameter appears to be a JSON specification, but "
          "'secret_manager' is not set. Defaulting to Raw.")
      _LOGGER.warning(msg)
      warnings.warn(msg, UserWarning)

    return RawSecret(spec)


class RawSecret(Secret):
  """Secret implementation wrapping a raw secret string or bytes directly."""
  def __init__(self, secret: Union[str, bytes]):
    super().__init__()

View on GitHub (pinned to 12126d8942)