apache/beam · error · ValueError

deidentification_template_name or deidentification_config…

Error message

deidentification_template_name or deidentification_config must be specified.

What it means

CloudDLPOptions requires deidentification information: at least one of deidentification_template_name or deidentification_config must be given. Supplying neither means the DLP deidentify call has no way to know how to transform the data, so __init__ raises ValueError.

Solutions

Add deidentification_template_name='projects/PROJECT/locations/LOCATION/deidentifyTemplates/TEMPLATE_ID' or an inline deidentification_config=dlp.DeidentifyConfig(...)
If no deidentification is desired, confirm whether your Beam version supports inspection-only options; otherwise supply a minimal DeidentifyConfig
Check config loading so empty strings are not converted to None unexpectedly

Example fix

// before
opts = CloudDLPOptions(project='p', inspection_config=cfg)

// after
opts = CloudDLPOptions(project='p', inspection_config=cfg,
    deidentification_template_name='projects/p/deidentifyTemplates/t')

Defensive patterns

Strategy: validation

Validate before calling

if deid_template is None and deid_config is None:
    raise ValueError('CloudDLPOptions needs deidentification_template_name or deidentification_config')

Try / catch

try:
    options = CloudDLPOptions(project=p, inspection_config=i)
except ValueError as e:
    if 'must be specified' in str(e):
        options = CloudDLPOptions(project=p, inspection_config=i,
            deidentification_config=dlp.DeidentifyConfig())

Prevention

Always configure deidentification when using CloudDLP transforms
Check config-file parsing doesn't drop required deidentify fields
Copy complete examples from the Beam docs, not partial snippets

When it happens

Trigger: Calling CloudDLPOptions(project=..., inspection_config=...) without either deidentification argument (both None).

Common situations: Omitting deidentification options assuming inspection-only usage is allowed; config-file loaders dropping empty/None values so both end up None; docs examples trimmed to remove the deidentify arguments.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/0bef9af8198f4374. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/ml/gcp/cloud_dlp.py:102

        Configuration for the inspector used to detect sensitive data in text.
        If both template name and config are supplied,
        config takes precedence.
      timeout (float): Optional. The amount of time, in seconds, to wait for
        the request to complete.

    """
    self.config = {}
    self.project = project
    self.timeout = timeout
    if deidentification_template_name is not None \
        and deidentification_config is not None:
      raise ValueError(
          'Both deidentification_template_name and '
          'deidentification_config were specified.'
          ' Please specify only one of these.')
    elif deidentification_template_name is None \
        and deidentification_config is None:
      raise ValueError(
          'deidentification_template_name or '
          'deidentification_config must be specified.')
    elif deidentification_template_name is not None:
      self.config['deidentify_template_name'] = deidentification_template_name
    else:
      self.config['deidentify_config'] = deidentification_config

    if inspection_config is None and inspection_template_name is None:
      raise ValueError(
          'inspection_template_name or inspection_config must be specified')
    if inspection_template_name is not None:
      self.config['inspect_template_name'] = inspection_template_name
    if inspection_config is not None:
      self.config['inspect_config'] = inspection_config

  def expand(self, pcoll):
    if self.project is None:
      self.project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project

View on GitHub (pinned to 12126d8942)