apache/beam · error · ValueError

Missing language for

Error message

Missing language for {identify_object(spec)}

What it means

Several generic-language transforms (e.g. AssignTimestamps, MapToFields, Partition) get rewritten by preprocess_languages to type-<language>. If no explicit language was configured (defaulting to 'generic') and no provider exists for the '<type>-generic' variant, the pipeline cannot resolve the transform, so this ValueError is raised telling the author to specify a language.

Solutions

  1. Add an explicit language in config, e.g. config: {language: python}.
  2. Register a provider for the generic variant of the transform.
  3. Choose a concrete-language transform type instead of the generic one.

Example fix

# before
- type: MapToFields
  config:
    fields: {x: expr}
# after
- type: MapToFields
  config:
    language: python
    fields: {x: expr}
Defensive patterns

Strategy: validation

Validate before calling

LANG_SENSITIVE = ('AssignTimestamps', 'MapToFields', 'Partition')
for t in spec.get('transforms', []):
    if t.get('type') in LANG_SENSITIVE and 'language' not in t.get('config', {}):
        print(f"warning: {t.get('name')} has no explicit language; set config.language")

Type guard

def has_language(t):
    return isinstance(t.get('config', {}).get('language'), str)

Try / catch

try:
    spec = expand_pipeline(spec, providers=providers)
except ValueError as e:
    if 'Missing language for' in str(e):
        raise SystemExit(f"{e} — add config.language (e.g. python)")
    raise

Prevention

When it happens

Trigger: preprocess_languages on a spec for a language-sensitive transform where spec.config has no 'language' key (so language='generic'), known_transforms is non-empty, and '<type>-generic' is not in known_transforms.

Common situations: Omitting config.language when the environment only offers a specific-language provider (e.g. only python or only java registered); copy-pasting specs across pipelines with different provider sets; assuming a default language that the runner does not ship.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/7902c81dd5d31f01. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/yaml/yaml_transform.py:1346

  def ensure_transforms_have_providers(spec):
    if known_transforms:
      if spec['type'] not in known_transforms:
        raise ValueError(
            'Unknown type or missing provider '
            f'for type {spec["type"]} for {identify_object(spec)}')
    return spec

  def preprocess_languages(spec):
    if spec['type'] in ('AssignTimestamps',
                        'Combine',
                        'Filter',
                        'MapToFields',
                        'Partition'):
      language = spec.get('config', {}).get('language', 'generic')
      new_type = spec['type'] + '-' + language
      if known_transforms and new_type not in known_transforms:
        if language == 'generic':
          raise ValueError(f'Missing language for {identify_object(spec)}')
        else:
          raise ValueError(
              f'Unknown language {language} for {identify_object(spec)}')
      return dict(spec, type=new_type, name=spec.get('name', spec['type']))
    else:
      return spec

  def validate_transform_references(spec):
    name = spec.get('name', '')
    transform_type = spec.get('type')
    inputs = spec.get('input').get('input', [])

    if not is_empty(inputs):
      input_values = [inputs] if isinstance(inputs, str) else inputs
      for input_value in input_values:
        if input_value in (name, transform_type):
          raise ValueError(
              f"Circular reference detected: Transform {name} "

View on GitHub (pinned to 12126d8942)