apache/beam · error · ValueError

Config for transform at

Error message

Config for transform at %s must be a mapping.

What it means

create_ptransform reads `spec['config']` and strips line metadata; the result must be a dict/mapping because it is passed as keyword-style options to the transform. A scalar, list, or string config raises this ValueError naming the spec location.

Solutions

  1. Rewrite the config as a mapping: each option as `key: value` under `config:`.
  2. Check the transform's documented config schema for exact option names.
  3. For a single option, use inline mapping syntax `config: {option: value}`.
  4. Inspect the reported location to confirm the YAML node parses as a dict.

Example fix

# before
- type: ReadFromBigQuery
  config: my-project:dataset.table
# after
- type: ReadFromBigQuery
  config:
    query: 'SELECT * FROM `my-project.dataset.table`'
Defensive patterns

Strategy: type-guard

Validate before calling

cfg = spec.get('config', {})
if not isinstance(cfg, dict):
    raise ValueError(f'config for {spec.get("name")} must be a mapping, got {type(cfg).__name__}')

Type guard

def has_mapping_config(spec):
    return isinstance(spec, dict) and isinstance(spec.get('config', {}), dict)

Try / catch

try:
    run_pipeline(spec)
except ValueError as e:
    if 'must be a mapping' in str(e):
        raise UserPipelineError('Rewrite config as indented key/value mapping') from e

Prevention

When it happens

Trigger: Writing `config: some_string`, `config: [a, b]`, or a bare value in the YAML instead of a mapping of option names to values; programmatic specs passing a non-dict `config`.

Common situations: Config written in flow style that parses as a list; forgetting config contents should be indented key/value pairs; converting from JSON where config was an array; `config: value` instead of `config: {option: value}`.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/d316b8bc82af50f5. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/yaml/yaml_transform.py:390

      raise ValueError(
          'Unknown transform type %r at %s' %
          (spec['type'], identify_object(spec)))

    # TODO(yaml): Perhaps we can do better than a greedy choice here.
    # TODO(yaml): Figure out why this is needed.
    providers_by_input = {k: v for k, v in self.input_providers.items()}
    input_providers = [
        providers_by_input[pcoll] for pcoll in input_pcolls
        if pcoll in providers_by_input
    ]
    provider = self.best_provider(spec, input_providers)
    extra_dependencies, spec = extract_extra_dependencies(spec)
    if extra_dependencies:
      provider = provider.with_extra_dependencies(frozenset(extra_dependencies))

    config = SafeLineLoader.strip_metadata(spec.get('config', {}))
    if not isinstance(config, dict):
      raise ValueError(
          'Config for transform at %s must be a mapping.' %
          identify_object(spec))

    if (not input_pcolls and not is_explicitly_empty(spec.get('input', {})) and
        provider.requires_inputs(spec['type'], config)):
      raise ValueError(
          f'Missing inputs for transform at {identify_object(spec)}')

    try:
      if spec['type'].endswith('-generic'):
        # Centralize the validation rather than require every implementation
        # to do it.
        validate_generic_expressions(
            spec['type'].rsplit('-', 1)[0], config, input_pcolls)

      # pylint: disable=undefined-loop-variable
      ptransform = maybe_with_resource_hints(
          provider.create_transform(

View on GitHub (pinned to 12126d8942)