apache/beam · error · ValueError

Invalid value " " for "fields". For every input key in…

Error message

Invalid value "{fields}" for "fields". For every input key in fields, the value must either be a list or dict.

What it means

Each value in the 'fields' mapping must be either a list of column names or a dict of {outputName: sourceColumn}. Any other value type (string, number, etc.) for an input key falls through both branches and raises this ValueError.

Solutions

Wrap the column in a list: left: [id].
Use the dict form for renamed columns: left: {id: id}.
Check YAML indentation so the value parses as a list or mapping, not a scalar.

Example fix

// before
fields:
  left: id
// after
fields:
  left: [id]

Defensive patterns

Strategy: type-guard

Validate before calling

def check_fields_value_types(fields):
    for alias, cols in fields.items():
        if not isinstance(cols, (list, dict)):
            raise ValueError(f'fields[{alias!r}] must be a list or dict, got {type(cols).__name__}')

Type guard

def is_list_or_dict(v):
    return isinstance(v, (list, dict))

Try / catch

try:
    result = SqlJoinTransform(config)
except ValueError as e:
    if 'must either be a list or dict' in str(e):
        raise ConfigError('wrap single columns in a list, e.g. left: [id]') from e
    raise

Prevention

Never assign a bare scalar as a fields value
Validate the parsed spec shape before running the pipeline
Follow the per-input convention: alias -> list of names or dict of renames

When it happens

Trigger: fields: {left: id} (a bare string), {left: 3}, or a value that YAML parsed as a scalar instead of a list/dict.

Common situations: Users expecting a single string to mean 'just this one column'; YAML indentation errors that flatten a list into a scalar; copying config from a transform with a different fields convention.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/d2424b017ae3408e. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/yaml/yaml_join.py:140

          raise ValueError(
              f'Invalid column "{col}" in "fields". Column name must be a str.')
        if col in named_columns:
          raise ValueError(
              f'The field name "{col}" was specified more than once.')
        output_fields.append(f'{input}.{col} AS {col}')
        named_columns.add(col)
    elif isinstance(cols, dict):
      for k, v in cols.items():
        if k in named_columns:
          raise ValueError(
              f'The field name "{k}" was specified more than once.')
        if not isinstance(v, str):
          raise ValueError(
              f'Invalid column "{v}" in "fields". Column name must be a str.')
        output_fields.append(f'{input}.{v} AS {k}')
        named_columns.add(k)
    else:
      raise ValueError(
          f'{error_prefix} '
          f'For every input key in fields, '
          f'the value must either be a list or dict.')
  for table in tables:
    if table not in fields.keys():
      output_fields.append(f'{table}.*')
  return output_fields


def _is_connected(edge_list, expected_node_count):
  graph = {}
  for edge_set in edge_list:
    for u in edge_set:
      if u not in graph:
        graph[u] = set()
      for v in edge_set:
        if u != v:
          graph[u].add(v)

View on GitHub (pinned to 12126d8942)