apache/beam · error · ValueError

Invalid column " " in "fields". Column name must be a str.

Error message

Invalid column "{col}" in "fields". Column name must be a str.

What it means

When the value for an input in 'fields' is a list, each entry is used directly as an output column name and must be a string. A non-string entry (number, bool, nested structure) triggers this ValueError.

Solutions

Quote each column name so YAML keeps it a string, e.g. '123'.
Remove or flatten any nested structures in the column list.
Rename numeric columns upstream so they are valid string names.

Example fix

// before
fields:
  left: [id, 42]
// after
fields:
  left: [id, "42"]

Defensive patterns

Strategy: type-guard

Validate before calling

def check_list_cols(fields):
    for alias, cols in fields.items():
        if isinstance(cols, list):
            assert all(isinstance(c, str) for c in cols), f'non-str column for {alias}'

Type guard

def all_str(items):
    return isinstance(items, list) and all(isinstance(c, str) for c in items)

Try / catch

try:
    result = SqlJoinTransform(config)
except ValueError as e:
    if 'Column name must be a str' in str(e):
        raise ConfigError('quote numeric column names in fields lists') from e
    raise

Prevention

Quote column names that look numeric or boolean in YAML
Inspect the parsed spec (yaml.safe_load) to confirm types
Avoid pasting schema objects where plain strings are expected

When it happens

Trigger: Writing fields: {left: [id, 42]} or {left: [id, [name]]} — any list element that is not a str.

Common situations: YAML auto-typing numeric column names into ints; accidentally nesting a list; pasting schema objects instead of plain column names.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/df08720004af05d2. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/yaml/yaml_join.py:122

        f'{error_prefix} '
        f'The provided equalities do not connect all of {list(pcolls.keys())}.')

  return equalities


def _parse_fields(tables, fields):
  error_prefix = f'Invalid value "{fields}" for "fields".'
  if not isinstance(fields, dict):
    raise ValueError(f'{error_prefix} Fields must be a dict.')
  output_fields = []
  named_columns = set()
  for input, cols in fields.items():
    if input not in tables:
      raise ValueError(f'An invalid input "{input}" was specified in "fields".')
    if isinstance(cols, list):
      for col in cols:
        if not isinstance(col, str):
          raise ValueError(
              f'Invalid column "{col}" in "fields". Column name must be a str.')
        if col in named_columns:
          raise ValueError(
              f'The field name "{col}" was specified more than once.')
        output_fields.append(f'{input}.{col} AS {col}')
        named_columns.add(col)
    elif isinstance(cols, dict):
      for k, v in cols.items():
        if k in named_columns:
          raise ValueError(
              f'The field name "{k}" was specified more than once.')
        if not isinstance(v, str):
          raise ValueError(
              f'Invalid column "{v}" in "fields". Column name must be a str.')
        output_fields.append(f'{input}.{v} AS {k}')
        named_columns.add(k)
    else:
      raise ValueError(

View on GitHub (pinned to 12126d8942)