apache/beam · error · ValueError

Invalid column " " in "fields". Column name must be a str.

Error message

Invalid column "{v}" in "fields". Column name must be a str.

What it means

Type guard inside _parse_fields for YAML Join: a column name listed under an input in 'fields' is not a string (e.g. a number or null parsed from the YAML), but SQL column references must be string identifiers.

Solutions

Quote the source column name so it stays a string, e.g. {x: "42"}.
Verify the value points at an actual string column in that input.
Compute/derive non-string values upstream in a separate transform before joining.

Example fix

// before
fields:
  left: {id: 42}
// after
fields:
  left: {id: "42"}

Defensive patterns

Strategy: type-guard

Validate before calling

def check_dict_values(fields):
    for alias, cols in fields.items():
        if isinstance(cols, dict):
            assert all(isinstance(v, str) for v in cols.values()), f'non-str source column for {alias}'

Type guard

def dict_values_all_str(d):
    return isinstance(d, dict) and all(isinstance(v, str) for v in d.values())

Try / catch

try:
    result = SqlJoinTransform(config)
except ValueError as e:
    if 'Column name must be a str' in str(e):
        raise ConfigError('source column values in fields dicts must be strings') from e
    raise

Prevention

Quote YAML values that could parse as int/float/bool
Verify with yaml.safe_load that source column values are str
Only use plain column names as dict values, not expressions

When it happens

Trigger: fields: {left: {id: 42}} or a value that YAML parsed as an int/bool/float instead of a string column name.

Common situations: Unquoted numeric or boolean-looking column names in YAML; pasting expressions instead of plain column names into the source-column position.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/9579a8cdc119ed02. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/yaml/yaml_join.py:135

    if input not in tables:
      raise ValueError(f'An invalid input "{input}" was specified in "fields".')
    if isinstance(cols, list):
      for col in cols:
        if not isinstance(col, str):
          raise ValueError(
              f'Invalid column "{col}" in "fields". Column name must be a str.')
        if col in named_columns:
          raise ValueError(
              f'The field name "{col}" was specified more than once.')
        output_fields.append(f'{input}.{col} AS {col}')
        named_columns.add(col)
    elif isinstance(cols, dict):
      for k, v in cols.items():
        if k in named_columns:
          raise ValueError(
              f'The field name "{k}" was specified more than once.')
        if not isinstance(v, str):
          raise ValueError(
              f'Invalid column "{v}" in "fields". Column name must be a str.')
        output_fields.append(f'{input}.{v} AS {k}')
        named_columns.add(k)
    else:
      raise ValueError(
          f'{error_prefix} '
          f'For every input key in fields, '
          f'the value must either be a list or dict.')
  for table in tables:
    if table not in fields.keys():
      output_fields.append(f'{table}.*')
  return output_fields


def _is_connected(edge_list, expected_node_count):
  graph = {}
  for edge_set in edge_list:
    for u in edge_set:

View on GitHub (pinned to 12126d8942)