apache/beam · error · ValueError

Invalid value " " for "equalities". When "equalities" is a…

Error message

Invalid value "{equalities}" for "equalities". When "equalities" is a str, it must be a field name that exists in all the specified inputs.

What it means

Thrown by _validate_equalities in apache_beam/yaml/yaml_join.py when 'equalities' is given as a str but that field name does not exist in the schema of at least one of the specified inputs. A shorthand equality string must be a shared join key across all inputs.

Solutions

1. Use the dict/list form of equalities to map per-input field names: equalities: [{input1: id, input2: user_id}].
2. Rename the field with a Map/SQL projection so all inputs share the key name.
3. Inspect pcoll.element_type._fields for each input to confirm exact field names.

Example fix

// before
equalities: id  # input2 has 'user_id'
// after
equalities:
  - input1: id
    input2: user_id

if isinstance(eq, str):
    for tag, pc in inputs.items():
        if eq not in getattr(pc.element_type, '_fields', []):
            raise ValueError(f'field {eq!r} missing from input {tag}')

When it happens

Trigger: equalities: id when one input's schema lacks an 'id' field (e.g. it is named 'user_id'); field renames upstream; case-sensitive mismatch.

Common situations: Joining tables from different sources with inconsistent column names; forgetting to rename/alias fields before the join.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Can only append fields on a schema'd input.
• Can only drop fields on a schema'd input.
• Cannot convert element of type
• Dropping unknown field
• Exploding unknown field

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/92c6d1609a8e6b42. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/yaml/yaml_join.py:73

        f'{error_prefix} When specifying the value for type as a str, '
        f'it must be one of the following: "inner", "outer", "left", "right"')


def _validate_equalities(equalities, pcolls):
  error_prefix = f'Invalid value "{equalities}" for "equalities".'

  valid_cols = {
      name: set(
          dict(fields).keys() if fields and all(
              isinstance(field, tuple) for field in fields) else fields)
      for (name, pcoll) in pcolls.items()
      for fields in [getattr(pcoll.element_type, '_fields', [])]
  }

  if isinstance(equalities, str):
    for cols in valid_cols.values():
      if equalities not in cols:
        raise ValueError(
            f'{error_prefix} When "equalities" is a str, '
            f'it must be a field name that exists in all the specified inputs.')
    equality = {pcoll_tag: equalities for pcoll_tag in pcolls}
    return [equality]

  if not isinstance(equalities, list):
    raise ValueError(f'{error_prefix} It should be a str or a list.')

  input_edge_list = []
  for equality in equalities:
    invalid_dict_error = ValueError(
        f'{error_prefix} {equality} '
        f'should be a dict[str, str] containing at least 2 items.')
    if not isinstance(equality, dict):
      raise invalid_dict_error
    if len(equality) < 2:
      raise invalid_dict_error

View on GitHub (pinned to 12126d8942)