apache/beam · error · ValueError

cross_product must be specified true or false when…

Error message

cross_product must be specified true or false when exploding multiple fields

What it means

The _Explode transform (FlatMap in YAML) accepts an optional `cross_product` flag that is only meaningful when exploding more than one field. If multiple fields are given and cross_product is not explicitly set to true or false, __init__ raises this ValueError because the combination semantics would be ambiguous.

Solutions

1. Add `cross_product: true` to produce all combinations, or `cross_product: false` to zip fields element-wise.
2. If you only need one field exploded, remove the extra field names from `fields`.
3. Pass an explicit boolean programmatically (cross_product=True/False) when constructing _Explode in code.

Example fix

# before
- type: FlatMap
  config:
    fields: [tags, scores]
# after
- type: FlatMap
  config:
    fields: [tags, scores]
    cross_product: true

fields = [fields] if isinstance(fields, str) else fields
if len(fields) > 1:
    assert cross_product in (True, False), 'cross_product required for multiple fields'

def explode_config_ok(cfg: dict) -> bool:
    f = cfg.get('fields')
    f = [f] if isinstance(f, str) else (f or [])
    return len(f) <= 1 or isinstance(cfg.get('cross_product'), bool)

try:
    out = pcoll | _Explode(fields=fields, cross_product=cross_product)
except ValueError as e:
    if 'cross_product must be specified' in str(e):
        out = pcoll | _Explode(fields=fields, cross_product=True)
    else:
        raise

When it happens

Trigger: Configuring `fields: [a, b]` (a list of two or more iterable fields) in a FlatMap/Explode transform without specifying `cross_product: true|false`.

Common situations: Adding a second field to an existing single-field explode config that never needed cross_product before, copy-pasting examples, or passing fields as an already-multi-element list programmatically with cross_product left None.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

• Ambiguous expression type (perhaps missing quoting?)
• CombineFn spec missing type
• Missing combine parameter in Combine config.
• Unknown CombineFn
• Unknown language for mapping transform

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/719aee16fc4281bd. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/yaml/yaml_mapping.py:575

          of the second field, etc. For example, the row
          `(['a', 'b'], [1, 2])` would expand to the four rows
          `('a', 1)`, `('a', 2)`, `('b', 1)`, and `('b', 2)` when
          `cross_product` is set to `true` but only the two rows
          `('a', 1)` and `('b', 2)` when it is set to `false`.
          Only meaningful (and required) if multiple rows are specified.
      error_handling: Whether and how to handle errors during iteration.
  """  # pylint: disable=line-too-long

  def __init__(
      self,
      fields: Union[str, Collection[str]],
      cross_product: Optional[bool] = None,
      error_handling: Optional[Mapping[str, Any]] = None):
    if isinstance(fields, str):
      fields = [fields]
    if cross_product is None:
      if len(fields) > 1:
        raise ValueError(
            'cross_product must be specified true or false '
            'when exploding multiple fields')
      else:
        # Doesn't matter.
        cross_product = True
    self._fields = fields
    self._cross_product = cross_product
    # TODO(yaml):
    # 1. Support standard error handling argument.
    # 2. Supposedly error_handling parameter is not an accepted parameter when
    #    executing.  Needs further investigation.
    self._exception_handling_args = exception_handling_args(error_handling)

  @maybe_with_exception_handling
  def expand(self, pcoll):
    all_fields = [
        x for x, _ in named_fields_from_element_type(pcoll.element_type)
    ]

View on GitHub (pinned to 12126d8942)