What causes "Missing combine parameter in Combine config." in apache/beam?

Add a 'combine' mapping in the transform config, e.g. combine: {total: {fn: sum, value: price}}.

apache/beam · error · ValueError

Missing combine parameter in Combine config.

Error message

Missing combine parameter in Combine config.

What it means

normalize_combine in apache_beam.yaml.yaml_combine requires the Combine transform's config to contain a 'combine' parameter mapping output fields to aggregation specs. Missing it means the transform has nothing to aggregate.

Solutions

Add a 'combine' mapping in the transform config, e.g. combine: {total: {fn: sum, value: price}}.
If you only need grouping without aggregation, use the GroupBy transform instead of Combine.
Validate the YAML against Beam's Combine schema docs before running.

Example fix

// before
- type: Combine
  input: my_input
  config:
    group_by: [user]
// after
- type: Combine
  input: my_input
  config:
    group_by: [user]
    combine:
      total: {fn: sum, value: amount}

Defensive patterns

Strategy: validation

Validate before calling

def validate_combine_config(config: dict):
    if not isinstance(config, dict) or 'combine' not in config:
        raise SystemExit('Combine transform requires config.combine, e.g. combine: {total: {fn: sum, value: price}}')

Type guard

def is_valid_combine_spec(config) -> bool:
    return isinstance(config, dict) and isinstance(config.get('combine'), dict) and len(config['combine']) > 0

Try / catch

try:
    normalize_combine(spec)
except ValueError as e:
    if 'Missing combine parameter' in str(e):
        raise SystemExit('Fix YAML: add a combine: mapping to the Combine transform config') from e
    raise

Prevention

Template Combine transforms with both group_by and combine sections
Lint YAML pipelines against the Beam transform schema in CI
Copy from official Beam YAML examples rather than hand-writing configs

When it happens

Trigger: Specifying a Combine transform in YAML whose config lacks the 'combine' key, e.g. config: {group_by: [...]} only.

Common situations: Hand-written YAML pipelines omitting the combine section; refactors that move aggregations out but leave an empty Combine; copy-paste from GroupBy examples.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/4869f65e86aed35a. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/yaml/yaml_combine.py:80

          type: fn_type
  """
  from apache_beam.yaml.yaml_transform import SafeLineLoader
  if spec['type'] == 'Combine':
    config = spec.get('config')
    if isinstance(config.get('group_by'), str):
      config['group_by'] = [config['group_by']]

    def normalize_agg(dest, agg):
      if isinstance(agg, str):
        agg = {'fn': agg}
      if 'value' not in agg and config.get('language') != 'sql':
        agg['value'] = dest
      if isinstance(agg['fn'], str):
        agg['fn'] = {'type': agg['fn']}
      return agg

    if 'combine' not in config:
      raise ValueError('Missing combine parameter in Combine config.')
    config['combine'] = {
        dest: normalize_agg(dest, agg)
        for (dest,
             agg) in SafeLineLoader.strip_metadata(config['combine']).items()
    }
  return spec


class PyJsYamlCombine(beam.PTransform):
  """Groups and combines records sharing common fields.

  Built-in combine functions are BUILTIN_COMBINE_FNS
  but custom aggregation functions can be used as well.

  See also the documentation on
  [YAML Aggregation](https://beam.apache.org/documentation/sdks/yaml-combine/).

  Args:

View on GitHub (pinned to 12126d8942)