apache/beam · error · TypeError

Unknown CombineFn

Error message

Unknown CombineFn: {fn_spec}

What it means

While building a CombineFn for a YAML group_by aggregation, create_combine_fn found a spec whose 'type' names neither a builtin CombineFn nor a loadable Python callable in the configured language; the full spec is shown so the unsupported name is identifiable.

Solutions

1. Use one of the documented builtin combine fn names (check BUILTIN_COMBINE_FNS).
2. For custom logic, set language: python and provide the full callable source as the type string.
3. Fix the casing/spelling of the fn type.

Example fix

// before
combine:
  total: {fn: {type: Sum}, value: amount}
// after
combine:
  total: {fn: {type: sum}, value: amount}

from apache_beam.yaml.yaml_combine import BUILTIN_COMBINE_FNS
def check_fn_type(fn_spec: dict, language: str):
    t = fn_spec.get('type')
    if t in BUILTIN_COMBINE_FNS:
        return
    if language != 'python':
        raise SystemExit(f'Unknown CombineFn {t!r}; use a builtin or set language: python with callable source')

def is_known_combine_fn(fn_spec: dict, language: str) -> bool:
    t = fn_spec.get('type')
    return isinstance(t, str) and (t in BUILTIN_COMBINE_FNS or language == 'python')

try:
    expand(pcoll)
except TypeError as e:
    if 'Unknown CombineFn' in str(e):
        raise SystemExit(f'Use one of {sorted(BUILTIN_COMBINE_FNS)} or provide a Python callable source') from e
    raise

When it happens

Trigger: fn: {type: sumAll} with no such builtin and language not python, or a Python source string that fails to load as a callable; also when language is java/sql but a Python-only spec is given.

Common situations: Typos in builtin names ('Sum' vs 'sum'); using a custom Python CombineFn while the transform is configured with language: java; forgetting to include the callable source inline.

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

• Ambiguous expression type (perhaps missing quoting?)
• CombineFn spec missing type
• cross_product must be specified true or false when…
• Missing combine parameter in Combine config.
• Unknown language for mapping transform

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/0a34793b5261b102. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/yaml/yaml_combine.py:133

    all_fields = list(input_types.keys())
    unknown_keys = set(self._group_by) - set(all_fields)
    if unknown_keys:
      raise ValueError(f'Unknown grouping columns: {list(unknown_keys)}')

    def create_combine_fn(fn_spec):
      if 'type' not in fn_spec:
        raise ValueError(f'CombineFn spec missing type: {fn_spec}')
      elif fn_spec['type'] in BUILTIN_COMBINE_FNS:
        return BUILTIN_COMBINE_FNS[fn_spec['type']]
      elif self._language == 'python':
        # TODO(yaml): Support output_type here as well.
        fn = python_callable.PythonCallableWithSource.load_from_source(
            fn_spec['type'])
        if 'config' in fn_spec:
          fn = fn(**fn_spec['config'])
        return fn
      else:
        raise TypeError('Unknown CombineFn: {fn_spec}')

    def extract_return_type(expr):
      if isinstance(expr, str) and expr in input_types:
        return input_types[expr]
      expr_hints = get_type_hints(expr)
      if (expr_hints and expr_hints.has_simple_output_type() and
          expr_hints.simple_output_type(None) != typehints.Any):
        return expr_hints.simple_output_type(None)
      elif callable(expr):
        return trivial_inference.infer_return_type(expr, [pcoll.element_type])
      else:
        return Any

    # TODO(yaml): Support error handling.
    transform = beam.GroupBy(*self._group_by)
    output_types = [(k, input_types[k]) for k in self._group_by]

    for output, agg in self._combine.items():

View on GitHub (pinned to 12126d8942)