apache/beam · error · ValueError

Missing language specification. Must specify a language…

Error message

Missing language specification. Must specify a language when using a map with custom logic for %s

What it means

When the mapping custom logic IS a dict, Beam YAML requires exactly one key 'expression' plus a language specification for custom map logic. This ValueError fires when the dict has more or fewer than one key, or lacks the 'expression' key, meaning the language (e.g. python/javascript/callable config) cannot be determined.

Solutions

1. Use exactly one key 'expression' in the map: {expression: "col + 1"}.
2. Remove extra keys from the mapping or move them to the correct config level.
3. Fix typos so the key is literally 'expression'.
4. Use 'callable' or 'path'/'name' options instead if you intend non-expression custom logic.

Example fix

# before
configuration:
  expression: "x + 1"
  language: python   # extra key inside wrong level
# after
configuration:
  language: python
  expression: "x + 1"   # language beside expression at spec level

if isinstance(cfg, dict) and (len(cfg) != 1 or 'expression' not in cfg):
    raise ValueError('use exactly one key: expression')

When it happens

Trigger: Calling validate_generic_expression with a dict that is not exactly {expression: <str>} — e.g. {expression: x, extra: y}, {callable: ...}, {} or {language: python} without 'expression'.

Common situations: Users combining options like expression and callable in one map; typos like 'expr' instead of 'expression'; forgetting to nest language config correctly in the YAML transform spec.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

• Ambiguous expression type (perhaps missing quoting?)
• Ambiguous expression type (perhaps missing quoting?)
• Can only use expressions on a schema'd input.
• CombineFn spec missing type
• Config for transform at

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/48674db721d226a4. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/yaml/yaml_mapping.py:115

    return True
  elif re.fullmatch(r'-?\d+\.\d*', expr):
    return True
  elif re.fullmatch(r'"[^\\"]*"', expr):
    return True
  else:
    return False


def validate_generic_expression(
    expr_dict: dict,
    input_fields: Collection[str],
    allow_cmp: bool,
    error_field: str) -> None:
  if not isinstance(expr_dict, dict):
    raise ValueError(
        f"Ambiguous expression type (perhaps missing quoting?): {expr_dict}")
  if len(expr_dict) != 1 or 'expression' not in expr_dict:
    raise ValueError(
        "Missing language specification. "
        "Must specify a language when using a map with custom logic for %s" %
        error_field)
  expr = str(expr_dict['expression'])

  def is_atomic(expr: str):
    return is_literal(expr) or expr in input_fields

  if is_atomic(expr):
    return

  if allow_cmp:
    maybe_cmp = re.fullmatch('(.*)([<>=!]+)(.*)', expr)
    if maybe_cmp:
      left, cmp, right = maybe_cmp.groups()
      if (is_atomic(left.strip()) and is_atomic(right.strip()) and
          cmp in {'==', '<=', '>=', '<', '>', '!='}):
        return

View on GitHub (pinned to 12126d8942)