{"record":{"id":"4914416fa1bb2ea7","repo":"apache/beam","slug":"missing-language-specification-unknown-input-fields-or","errorCode":null,"errorMessage":"Missing language specification, unknown input fields, or invalid generic expression: {expr}. The given input fields are {input_fields}. See https://beam.apache.org/documentation/sdks/yaml-udf/#generic","messagePattern":"Missing language specification, unknown input fields, or invalid generic expression: (.+?)\\. The given input fields are (.+?)\\. See https://beam\\.apache\\.org/documentation/sdks/yaml-udf/#generic","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/yaml/yaml_mapping.py","lineNumber":135,"sourceCode":"        \"Must specify a language when using a map with custom logic for %s\" %\n        error_field)\n  expr = str(expr_dict['expression'])\n\n  def is_atomic(expr: str):\n    return is_literal(expr) or expr in input_fields\n\n  if is_atomic(expr):\n    return\n\n  if allow_cmp:\n    maybe_cmp = re.fullmatch('(.*)([<>=!]+)(.*)', expr)\n    if maybe_cmp:\n      left, cmp, right = maybe_cmp.groups()\n      if (is_atomic(left.strip()) and is_atomic(right.strip()) and\n          cmp in {'==', '<=', '>=', '<', '>', '!='}):\n        return\n\n  raise ValueError(\n      \"Missing language specification, unknown input fields, \"\n      f\"or invalid generic expression: {expr}. \"\n      f\"The given input fields are {input_fields}. \"\n      \"See https://beam.apache.org/documentation/sdks/yaml-udf/#generic\")\n\n\ndef validate_generic_expressions(base_type, config, input_pcolls) -> None:\n  if not input_pcolls:\n    return\n  try:\n    input_fields = [\n        name for (name, _) in named_fields_from_element_type(\n            next(iter(input_pcolls)).element_type)\n    ]\n  except (TypeError, ValueError):\n    input_fields = []\n\n  if base_type == 'MapToFields':","sourceCodeStart":117,"sourceCodeEnd":153,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/yaml/yaml_mapping.py#L117-L153","documentation":"After checking the map form, validate_generic_expression verifies a bare-string expression is either atomic (a known input field or literal) or a simple comparison of atomic operands. This final ValueError fires when the string expression references unknown input fields or is too complex to be handled generically, and no language was specified.","triggerScenarios":"Passing a plain-string expression like \"a + b\" or \"foo == 1\" where 'a', 'b', 'foo' are not in input_fields, or an expression more complex than atomic/atomic comparisons, to validate_generic_expression without a language-specifying map.","commonSituations":"Typos in field names; referencing fields that don't exist in the input schema; using arithmetic/complex logic that requires explicitly choosing a language (python/jinja/javascript).","solutions":["Fix field names to match the input schema (compare against input_fields).","Simplify the expression to atomic terms or simple comparisons (==, <=, >=, <, >, !=).","Specify a language explicitly via the map form: {expression: ..., language: python}.","Consult the YAML UDF docs at the URL in the message for supported generic expressions."],"exampleFix":"# before\nexpression: \"toatal + amount\"\n# after\nexpression: \"total + amount\"  # or\nconfiguration:\n  language: python\n  expression: \"total + amount\"","handlingStrategy":"validation","validationCode":"fields = set(input['schema'].keys())\nused = re.findall(r'[A-Za-z_]\\w*', expr)\nunknown = set(used) - fields - {'True', 'False', 'None', 'and', 'or', 'not', 'in'}\nassert not unknown, f'unknown fields: {unknown}'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Cross-check expression identifiers against the input schema","Use language-tagged maps for anything beyond simple comparisons","Test pipelines with a small sample input first"],"tags":["yaml","beam-yaml","expression-validation"],"backgroundTag":"invalid-argument-value","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}