{"record":{"id":"8a5c984e8e52b75d","repo":"apache/beam","slug":"union-type-is-not-supported-for-column-col-name-please-pass","errorCode":null,"errorMessage":"Union type is not supported for column: {col_name}. Please pass a PCollection with valid schema for column {col_name} by passing a single type in container. For example, list[int].","messagePattern":"Union type is not supported for column: (.+?)\\. Please pass a PCollection with valid schema for column (.+?) by passing a single type in container\\. For example, list\\[int\\]\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/ml/transforms/handlers.py","lineNumber":263,"sourceCode":"    Args:\n      typ: A type of the column.\n      col_name: A name of the column.\n    Returns:\n      A FeatureSpec object.\n    \"\"\"\n    # lets conver the builtin types to typing types for consistency.\n    typ = native_type_compatibility.convert_builtin_to_typing(typ)\n    primitive_containers_type = (\n        list,\n        collections.abc.Sequence,\n    )\n    is_primitive_container = (\n        typing.get_origin(typ) in primitive_containers_type)\n\n    if is_primitive_container:\n      dtype = typing.get_args(typ)[0]\n      if len(typing.get_args(typ)) > 1 or typing.get_origin(dtype) == Union:\n        raise RuntimeError(\n            f\"Union type is not supported for column: {col_name}. \"\n            f\"Please pass a PCollection with valid schema for column \"\n            f\"{col_name} by passing a single type \"\n            \"in container. For example, list[int].\")\n    elif issubclass(typ, np.generic) or typ in _default_type_to_tensor_type_map:\n      dtype = typ\n    else:\n      raise TypeError(\n          f\"Unable to identify type: {typ} specified on column: {col_name}. \"\n          f\"Please provide a valid type from the following: \"\n          f\"{_default_type_to_tensor_type_map.keys()}\")\n    return tf.io.VarLenFeature(_default_type_to_tensor_type_map[dtype])\n\n  def get_raw_data_metadata(\n      self, input_types: dict[str, type]) -> dataset_metadata.DatasetMetadata:\n    raw_data_feature_spec = self.get_raw_data_feature_spec(input_types)\n    raw_data_feature_spec[_TEMP_KEY] = tf.io.VarLenFeature(dtype=tf.string)\n    return self.convert_raw_data_feature_spec_to_dataset_metadata(","sourceCodeStart":245,"sourceCodeEnd":281,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/ml/transforms/handlers.py#L245-L281","documentation":"When building the feature spec for each column, MLTransform inspects the PCollection row type. If a column's type is a container whose element type is a Union (e.g. list[Union[int, str]] or list[int|str]), or has multiple type args, it cannot map to a single tensor dtype, so RuntimeError is raised.","triggerScenarios":"Declaring a PCollection schema column typed as Optional/list with union element types, then running MLTransform.get_raw_data_feature_spec over it.","commonSituations":"Using TypedDict/NamedTuple rows where a field is Optional[str] inside a list; inference-produced types like list[int | None]; loose annotations like list[Any] resolving to Union.","solutions":["Constrain the column to a single concrete type, e.g. list[int] instead of list[Union[int, str]].","Split mixed-type data into separate columns, one per type.","Coerce/normalize values before the transform so the annotation is a single type."],"exampleFix":"// before\nclass Row(TypedDict):\n    values: List[Union[int, str]]\n// after\nclass Row(TypedDict):\n    values: List[int]","handlingStrategy":"type-guard","validationCode":"import typing\ndef col_type_is_simple(typ) -> bool:\n    origin = typing.get_origin(typ)\n    if origin in (list, tuple, set):\n        args = typing.get_args(typ)\n        return len(args) == 1 and typing.get_origin(args[0]) != Union\n    return True","typeGuard":"def has_union_element(typ) -> bool:\n    if typing.get_origin(typ) in (list, tuple, set):\n        args = typing.get_args(typ)\n        return len(args) > 1 or typing.get_origin(args[0]) == Union\n    return False","tryCatchPattern":"try:\n    spec = mltransform.get_raw_data_feature_spec(schema)\nexcept RuntimeError as e:\n    if 'Union type is not supported' in str(e):\n        raise SchemaError('normalize column types before MLTransform') from e","preventionTips":["Annotate schema fields with single concrete types","Avoid Union/Optional inside container element types","Unit-test feature spec generation on your schema"],"tags":["python","schema","typing","mltransform"],"backgroundTag":"schema-validation-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}