apache/beam · error · ValueError

Unknown or unsupported atomic type

Error message

Unknown or unsupported atomic type: {beam_type.atomic_type}

What it means

The `_validator` helper builds runtime type-check callables from a Beam schema FieldType. It only understands the atomic types BOOLEAN, INT64, DOUBLE, STRING, and BYTES; any other atomic type (e.g. INT32, FLOAT, TIMESTAMP, or logical types) hits this ValueError.

Solutions

1. Use a supported output_type: one of boolean, string, bytes, integer ('int64'), or number ('double').
2. For dates/timestamps, emit strings and declare output_type: string, converting at a later step.
3. If a new type is genuinely needed, extend _validator in yaml_mapping.py to handle that schema_pb2 atomic type.

Example fix

# before
fields:
  ts:
    expression: "parse_time(created)"
    output_type: timestamp
# after
fields:
  ts:
    expression: "str(parse_time(created))"
    output_type: string

SUPPORTED = {'boolean', 'string', 'bytes', 'integer', 'number'}
assert output_type in SUPPORTED, f'unsupported output_type: {output_type}'

def has_supported_atomic_type(spec: dict) -> bool:
    t = spec.get('type') if isinstance(spec, dict) else spec
    return t in ('boolean', 'string', 'bytes', 'integer', 'number')

try:
    run_pipeline(yaml_spec)
except ValueError as e:
    if 'Unknown or unsupported atomic type' in str(e):
        yaml_spec = rewrite_output_types_to_supported(yaml_spec)
        run_pipeline(yaml_spec)
    else:
        raise

When it happens

Trigger: Specifying `output_type` on a MapToFields field with a type that maps to an unsupported Beam atomic type, e.g. output_type: 'timestamp', 'float', 'int32', or a logical/logicalType-based schema, when _as_callable wraps the func in checking_func.

Common situations: Declaring output_type with a JSON-schema style type name that json_utils.json_type_to_beam_type maps to an unsupported atomic; users writing 'float' (maps to FLOAT atomic) instead of the supported 'number', or using date/timestamp types.

Related errors

• Can only use expressions on a schema'd input.
• File " " is not a valid .js file.
• File " " is not a valid .py file.
• Received an empty Map, but output schema contains required…
• Received an empty YAML string, but output schema contains…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/7a2174d401e9f1f0. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/yaml/yaml_mapping.py:344

  return python_callable.PythonCallableWithSource(source)


def _validator(beam_type: schema_pb2.FieldType) -> Callable[[Any], bool]:
  """Returns a callable converting rows of the given type to Json objects."""
  type_info = beam_type.WhichOneof("type_info")
  if type_info == "atomic_type":
    if beam_type.atomic_type == schema_pb2.BOOLEAN:
      return lambda x: isinstance(x, bool)
    elif beam_type.atomic_type == schema_pb2.INT64:
      return lambda x: isinstance(x, int)
    elif beam_type.atomic_type == schema_pb2.DOUBLE:
      return lambda x: isinstance(x, (int, float))
    elif beam_type.atomic_type == schema_pb2.STRING:
      return lambda x: isinstance(x, str)
    elif beam_type.atomic_type == schema_pb2.BYTES:
      return lambda x: isinstance(x, bytes)
    else:
      raise ValueError(
          f'Unknown or unsupported atomic type: {beam_type.atomic_type}')
  elif type_info == "array_type":
    element_validator = _validator(beam_type.array_type.element_type)
    return lambda value: all(element_validator(e) for e in value)
  elif type_info == "iterable_type":
    element_validator = _validator(beam_type.iterable_type.element_type)
    return lambda value: all(element_validator(e) for e in value)
  elif type_info == "map_type":
    key_validator = _validator(beam_type.map_type.key_type)
    value_validator = _validator(beam_type.map_type.value_type)
    return lambda value: all(
        key_validator(k) and value_validator(v) for (k, v) in value.items())
  elif type_info == "row_type":
    validators = {
        field.name: _validator(field.type)
        for field in beam_type.row_type.schema.fields
    }
    return lambda row: all(

View on GitHub (pinned to 12126d8942)