apache/beam · error · ValueError

Expecting exactly one field, found {field_names}

Error message

Expecting exactly one field, found {field_names}

What it means

In Apache Beam YAML's write_to_pubsub, the RAW format sends only a single field of each row as the raw Pub/Sub message payload. _create_formatter raises this ValueError when the beam schema does not have exactly one field, listing the fields it actually found.

Source

Thrown at sdks/python/apache_beam/yaml/yaml_io.py:287

    beam_schema = json_utils.json_schema_to_beam_schema(schema)
    return beam_schema, RowCoder(beam_schema).decode
  else:
    raise ValueError(f'Unknown format: {format}')


def _create_formatter(
    format, schema: Any,
    beam_schema: schema_pb2.Schema) -> Callable[[beam.Row], bytes]:

  if format.islower():
    format = format.upper()

  if format == 'RAW':
    if schema:
      raise ValueError('RAW format does not take a schema')
    field_names = [field.name for field in beam_schema.fields]
    if len(field_names) != 1:
      raise ValueError(f'Expecting exactly one field, found {field_names}')

    def convert_to_bytes(row):
      output = getattr(row, field_names[0])
      if isinstance(output, bytes):
        return output
      elif isinstance(output, str):
        return output.encode('utf-8')
      else:
        raise ValueError(
            f"Cannot encode payload for WriteToPubSub. "
            f"Expected valid string or bytes object, "
            f"got {repr(output)} of type {type(output)}.")

    return convert_to_bytes
  elif format == 'JSON':
    return json_utils.json_formater(beam_schema)
  elif format == 'AVRO':
    avro_schema = schema or avroio.beam_schema_to_avro_schema(beam_schema)

View on GitHub (pinned to 12126d8942)

Solutions

1. Change format to JSON or PROTO if you need multi-field payloads
2. Ensure the input schema has exactly one field (bytes or str) whose value is the raw payload
3. Move extra fields into Pub/Sub attributes instead of the payload

Example fix

// before
format: RAW, schema: {id: STRING, payload: BYTES}
// after
format: RAW, schema: {payload: BYTES}

names = [f.name for f in schema.fields]
if len(names) != 1:
    raise ValueError(f'RAW format needs exactly one field, got {names}')

def is_single_field_schema(schema):
    return schema is not None and len(schema.fields) == 1

When it happens

Trigger: Calling write_to_pubsub with format='RAW' and a schema whose beam_schema has zero or multiple fields (e.g. a two-column row schema).

Common situations: Users configure RAW Pub/Sub output but forget RAW only supports single-field payloads; often the schema still carries id+value or attribute columns instead of a lone bytes/str field.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Attribute fields {missing_attribute_names} not found in sche
• Schema must contain at least one field. Schema: %s
• serializable Row does not exist for payload of type: %s
• Unsupported schema specified for Pubsub source in CREATE TAB
• Unsupported schema specified for Pubsub source in CREATE TAB

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/646665e15466a236. Report an issue: GitHub.