apache/beam · error · ValueError

RAW format does not take a schema

Error message

RAW format does not take a schema

What it means

For the 'RAW' format in _create_parser (read_from_pubsub), each message is treated as opaque bytes mapped to a single 'payload' field, so no schema is meaningful. If a schema is supplied anyway, the wrapper raises ValueError('RAW format does not take a schema').

Source

Thrown at sdks/python/apache_beam/yaml/yaml_io.py:246

  return WriteToBigQueryHandlingErrors()


def _create_parser(
    format,
    schema: Any) -> tuple[schema_pb2.Schema, Callable[[bytes], beam.Row]]:

  format = format.upper()

  def _validate_schema():
    if not schema:
      raise ValueError(
          f'{format} format requires valid {format} schema to be passed to '
          f'schema parameter.')

  if format == 'RAW':
    if schema:
      raise ValueError('RAW format does not take a schema')
    return (
        schema_pb2.Schema(fields=[schemas.schema_field('payload', bytes)]),
        lambda payload: beam.Row(payload=payload))
  if format == 'STRING':
    if schema:
      raise ValueError('STRING format does not take a schema')
    return (
        schema_pb2.Schema(fields=[schemas.schema_field('payload', str)]),
        lambda payload: beam.Row(payload=payload.decode('utf-8')))
  elif format == 'JSON':
    _validate_schema()
    beam_schema = json_utils.json_schema_to_beam_schema(schema)
    return beam_schema, json_utils.json_parser(beam_schema, schema)
  elif format == 'AVRO':
    _validate_schema()
    beam_schema = avroio.avro_schema_to_beam_schema(schema)
    covert_to_row = avroio.avro_dict_to_beam_row(schema, beam_schema)
    return (

View on GitHub (pinned to 12126d8942)

Solutions

1. Remove the schema parameter when format is 'RAW'.
2. If a schema is genuinely needed to parse structured messages, keep format as 'JSON'/'AVRO'/'PROTO' instead.
3. Use 'STRING' for plain text messages (also schema-less).

Example fix

// before
- type: ReadFromPubSub
  format: RAW
  schema: '{...}'
// after
- type: ReadFromPubSub
  format: RAW

if fmt.upper() == 'RAW' and schema:
    raise ValueError("Drop 'schema' when format is RAW")

try:
    read_from_pubsub(format='RAW', schema=schema)
except ValueError as e:
    if 'RAW format does not take a schema' in str(e):
        schema = None

When it happens

Trigger: Calling read_from_pubsub with format='RAW' and a non-empty schema argument; leaving a schema configured while switching format from JSON/AVRO to RAW.

Common situations: Switching formats in an existing pipeline config and forgetting to remove the now-unneeded schema field.

Related errors

• STRING format does not take a schema
• {format} format requires valid {format} schema to be passed
• Unknown format: {format}
• Only one of topic and subscription may be specified.
• Either data (%r) or attributes (%r) must be set.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/0b35ff2fe74436b9. Report an issue: GitHub.