apache/beam · error · IllegalArgumentException

Raw output only supported for single-field schemas, got

Error message

Raw output only supported for single-field schemas, got %s

What it means

PubsubWriteSchemaTransform.expand throws IllegalArgumentException when format is RAW but the payload schema does not have exactly one field. RAW mode serializes a single bytes/string field directly as the message payload, so multi-field (or zero-field) schemas cannot be published raw. The message includes the offending schema.

Solutions

Use format=JSON (or AVRO) for multi-field schemas instead of RAW.
If RAW is required, project exactly one bytes/string field upstream (e.g. Select or SqlTransform selecting only the payload column).
Verify payloadSchema.getFieldCount() == 1 in your pipeline construction before applying the transform.

Example fix

// before
transform = PubsubWriteSchemaTransformProvider...
  .from(config.toBuilder().setFormat("RAW").build()); // schema has 2 fields
// after
row.apply(Select.fieldNames("payload")) // keep single field
   .apply(pubsubWrite.withFormat("RAW"));
// or switch:
.from(config.toBuilder().setFormat("JSON").build());

Defensive patterns

Strategy: validation

Validate before calling

if ("RAW".equals(format) && payloadSchema.getFieldCount() != 1) throw new IllegalArgumentException("RAW requires exactly one field, got " + payloadSchema.getFieldCount());

Type guard

null

Try / catch

try { expand(input); } catch (IllegalArgumentException e) { /* switch to JSON or project a single field */ }

Prevention

Only pair RAW with explicitly declared single-field schemas.
Project/Select the payload column before the sink.
Add a schema check step in pipeline construction for RAW sinks.

When it happens

Trigger: Configuring format=RAW with a schema having multiple columns (e.g. {id: INT64, payload: BYTES}) or an empty schema, typically from an inferred table schema in Beam YAML/SQL.

Common situations: Passing a full row schema from a database read straight into a RAW Pubsub sink; schema changed to add a column, breaking a previously working RAW sink.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/d65f4f03852fb791. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubWriteSchemaTransformProvider.java:191

      List<String> attributes = configuration.getAttributes();
      String attributesMap = configuration.getAttributesMap();
      if (attributes == null && attributesMap == null) {
        payloadSchema = beamSchema;
      } else {
        Schema.Builder payloadSchemaBuilder = Schema.builder();
        for (Schema.Field f : beamSchema.getFields()) {
          boolean isAttribute = attributes != null && attributes.contains(f.getName());
          boolean isAttributesMap = f.getName().equals(attributesMap);
          if (!isAttribute && !isAttributesMap) {
            payloadSchemaBuilder.addField(f);
          }
        }
        payloadSchema = payloadSchemaBuilder.build();
      }
      SerializableFunction<Row, byte[]> fn;
      if (Objects.equals(format, "RAW")) {
        if (payloadSchema.getFieldCount() != 1) {
          throw new IllegalArgumentException(
              String.format(
                  "Raw output only supported for single-field schemas, got %s", payloadSchema));
        }
        if (payloadSchema.getField(0).getType().equals(Schema.FieldType.BYTES)) {
          fn = row -> checkArgumentNotNull(row.getBytes(0), "Payload bytes value cannot be null");
        } else if (payloadSchema.getField(0).getType().equals(Schema.FieldType.STRING)) {
          fn =
              row ->
                  checkArgumentNotNull(row.getString(0), "Payload string value cannot be null")
                      .getBytes(StandardCharsets.UTF_8);
        } else {
          throw new IllegalArgumentException(
              String.format(
                  "Raw output only supports bytes and string fields, got %s",
                  payloadSchema.getField(0)));
        }
      } else if (Objects.equals(format, "JSON")) {
        fn = JsonUtils.getRowToJsonBytesFunction(payloadSchema);

View on GitHub (pinned to 12126d8942)