apache/beam · error · IllegalArgumentException

Raw output only supports bytes and string fields, got

Error message

Raw output only supports bytes and string fields, got %s

What it means

PubsubWriteSchemaTransform.expand throws IllegalArgumentException when format is RAW and the single payload field's type is neither BYTES nor STRING. RAW publishing writes the field's bytes directly; other field types (INT64, ROW, ARRAY, etc.) have no defined raw byte encoding. The message includes the offending field.

Solutions

Cast the field to BYTES or STRING upstream (e.g. CAST(id AS VARCHAR) in SQL, or a MapElements converting to bytes).
Switch to format=JSON, which supports arbitrary schema types.
If the field is STRING, ensure values are non-null or add a null-check/default upstream.

Example fix

// before
rows.apply("write", pubsubWrite.withFormat("RAW")); // single INT64 field
// after
rows.apply(SqlTransform.query("SELECT CAST(id AS VARCHAR) AS id FROM PCOLLECTION"))
    .apply("write", pubsubWrite.withFormat("RAW"));

Defensive patterns

Strategy: validation

Validate before calling

Schema.FieldType t = schema.getField(0).getType();
if (!t.equals(Schema.FieldType.BYTES) && !t.equals(Schema.FieldType.STRING)) throw new IllegalArgumentException("RAW field must be BYTES/STRING, got " + t);

Type guard

null

Try / catch

try { expand(input); } catch (IllegalArgumentException e) { /* cast field or switch format */ }

Prevention

For RAW sinks, enforce schema fields to BYTES or STRING at design time.
Use JSON format for heterogeneous or numeric single-field payloads.
Add unit tests asserting the payload schema type before expansion.

When it happens

Trigger: format=RAW with a single field of type INT64, TIMESTAMP, STRUCT/ROW, or ARRAY — e.g. publishing a numeric ID column as raw payload.

Common situations: Assuming RAW accepts any single-field schema; schema inference picked a numeric/logical type; Beam SQL query output a non-bytes single column into a RAW sink.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/29dbb40e342fbd49. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubWriteSchemaTransformProvider.java:203

        }
        payloadSchema = payloadSchemaBuilder.build();
      }
      SerializableFunction<Row, byte[]> fn;
      if (Objects.equals(format, "RAW")) {
        if (payloadSchema.getFieldCount() != 1) {
          throw new IllegalArgumentException(
              String.format(
                  "Raw output only supported for single-field schemas, got %s", payloadSchema));
        }
        if (payloadSchema.getField(0).getType().equals(Schema.FieldType.BYTES)) {
          fn = row -> checkArgumentNotNull(row.getBytes(0), "Payload bytes value cannot be null");
        } else if (payloadSchema.getField(0).getType().equals(Schema.FieldType.STRING)) {
          fn =
              row ->
                  checkArgumentNotNull(row.getString(0), "Payload string value cannot be null")
                      .getBytes(StandardCharsets.UTF_8);
        } else {
          throw new IllegalArgumentException(
              String.format(
                  "Raw output only supports bytes and string fields, got %s",
                  payloadSchema.getField(0)));
        }
      } else if (Objects.equals(format, "JSON")) {
        fn = JsonUtils.getRowToJsonBytesFunction(payloadSchema);
      } else if (Objects.equals(format, "AVRO")) {
        fn = AvroUtils.getRowToAvroBytesFunction(payloadSchema);
      } else {
        throw new IllegalArgumentException(
            String.format(
                "Format %s not supported. Only supported formats are %s",
                format, VALID_FORMATS_STR));
      }

      PCollectionTuple outputTuple =
          input
              .get("input")

View on GitHub (pinned to 12126d8942)