{"record":{"id":"29dbb40e342fbd49","repo":"apache/beam","slug":"raw-output-only-supports-bytes-and-string-fields-got-s","errorCode":null,"errorMessage":"Raw output only supports bytes and string fields, got %s","messagePattern":"Raw output only supports bytes and string fields, got (.+?)","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubWriteSchemaTransformProvider.java","lineNumber":203,"sourceCode":"        }\n        payloadSchema = payloadSchemaBuilder.build();\n      }\n      SerializableFunction<Row, byte[]> fn;\n      if (Objects.equals(format, \"RAW\")) {\n        if (payloadSchema.getFieldCount() != 1) {\n          throw new IllegalArgumentException(\n              String.format(\n                  \"Raw output only supported for single-field schemas, got %s\", payloadSchema));\n        }\n        if (payloadSchema.getField(0).getType().equals(Schema.FieldType.BYTES)) {\n          fn = row -> checkArgumentNotNull(row.getBytes(0), \"Payload bytes value cannot be null\");\n        } else if (payloadSchema.getField(0).getType().equals(Schema.FieldType.STRING)) {\n          fn =\n              row ->\n                  checkArgumentNotNull(row.getString(0), \"Payload string value cannot be null\")\n                      .getBytes(StandardCharsets.UTF_8);\n        } else {\n          throw new IllegalArgumentException(\n              String.format(\n                  \"Raw output only supports bytes and string fields, got %s\",\n                  payloadSchema.getField(0)));\n        }\n      } else if (Objects.equals(format, \"JSON\")) {\n        fn = JsonUtils.getRowToJsonBytesFunction(payloadSchema);\n      } else if (Objects.equals(format, \"AVRO\")) {\n        fn = AvroUtils.getRowToAvroBytesFunction(payloadSchema);\n      } else {\n        throw new IllegalArgumentException(\n            String.format(\n                \"Format %s not supported. Only supported formats are %s\",\n                format, VALID_FORMATS_STR));\n      }\n\n      PCollectionTuple outputTuple =\n          input\n              .get(\"input\")","sourceCodeStart":185,"sourceCodeEnd":221,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubWriteSchemaTransformProvider.java#L185-L221","documentation":"PubsubWriteSchemaTransform.expand throws IllegalArgumentException when format is RAW and the single payload field's type is neither BYTES nor STRING. RAW publishing writes the field's bytes directly; other field types (INT64, ROW, ARRAY, etc.) have no defined raw byte encoding. The message includes the offending field.","triggerScenarios":"format=RAW with a single field of type INT64, TIMESTAMP, STRUCT/ROW, or ARRAY — e.g. publishing a numeric ID column as raw payload.","commonSituations":"Assuming RAW accepts any single-field schema; schema inference picked a numeric/logical type; Beam SQL query output a non-bytes single column into a RAW sink.","solutions":["Cast the field to BYTES or STRING upstream (e.g. CAST(id AS VARCHAR) in SQL, or a MapElements converting to bytes).","Switch to format=JSON, which supports arbitrary schema types.","If the field is STRING, ensure values are non-null or add a null-check/default upstream."],"exampleFix":"// before\nrows.apply(\"write\", pubsubWrite.withFormat(\"RAW\")); // single INT64 field\n// after\nrows.apply(SqlTransform.query(\"SELECT CAST(id AS VARCHAR) AS id FROM PCOLLECTION\"))\n    .apply(\"write\", pubsubWrite.withFormat(\"RAW\"));","handlingStrategy":"validation","validationCode":"Schema.FieldType t = schema.getField(0).getType();\nif (!t.equals(Schema.FieldType.BYTES) && !t.equals(Schema.FieldType.STRING)) throw new IllegalArgumentException(\"RAW field must be BYTES/STRING, got \" + t);","typeGuard":"null","tryCatchPattern":"try { expand(input); } catch (IllegalArgumentException e) { /* cast field or switch format */ }","preventionTips":["For RAW sinks, enforce schema fields to BYTES or STRING at design time.","Use JSON format for heterogeneous or numeric single-field payloads.","Add unit tests asserting the payload schema type before expansion."],"tags":["java","pubsub","schema","type-mismatch"],"backgroundTag":"schema-validation-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}