apache/beam · error · IllegalArgumentException

serializable Row does not exist for payload of type: %s

Error message

serializable Row does not exist for payload of type: %s

What it means

In PubsubRowToMessage's DoFn, serializableRow() converts a Row's payload field into a serializable Row representation. If the payload field is of the BYTES type (PAYLOAD_BYTES_TYPE_NAME), no serializable Row exists for it, so it throws an IllegalArgumentException.

Source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubRowToMessage.java:526

        return checkArgumentNotNull(
            row.getBytes(payloadKeyName), "Payload field '%s' cannot be null", payloadKeyName);
      }
      return checkStateNotNull(payloadSerializer).serialize(serializableRow(row));
    }

    /**
     * Extracts the serializable part of a {@link Row} from the following mutually exclusive
     * sources. <br>
     * - serialized {@link #payloadKeyName} {@link Field} with {@link TypeName#ROW} using the {@link
     * #payloadSerializer} <br>
     * - serialized user fields provided that are not {@link #attributesKeyName} and {@link
     * #sourceTimestampKeyName}
     */
    Row serializableRow(Row row) {
      SchemaReflection schemaReflection = SchemaReflection.of(row.getSchema());

      if (schemaReflection.matchesAll(FieldMatcher.of(payloadKeyName, PAYLOAD_BYTES_TYPE_NAME))) {
        throw new IllegalArgumentException(
            String.format(
                "serializable Row does not exist for payload of type: %s",
                PAYLOAD_BYTES_TYPE_NAME));
      }

      if (schemaReflection.matchesAll(FieldMatcher.of(payloadKeyName, PAYLOAD_ROW_TYPE_NAME))) {
        return checkArgumentNotNull(
            row.getRow(payloadKeyName), "Payload field '%s' cannot be null", payloadKeyName);
      }
      Schema withUserFieldsOnly =
          removeFields(row.getSchema(), attributesKeyName, sourceTimestampKeyName);
      Map<String, Object> values = new HashMap<>();
      for (String name : withUserFieldsOnly.getFieldNames()) {
        values.put(name, row.getValue(name));
      }
      return Row.withSchema(withUserFieldsOnly).withFieldValues(values).build();
    }
  }

View on GitHub (pinned to 12126d8942)

Solutions

Change the payload field's type in the schema from BYTES to a ROW type that serializableRow() can convert.
If the payload is genuinely raw bytes, use a pipeline path that supports byte payloads (e.g. RAW format) instead of PubsubRowToMessage.
Decode/parse the bytes into a Row upstream so the payload field arrives as a structured Row.

Example fix

// before
Schema schema = Schema.builder().addByteArrayField("payload").build();
// after
Schema schema = Schema.builder().addRowField("payload", Schema.builder().addStringField("key").build()).build();

Defensive patterns

Strategy: validation

Validate before calling

Schema.Field payload = row.getSchema().getField(payloadKeyName);
if (payload != null && payload.getType().getTypeName().equals(org.apache.beam.sdk.schemas.Schema.TypeName.BYTES)) { throw new IllegalArgumentException("payload must be ROW, not BYTES, for PubsubRowToMessage"); }

Type guard

boolean payloadIsRow(org.apache.beam.sdk.values.Row row, String name) { Schema.Field f = row.getSchema().getField(name); return f != null && f.getType().getTypeName() == org.apache.beam.sdk.schemas.Schema.TypeName.ROW; }

Try / catch

try { rowToMessage.expand(input); } catch (IllegalArgumentException e) { if (e.getMessage().contains("serializable Row does not exist")) { /* change payload field type to ROW */ } throw e; }

Prevention

Declare the payload field as a ROW type when using Row-to-Message conversion; reserve BYTES for the RAW path.
Check payload field types at pipeline construction time, not at runtime in the DoFn.

When it happens

Trigger: Processing a Row whose payload field (named per payloadKeyName) is declared as BYTES in the schema, causing schemaReflection.matchesAll(FieldMatcher.of(payloadKeyName, PAYLOAD_BYTES_TYPE_NAME)) to be true during message conversion.

Common situations: Schemas where the payload was declared as bytes (raw payload) but the Row-to-Message transform expects a structured ROW-typed payload; schema drift between pipeline stages; switching from RAW format to structured format without changing the schema.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/d41f1876d41478db. Report an issue: GitHub.