apache/beam · error · IllegalArgumentException

Missing serialized typeDescriptor

Error message

Missing serialized typeDescriptor

What it means

When reconstructing a SchemaCoder from its proto payload, the payload must carry at least one additional coder info containing the serialized TypeDescriptor. If SchemaCoderPayload has none, fromComponents throws IllegalArgumentException('Missing serialized typeDescriptor'), since a SchemaCoder cannot be rebuilt without knowing its element type.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/util/construction/CoderTranslators.java:233

        payload
            .addAdditionalCoderInfosBuilder()
            .setUrn(TYPE_DESCRIPTOR_URN)
            .setPayload(
                ByteString.copyFrom(
                    SerializableUtils.serializeToByteArray(from.getEncodedTypeDescriptor())));
        return payload.build().toByteArray();
      }

      @Override
      public SchemaCoder<T> fromComponents(
          List<Coder<?>> components, byte[] payload, TranslationContext context) {
        checkArgument(
            components.isEmpty(), "Expected empty component list, but received: %s", components);
        try {
          SchemaApi.SchemaCoderPayload schemaCoderPayload =
              SchemaApi.SchemaCoderPayload.parseFrom(payload);
          if (schemaCoderPayload.getAdditionalCoderInfosCount() == 0) {
            throw new IllegalArgumentException("Missing serialized typeDescriptor");
          }
          TypeDescriptor<T> typeDescriptor =
              (TypeDescriptor<T>)
                  SerializableUtils.deserializeFromByteArray(
                      schemaCoderPayload.getAdditionalCoderInfos(0).getPayload().toByteArray(),
                      "typeDescriptor");
          SerializableFunction<T, Row> toRowFunction =
              (SerializableFunction<T, Row>)
                  SerializableUtils.deserializeFromByteArray(
                      schemaCoderPayload.getToRowFn().getPayload().toByteArray(), "toRowFunction");
          SerializableFunction<Row, T> fromRowFunction =
              (SerializableFunction<Row, T>)
                  SerializableUtils.deserializeFromByteArray(
                      schemaCoderPayload.getFromRowFn().getPayload().toByteArray(),
                      "fromRowFunction");

          Schema schema = SchemaTranslation.schemaFromProto(schemaCoderPayload.getSchema());
          return SchemaCoder.of(schema, typeDescriptor, toRowFunction, fromRowFunction);

View on GitHub (pinned to 12126d8942)

Solutions

1. Regenerate the pipeline proto with the same (or newer, compatible) Beam SDK version that writes the typeDescriptor info
2. If building SchemaCoderPayload manually, set additional coder infos with the SerializableUtils-serialized TypeDescriptor
3. Audit custom graph-export code for dropped additionalCoderInfos

Example fix

// before
SchemaCoderPayload.newBuilder().setSchema(schemaProto).build(); // no infos
// after
SchemaCoderPayload.newBuilder()
    .setSchema(schemaProto)
    .addAdditionalCoderInfos(CoderInfo.newBuilder()
        .setPayload(ByteString.copyFrom(SerializableUtils.serializeToByteArray(typeDescriptor))))
    .build();

SchemaApi.SchemaCoderPayload p = SchemaApi.SchemaCoderPayload.parseFrom(payload);
if (p.getAdditionalCoderInfosCount() == 0) {
  throw new IllegalArgumentException("payload lacks typeDescriptor; regenerate with Beam >= writer version");
}

try {
  return CoderTranslation.fromProto(coderProto, components);
} catch (IllegalArgumentException e) {
  if (e.getMessage().contains("Missing serialized typeDescriptor")) {
    throw new IOException("SchemaCoder payload missing typeDescriptor", e);
  }
  throw e;
}

When it happens

Trigger: Decoding a SchemaCoder payload that was written by an older Beam version lacking additionalCoderInfos, or a hand-constructed SchemaApi.SchemaCoderPayload without the typeDescriptor info entry.

Common situations: Cross-version pipeline proto incompatibilities (schema coder payload format changed over Beam releases); custom serialization tools that build the payload incompletely.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

• Unable to parse schema for RowCoder:
• Failed to convert PipelineOptions to JSON
• Internal error determining boundedness of Read
• Required field 'timestamp_transform' not set in %s
• Required field 'trigger' not set in %s

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/a3cd5cb4ef881e5f. Report an issue: GitHub.