apache/beam · error · java.lang.IllegalArgumentException

Invalid payload type for URN

Error message

Invalid payload type for URN 

What it means

getTransform attempts SchemaTransformPayload.parseFrom(spec.getPayload()) and, if the bytes are not a valid SchemaTransformPayload protobuf (InvalidProtocolBufferException), throws IllegalArgumentException with this message. It indicates the FunctionSpec payload was built for a different payload type or is corrupt.

Solutions

1. Ensure the client uses the matching Beam SDK version to construct the FunctionSpec so the payload is SchemaTransformPayload
2. Verify the URN actually corresponds to SCHEMA_TRANSFORM expansion (not JAVA_CLASS_LOOKUP or another method that expects a different payload)
3. Regenerate the request with the standard Beam transform expansion path instead of manual protobuf construction

Example fix

// before
FunctionSpec.newBuilder().setUrn(urn).setPayload(javaClassLookupPayload.toByteString())
// after
FunctionSpec.newBuilder().setUrn(urn).setPayload(SchemaTransformPayload.newBuilder().setIdentifier(id)...build().toByteString())

// Ensure payload type matches URN before sending:
assert spec.getUrn().equals(getUrn(ExpansionMethods.Enum.SCHEMA_TRANSFORM));

try { expanded = service.expand(request); } catch (IllegalArgumentException e) { if (e.getMessage().startsWith("Invalid payload type for URN")) { /* rebuild request payload */ } throw e; }

When it happens

Trigger: An expansion request for the SCHEMA_TRANSFORM URN whose spec payload is not serialized SchemaTransformPayload — e.g., payload produced by a different expansion method (JavaClassLookupPayload) or by a mismatched Beam/proto version.

Common situations: Cross-version client/server where the payload proto changed; custom runner sending the wrong payload bytes for the URN; hand-crafted FunctionSpec with wrong payload.

Understand the failure class

Background: "cannot parse invalid wire-format data", "cannot unmarshal", "failed unmarshalling": protobuf unmarshal errors explained — this error's family across 10 libraries.

Related errors

• Invalid payload type for URN
• Aliased enumerations not currently supported.
• Any not yet supported
• Cannot create creator for
• cannot encode a null

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/5ee270f6000489ba. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/expansion-service/src/main/java/org/apache/beam/sdk/expansion/service/ExpansionServiceSchemaTransformProvider.java:105

    for (String key : output.getAll().keySet()) {
      pCollectionMap.put(key, output.get(key));
    }
    return pCollectionMap.build();
  }

  @Override
  public PTransform getTransform(FunctionSpec spec, PipelineOptions options) {
    SchemaTransformPayload payload;
    try {
      payload = SchemaTransformPayload.parseFrom(spec.getPayload());
      String identifier = payload.getIdentifier();
      if (!schemaTransformProviders.containsKey(identifier)) {
        throw new RuntimeException(
            "Did not find a SchemaTransformProvider with the identifier " + identifier);
      }

    } catch (InvalidProtocolBufferException e) {
      throw new IllegalArgumentException(
          "Invalid payload type for URN " + getUrn(ExpansionMethods.Enum.SCHEMA_TRANSFORM), e);
    }

    String identifier = payload.getIdentifier();
    org.apache.beam.sdk.schemas.transforms.SchemaTransformProvider provider =
        schemaTransformProviders.get(identifier);
    if (provider == null) {
      throw new IllegalArgumentException(
          "Could not find a SchemaTransform with identifier " + identifier);
    }

    Schema configSchemaFromRequest =
        SchemaTranslation.schemaFromProto(payload.getConfigurationSchema());
    Schema configSchemaFromProvider = provider.configurationSchema();

    if (!configSchemaFromRequest.assignableTo(configSchemaFromProvider)) {
      throw new IllegalArgumentException(
          String.format(

View on GitHub (pinned to 12126d8942)