apache/beam · error · RuntimeException

Unable to infer data schema from configuration proto.

Error message

Unable to infer data schema from configuration proto.

What it means

This RuntimeException is thrown by ExternalSchemaIOTransformRegistrar.translateSchema when the serialized configuration schema bytes cannot be parsed as a SchemaApi.Schema protobuf or translated into a Beam Schema. It indicates the expansion service received configuration whose schema payload is corrupt, truncated, or not in the expected protobuf format.

Source

Thrown at sdks/java/extensions/schemaio-expansion-service/src/main/java/org/apache/beam/sdk/extensions/schemaio/expansion/ExternalSchemaIOTransformRegistrar.java:110

      this.config = config;
    }

    public void setDataSchema(@Nullable byte[] dataSchema) {
      this.dataSchema = dataSchema;
    }
  }

  @Nullable
  private static Schema translateSchema(@Nullable byte[] schemaBytes) {
    if (schemaBytes == null) {
      return null;
    }

    try {
      SchemaApi.Schema protoSchema = SchemaApi.Schema.parseFrom(schemaBytes);
      return SchemaTranslation.schemaFromProto(protoSchema);
    } catch (InvalidProtocolBufferException e) {
      throw new RuntimeException("Unable to infer data schema from configuration proto.", e);
    }
  }

  private static Row translateRow(byte[] rowBytes, Schema configSchema) {
    RowCoder rowCoder = RowCoder.of(configSchema);
    InputStream stream = new ByteArrayInputStream(rowBytes);

    try {
      return rowCoder.decode(stream);
    } catch (IOException e) {
      throw new RuntimeException(
          "Unable to infer configuration row from configuration proto and schema.", e);
    }
  }

  @VisibleForTesting
  static class ReaderBuilder
      implements ExternalTransformBuilder<Configuration, PBegin, PCollection<Row>> {

View on GitHub (pinned to 12126d8942)

Solutions

Ensure the client and expansion service use the same Beam version so schema proto serialization is compatible.
Verify the configuration payload actually contains SchemaApi.Schema bytes for the schema field, not row bytes.
Regenerate/rebuild the expansion service jar and reconnect.
Log/inspect the failing bytes to confirm format; re-serialize the schema using SchemaTranslation.schemaToProto before sending.

Example fix

// before
byte[] payload = rowBytes; // wrong bytes
configProto = configProto.toBuilder().setSchemaBytes(payload).build();
// after
SchemaApi.Schema protoSchema = SchemaTranslation.schemaToProto(schema, true);
configProto = configProto.toBuilder().setSchemaBytes(protoSchema.toByteString()).build();

Defensive patterns

Strategy: try-catch

Validate before calling

// Validate config schema bytes before sending to the expansion service
try {
  SchemaApi.Schema.parseFrom(schemaBytes);
} catch (InvalidProtocolBufferException e) {
  throw new IllegalStateException("config schema bytes are not a valid SchemaApi.Schema proto", e);
}

Try / catch

try {
  result = pipeline.apply(SchemaIO.read(...));
} catch (RuntimeException e) {
  if (e.getMessage() != null && e.getMessage().contains("Unable to infer data schema from configuration proto")) {
    // check Beam version parity between client and expansion service; re-serialize schema
  }
}

Prevention

Use the same Beam version for the client and the schemaio expansion service.
Always serialize the schema with SchemaTranslation.schemaToProto, never raw row bytes.
Regenerate expansion service artifacts after Beam upgrades.
Validate schema protos round-trip on the client before expansion.

When it happens

Trigger: Calling the SchemaIO transform via the expansion service with a configuration payload whose schema bytes fail SchemaApi.Schema.parseFrom or SchemaTranslation.schemaFromProto (InvalidProtocolBufferException).

Common situations: Cross-version Beam pipelines where schema proto formats differ; corrupted or hand-crafted expansion payloads; config bytes that are not a Schema proto at all (e.g. row bytes mistakenly passed as schema).

Understand the failure class

Background: "cannot parse invalid wire-format data", "cannot unmarshal", "failed unmarshalling": protobuf unmarshal errors explained — this error's family across 10 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/23758fe5bc44d4b8. Report an issue: GitHub.