{"record":{"id":"23758fe5bc44d4b8","repo":"apache/beam","slug":"unable-to-infer-data-schema-from-configuration-proto","errorCode":null,"errorMessage":"Unable to infer data schema from configuration proto.","messagePattern":"Unable to infer data schema from configuration proto\\.","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"sdks/java/extensions/schemaio-expansion-service/src/main/java/org/apache/beam/sdk/extensions/schemaio/expansion/ExternalSchemaIOTransformRegistrar.java","lineNumber":110,"sourceCode":"      this.config = config;\n    }\n\n    public void setDataSchema(@Nullable byte[] dataSchema) {\n      this.dataSchema = dataSchema;\n    }\n  }\n\n  @Nullable\n  private static Schema translateSchema(@Nullable byte[] schemaBytes) {\n    if (schemaBytes == null) {\n      return null;\n    }\n\n    try {\n      SchemaApi.Schema protoSchema = SchemaApi.Schema.parseFrom(schemaBytes);\n      return SchemaTranslation.schemaFromProto(protoSchema);\n    } catch (InvalidProtocolBufferException e) {\n      throw new RuntimeException(\"Unable to infer data schema from configuration proto.\", e);\n    }\n  }\n\n  private static Row translateRow(byte[] rowBytes, Schema configSchema) {\n    RowCoder rowCoder = RowCoder.of(configSchema);\n    InputStream stream = new ByteArrayInputStream(rowBytes);\n\n    try {\n      return rowCoder.decode(stream);\n    } catch (IOException e) {\n      throw new RuntimeException(\n          \"Unable to infer configuration row from configuration proto and schema.\", e);\n    }\n  }\n\n  @VisibleForTesting\n  static class ReaderBuilder\n      implements ExternalTransformBuilder<Configuration, PBegin, PCollection<Row>> {","sourceCodeStart":92,"sourceCodeEnd":128,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/extensions/schemaio-expansion-service/src/main/java/org/apache/beam/sdk/extensions/schemaio/expansion/ExternalSchemaIOTransformRegistrar.java#L92-L128","documentation":"This RuntimeException is thrown by ExternalSchemaIOTransformRegistrar.translateSchema when the serialized configuration schema bytes cannot be parsed as a SchemaApi.Schema protobuf or translated into a Beam Schema. It indicates the expansion service received configuration whose schema payload is corrupt, truncated, or not in the expected protobuf format.","triggerScenarios":"Calling the SchemaIO transform via the expansion service with a configuration payload whose schema bytes fail SchemaApi.Schema.parseFrom or SchemaTranslation.schemaFromProto (InvalidProtocolBufferException).","commonSituations":"Cross-version Beam pipelines where schema proto formats differ; corrupted or hand-crafted expansion payloads; config bytes that are not a Schema proto at all (e.g. row bytes mistakenly passed as schema).","solutions":["Ensure the client and expansion service use the same Beam version so schema proto serialization is compatible.","Verify the configuration payload actually contains SchemaApi.Schema bytes for the schema field, not row bytes.","Regenerate/rebuild the expansion service jar and reconnect.","Log/inspect the failing bytes to confirm format; re-serialize the schema using SchemaTranslation.schemaToProto before sending."],"exampleFix":"// before\nbyte[] payload = rowBytes; // wrong bytes\nconfigProto = configProto.toBuilder().setSchemaBytes(payload).build();\n// after\nSchemaApi.Schema protoSchema = SchemaTranslation.schemaToProto(schema, true);\nconfigProto = configProto.toBuilder().setSchemaBytes(protoSchema.toByteString()).build();","handlingStrategy":"try-catch","validationCode":"// Validate config schema bytes before sending to the expansion service\ntry {\n  SchemaApi.Schema.parseFrom(schemaBytes);\n} catch (InvalidProtocolBufferException e) {\n  throw new IllegalStateException(\"config schema bytes are not a valid SchemaApi.Schema proto\", e);\n}","typeGuard":null,"tryCatchPattern":"try {\n  result = pipeline.apply(SchemaIO.read(...));\n} catch (RuntimeException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"Unable to infer data schema from configuration proto\")) {\n    // check Beam version parity between client and expansion service; re-serialize schema\n  }\n}","preventionTips":["Use the same Beam version for the client and the schemaio expansion service.","Always serialize the schema with SchemaTranslation.schemaToProto, never raw row bytes.","Regenerate expansion service artifacts after Beam upgrades.","Validate schema protos round-trip on the client before expansion."],"tags":["java","schemaio","protobuf","expansion-service","deserialization"],"backgroundTag":"protobuf-unmarshal-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}