{"record":{"id":"16016d7e4c0a7f39","repo":"apache/beam","slug":"cannot-infer-schema-with-a-circular-reference-proto-field","errorCode":null,"errorMessage":"Cannot infer schema with a circular reference. Proto Field: ${name}","messagePattern":"Cannot infer schema with a circular reference\\. Proto Field: (.+?)","errorType":"exception","errorClass":"java.lang.IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"sdks/java/extensions/protobuf/src/main/java/org/apache/beam/sdk/extensions/protobuf/ProtoSchemaTranslator.java","lineNumber":186,"sourceCode":"  /** Return the proto field number for a type. */\n  static int getFieldNumber(Field field) {\n    return field.getOptions().getValue(SCHEMA_OPTION_META_NUMBER);\n  }\n\n  /** Return a Beam schema representing a proto class. */\n  static Schema getSchema(Class<? extends Message> clazz) {\n    return getSchema(ProtobufUtil.getDescriptorForClass(clazz));\n  }\n\n  static synchronized Schema getSchema(Descriptors.Descriptor descriptor) {\n    if (alreadyVisitedSchemas.containsKey(descriptor)) {\n      @Nullable Schema existingSchema = alreadyVisitedSchemas.get(descriptor);\n      if (existingSchema == null) {\n        String name = descriptor.getFullName();\n        if (\"google.protobuf.Struct\".equals(name)) {\n          throw new UnsupportedOperationException(\"Infer schema of Struct type is not supported.\");\n        }\n        throw new IllegalArgumentException(\n            \"Cannot infer schema with a circular reference. Proto Field: \" + name);\n      }\n      return existingSchema;\n    }\n    alreadyVisitedSchemas.put(descriptor, null);\n    /* OneOfComponentFields refers to the field number in the protobuf where the component subfields\n     * are. This is needed to prevent double inclusion of the component fields.*/\n    Set<Integer> oneOfComponentFields = Sets.newHashSet();\n    /* OneOfFieldLocation stores the field number of the first field in the OneOf. Using this, we can use the location\n    of the first field in the OneOf as the location of the entire OneOf.*/\n    Map<Integer, Field> oneOfFieldLocation = Maps.newHashMap();\n    List<Field> fields = Lists.newArrayListWithCapacity(descriptor.getFields().size());\n\n    // In proto3, an optional field is internally implemented by wrapping it in a synthetic oneof.\n    // The Descriptor.getRealOneOfs() method is then used to retrieve only the \"real\" oneofs that\n    // you explicitly defined, filtering out these automatically generated ones.\n    // https://github.com/protocolbuffers/protobuf/blob/main/docs/implementing_proto3_presence.md#updating-a-\n    for (OneofDescriptor oneofDescriptor : descriptor.getRealOneofs()) {","sourceCodeStart":168,"sourceCodeEnd":204,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/extensions/protobuf/src/main/java/org/apache/beam/sdk/extensions/protobuf/ProtoSchemaTranslator.java#L168-L204","documentation":"ProtoSchemaTranslator.getSchema() infers a Beam Schema from a protobuf Descriptor. It tracks descriptors currently being visited; if it re-enters a descriptor already on the stack (and it isn't the specially allowed Struct case, which has its own error), the proto message type is recursive and no finite schema can be inferred. The library throws IllegalArgumentException naming the offending proto field's full name.","triggerScenarios":"Calling getSchema (directly or via protoToRow-based sources/transforms) on a proto message whose type graph contains a cycle, e.g. a message with a repeated field of its own type like `message Node { repeated Node children = 1; }`.","commonSituations":"Tree/graph-shaped protobuf definitions (linked lists, org charts, AST nodes); messages that mutually reference each other through two or more message types; reusing existing company protos designed for RPC, not schema inference.","solutions":["Break the recursion: wrap the self-referential field in google.protobuf.Struct or WellKnownTypes wrapper types, or convert that field to bytes/string and encode manually.","Flatten the recursive message into an iterative representation (e.g. a flat repeated list of nodes with parent ids) and infer the schema from that.","Provide the schema explicitly instead of inferring it (implement/construct the Beam Schema and coder by hand rather than using ProtoSchemaTranslator.getSchema).","If only Struct recursion is the issue, note google.protobuf.Struct is explicitly rejected too — redesign the payload to avoid Struct."],"exampleFix":"// before\nmessage Node {\n  string name = 1;\n  repeated Node children = 2; // circular reference -> IllegalArgumentException\n}\n// after\nmessage Node {\n  string name = 1;\n  repeated string child_ids = 2; // flatten the tree into a sibling table\n}","handlingStrategy":"validation","validationCode":"// Java: reject recursive protos before calling getSchema\nstatic boolean isRecursive(Descriptors.Descriptor d, Set<String> seen) {\n  if (!seen.add(d.getFullName())) return true;\n  for (Descriptors.FieldDescriptor f : d.getFields()) {\n    if (f.getType() == Descriptors.FieldDescriptor.Type.MESSAGE\n        && isRecursive(f.getMessageType(), new HashSet<>(seen))) return true;\n  }\n  return false;\n}\n// if (isRecursive(myMsg.getDescriptor(), new HashSet<>())) throw new IllegalArgumentException(\"recursive proto\");","typeGuard":null,"tryCatchPattern":"try { Schema s = ProtoSchemaTranslator.getSchema(descriptor); } catch (IllegalArgumentException e) { if (e.getMessage().startsWith(\"Cannot infer schema\")) { /* redesign proto or supply schema manually */ } else throw e; }","preventionTips":["Design protos intended for Beam schema inference without cycles; prefer flattened repeated-entry representations for trees.","Add a unit test that infers schemas for every proto used in the pipeline so recursion is caught in CI.","Keep google.protobuf.Struct and self-referential types out of messages fed to ProtoSchemaTranslator."],"tags":["protobuf","schema-inference","java","recursion"],"backgroundTag":"schema-validation-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}