{"record":{"id":"f772921a99300fab","repo":"apache/beam","slug":"schema-with-id-schema-id-has-encoding-positions-set-true-but","errorCode":null,"errorMessage":"Schema with id {schema.id} has encoding_positions_set=True,\n            but found fields without encoding_position set: {names_no_pos}","messagePattern":"Schema with id (.+?) has encoding_positions_set=True,\n            but found fields without encoding_position set: (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/coders/coder_impl.py","lineNumber":1945,"sourceCode":"\n\nclass RowCoderImpl(StreamCoderImpl):\n  \"\"\"For internal use only; no backwards-compatibility guarantees.\"\"\"\n  def __init__(self, schema, components):\n    self.schema = schema\n    self.num_fields = len(self.schema.fields)\n    self.field_names = [f.name for f in self.schema.fields]\n    self.field_nullable = [field.type.nullable for field in self.schema.fields]\n    self.constructor = named_tuple_from_schema(schema)\n    self.encoding_positions = list(range(len(self.schema.fields)))\n    if self.schema.encoding_positions_set:\n      # should never be duplicate encoding positions.\n      enc_posx = list(\n          set(field.encoding_position for field in self.schema.fields))\n      if len(enc_posx) != len(self.schema.fields):\n        names_no_pos = \", \".join(\n            [f.name for f in self.schema.fields if f.encoding_position is None])\n        raise ValueError(\n            f'''Schema with id {schema.id} has encoding_positions_set=True,\n            but found fields without encoding_position set: {names_no_pos}''')\n      self.encoding_positions = list(\n          field.encoding_position for field in self.schema.fields)\n    self.encoding_positions_argsort = list(np.argsort(self.encoding_positions))\n    self.encoding_positions_are_trivial = self.encoding_positions == list(\n        range(len(self.encoding_positions)))\n    self.components = list(\n        components[self.encoding_positions.index(i)].get_impl()\n        for i in self.encoding_positions)\n    self.has_nullable_fields = any(\n        field.type.nullable for field in self.schema.fields)\n\n  def encode_to_stream(self, value, out, nested):\n    out.write_var_int64(self.num_fields)\n    attrs = [getattr(value, name) for name in self.field_names]\n\n    if self.has_nullable_fields:","sourceCodeStart":1927,"sourceCodeEnd":1963,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/coders/coder_impl.py#L1927-L1963","documentation":"When a Schema has encoding_positions_set=True, SchemaCoder requires every field to carry an encoding_position so fields can be encoded in a deterministic order. If positions were flagged as set but at least one field has encoding_position=None, Beam raises ValueError naming the offending fields, since the position list would be invalid. The check also guards against duplicate positions.","triggerScenarios":"Constructing SchemaCoder (or decoding rows) from a Schema whose fields were marked encoding_positions_set=True while some fields lack encoding_position — e.g., a schema built programmatically where set_encoding_positions was half-applied, or a schema deserialized from a proto that claims positions are set but has None on new/renamed fields.","commonSituations":"Programmatic schema construction where fields were added after calling the position-setting API; schema evolution (adding fields) without reassigning encoding positions; schemas round-tripped through proto serialization with inconsistent flags; Beam version differences in how positions are defaulted.","solutions":["Assign an encoding_position to every field (call the schema API that sets positions consistently, e.g., 0..n-1)","Set encoding_positions_set=False to let Beam fall back to declaration order if explicit positions aren't needed","Rebuild the schema from the current field list so positions and the set-flag stay consistent","Fix schema-evolution code to reassign positions whenever fields are added/removed"],"exampleFix":"// before\nschema = Schema(fields=[Field('a', int64), Field('b', str)])\nschema.encoding_positions_set = True  # but no field has encoding_position\nSchemaCoder(schema, ...)\n// after\nfor i, f in enumerate(schema.fields):\n    f.encoding_position = i\nschema.encoding_positions_set = True\nSchemaCoder(schema, ...)","handlingStrategy":"validation","validationCode":"def schema_positions_ok(schema) -> bool:\n    if not schema.encoding_positions_set:\n        return True\n    return all(f.encoding_position is not None for f in schema.fields) and \\\n        len({f.encoding_position for f in schema.fields}) == len(schema.fields)","typeGuard":"def all_fields_have_positions(schema) -> bool:\n    return all(getattr(f, 'encoding_position', None) is not None for f in schema.fields)","tryCatchPattern":"try:\n    coder = SchemaCoder(schema, tuple(coders))\nexcept ValueError as e:\n    if 'encoding_position' in str(e):\n        for i, f in enumerate(schema.fields):\n            f.encoding_position = i\n        coder = SchemaCoder(schema, tuple(coders))\n    else:\n        raise","preventionTips":["Assign encoding_position to every field whenever encoding_positions_set is true","Reassign positions after any schema evolution (add/remove fields)","Build schemas via Beam's schema APIs so position flags stay consistent; verify with a pre-flight check"],"tags":["python","apache-beam","coding","schema","encoding-position"],"backgroundTag":"schema-validation-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}