{"record":{"id":"8926514e2e4c6317","repo":"apache/beam","slug":"attempted-to-encode-null-for-non-nullable-field","errorCode":null,"errorMessage":"Attempted to encode null for non-nullable field \"{}\".","messagePattern":"Attempted to encode null for non-nullable field \"(.+?)\"\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/coders/coder_impl.py","lineNumber":1990,"sourceCode":"        running = 0\n        for i, attr in enumerate(attrs):\n          if i and i % 8 == 0:\n            out.write_byte(running)\n            running = 0\n          running |= (attr is None) << (i % 8)\n        out.write_byte(running)\n      else:\n        out.write_byte(0)\n    else:\n      out.write_byte(0)\n\n    for i in range(self.num_fields):\n      if not self.encoding_positions_are_trivial:\n        i = self.encoding_positions_argsort[i]\n      attr = attrs[i]\n      if attr is None:\n        if not self.field_nullable[i]:\n          raise ValueError(\n              \"Attempted to encode null for non-nullable field \\\"{}\\\".\".format(\n                  self.schema.fields[i].name))\n        continue\n      component_coder = self.components[i]  # for typing\n      component_coder.encode_to_stream(attr, out, True)\n\n  def _row_column_encoders(self, columns):\n    return [\n        RowColumnEncoder.create(\n            self.schema.fields[i].type.atomic_type,\n            self.components[i],\n            columns[name]) for i, name in enumerate(self.field_names)\n    ]\n\n  def encode_batch_to_stream(self, columns: Dict[str, np.ndarray], out):\n    attrs = self._row_column_encoders(columns)\n    n = len(next(iter(columns.values())))\n    if self.has_nullable_fields:","sourceCodeStart":1972,"sourceCodeEnd":2008,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/coders/coder_impl.py#L1972-L2008","documentation":"apache_beam raises this ValueError when a Row (schema-encoded) value contains None for a field that was declared non-nullable (nullable=False) in the schema. The row coder encodes fields sequentially and explicitly rejects nulls where the schema does not permit them, because downstream consumers would not be able to decode a valid value for that field.","triggerScenarios":"Encoding a beam Row via a SchemaAwareCoderImpl/_RowEncoderImpl when an attribute is None but the corresponding field in the schema was created with nullable=False; typically when passing dicts/rows to Create() or a schema-bearing PTransform with missing values.","commonSituations":"Building rows from data sources with missing values (CSV blanks, missing JSON keys, left-outer-join results) while the schema (e.g. inferred or explicitly given to beam.RowTypeConstraint / Create) marks fields non-nullable; version changes where a field became NOT NULL.","solutions":["Make the field nullable in the schema (nullable=True) for the affected field name shown in the message","Coalesce None to a sentinel/default before encoding (e.g. `x if x is not None else default`)","Filter out records with the missing field before the coding point"],"exampleFix":"// before\nbeam.Create([{'id': 1, 'score': None}]).with_schema('id': int, 'score': int)\n// after\nbeam.Create([{'id': 1, 'score': None}]) with schema field 'score' nullable=True, or {'id': 1, 'score': 0}","handlingStrategy":"validation","validationCode":"for f in schema.fields:\n    if not f.nullable:\n        assert all(getattr(row, f.name) is not None for row in rows), f\"null in non-nullable field {f.name}\"","typeGuard":"def is_row_safe(row, schema):\n    return all(getattr(row, f.name, None) is not None or f.nullable for f in schema.fields)","tryCatchPattern":null,"preventionTips":["Declare nullable=True for fields that can be missing","Coalesce nulls to defaults at ingestion time","Filter incomplete records before schema coding"],"tags":["python","apache-beam","serialization","schema"],"backgroundTag":"schema-validation-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}