{"record":{"id":"0d56da60453db4eb","repo":"apache/beam","slug":"unable-to-deterministically-encode-s-of-type-s-please","errorCode":null,"errorMessage":"Unable to deterministically encode '%s' of type '%s', please provide a type hint for the input of '%s'","messagePattern":"Unable to deterministically encode '(.+?)' of type '(.+?)', please provide a type hint for the input of '(.+?)'","errorType":"exception","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/coders/coder_impl.py","lineNumber":518,"sourceCode":"            \"Unable to deterministically encode non-frozen '%s' of type '%s' \"\n            \"for the input of '%s'\" %\n            (value, type(value), self.requires_deterministic_step_label))\n      init_fields = [field for field in dataclasses.fields(value) if field.init]\n      try:\n        if any(field.kw_only for field in init_fields):\n          stream.write_byte(DATACLASS_KW_ONLY_TYPE)\n          self.encode_type(type(value), stream)\n          stream.write_var_int64(len(init_fields))\n          for field in init_fields:\n            stream.write(field.name.encode(\"utf-8\"), True)\n            self.encode_to_stream(getattr(value, field.name), stream, True)\n        else:  # Not using kw_only, we can pass parameters by position.\n          stream.write_byte(DATACLASS_TYPE)\n          self.encode_type(type(value), stream)\n          values = [getattr(value, field.name) for field in init_fields]\n          self.iterable_coder_impl.encode_to_stream(values, stream, True)\n      except Exception as e:\n        raise TypeError(self._deterministic_encoding_error_msg(value)) from e\n    elif isinstance(value, tuple) and hasattr(type(value), '_fields'):\n      stream.write_byte(NAMED_TUPLE_TYPE)\n      self.encode_type(type(value), stream)\n      try:\n        self.iterable_coder_impl.encode_to_stream(value, stream, True)\n      except Exception as e:\n        raise TypeError(self._deterministic_encoding_error_msg(value)) from e\n    elif isinstance(value, enum.Enum):\n      stream.write_byte(ENUM_TYPE)\n      self.encode_type(type(value), stream)\n      # Enum values can be of any type.\n      try:\n        self.encode_to_stream(value.value, stream, True)\n      except Exception as e:\n        raise TypeError(self._deterministic_encoding_error_msg(value)) from e\n    elif (hasattr(value, \"__getstate__\") and\n          # https://github.com/apache/beam/issues/33020\n          type(value).__reduce__ == object.__reduce__):","sourceCodeStart":500,"sourceCodeEnd":536,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/coders/coder_impl.py#L500-L536","documentation":"In encode_special_deterministic, when a value is not a proto/frozen-dataclass/namedtuple/enum/stateful object, the coder falls back to raising TypeError via _deterministic_encoding_error_msg. Without a known structured type, Beam cannot guarantee reproducible bytes for objects like arbitrary class instances (it would otherwise pickle them, which is not deterministic). The message asks for a type hint so Beam can pick a deterministic coder.","triggerScenarios":"Encoding an arbitrary class instance (or set/dict or other unstructured value) as the input to a determinism-requiring step (e.g. GroupByKey keys) with no type hint registered; the final else branch in encode_special_deterministic raises via self._deterministic_encoding_error_msg(value).","commonSituations":"Passing custom objects or sets as GroupByKey/CoGroupByKey keys; omitting type hints on DoFn outputs so Beam falls back to Any/pickle; dict/set used as a key which has no deterministic iteration order.","solutions":["Provide a type hint (e.g. input/output type on the DoFn or PTransform) matching a deterministically encodable type (str, bytes, int, Tuple, NamedTuple, frozen dataclass).","Replace dict/set values with sorted tuples or NamedTuples before using them as keys.","Define __getstate__/__setstate__ (and keep default __reduce__) on the class so the nested-state deterministic path is used.","Wrap the value into a frozen dataclass or NamedTuple key."],"exampleFix":"// before\nclass ExtractKey(beam.DoFn):\n    def process(self, element):\n        yield (element['meta'], element['value'])  # dict key\n\n// after\nclass MetaKey(typing.NamedTuple):\n    a: str\n    b: int\n\nclass ExtractKey(beam.DoFn):\n    def process(self, element) -> Tuple[MetaKey, int]:\n        yield (MetaKey(element['a'], element['b']), element['value'])","handlingStrategy":"validation","validationCode":"def is_deterministically_encodable(value):\n    import dataclasses, enum\n    if isinstance(value, (str, bytes, int, float, bool)):\n        return True\n    if dataclasses.is_dataclass(value) and type(value).__dataclass_params__.frozen:\n        return True\n    if isinstance(value, tuple) and hasattr(type(value), '_fields'):\n        return all(is_deterministically_encodable(v) for v in value)\n    if isinstance(value, enum.Enum):\n        return is_deterministically_encodable(value.value)\n    return False","typeGuard":"def is_encodable_key(value) -> bool:\n    return isinstance(value, (str, bytes, int, float, bool)) or (isinstance(value, tuple) and hasattr(type(value), '_fields'))","tryCatchPattern":"try:\n    coder.encode(key)\nexcept TypeError as e:\n    raise ValueError(f\"Key {key!r} is not deterministically encodable; use a NamedTuple/frozen dataclass\") from e","preventionTips":["Add explicit type hints to every DoFn output used as a key","Never use dict or set as a GroupByKey key","Test encode round-trips in unit tests before running pipelines"],"tags":["apache-beam","python","coder","serialization","type-hints"],"backgroundTag":"missing-required-argument","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}