apache/beam · error · TypeError

Unable to deterministically encode

Error message

Unable to deterministically encode '%s' of type '%s', for the input of '%s'. The object defines __getstate__ but not __setstate__.

What it means

Beam can deterministically encode objects that define __getstate__ (with default object.__reduce__) by encoding their state. If the object defines __getstate__ but not __setstate__, it cannot be reconstructed on the other side of the wire, so the coder raises TypeError naming the input step. This guards against silently producing data the pipeline cannot decode.

Solutions

1. Add a matching __setstate__ method that restores state from the __getstate__ payload.
2. Remove the custom __getstate__ if unnecessary so a deterministic fallback applies.
3. Convert the object into a NamedTuple or frozen dataclass before sending it through the pipeline.
4. Give the PTransform a concrete type hint with a deterministically encodable type and map the object to that type.

Example fix

// before
class Thing:
    def __getstate__(self):
        return self.__dict__

// after
class Thing:
    def __getstate__(self):
        return self.__dict__
    def __setstate__(self, state):
        self.__dict__.update(state)

def supports_deterministic_state(obj):
    return (hasattr(obj, '__getstate__') and hasattr(obj, '__setstate__')
            and type(obj).__reduce__ == object.__reduce__)

def is_statefully_encodable(obj) -> bool:
    t = type(obj)
    return hasattr(obj, '__getstate__') and hasattr(obj, '__setstate__') and t.__reduce__ == object.__reduce__

try:
    coder.encode(obj)
except TypeError:
    obj = to_namedtuple(obj)
    coder.encode(obj)

When it happens

Trigger: Encoding a custom class instance that defines __getstate__ (inheriting pickle behavior from an ancestor that does) but not __setstate__, as input to a step requiring deterministic encoding; the hasattr(value, '__setstate__') check fails in encode_special_deterministic.

Common situations: Subclassing a class that defines __getstate__ (e.g. some library base classes) and using instances as pipeline data; adding __getstate__ for pickling without __setstate__; Python 3.11+ default object __getstate__ interactions (see Beam issue #33020).

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• Unable to deterministically encode non-frozen
• Unable to deterministically encode
• Attempted to encode null for non-nullable field
• cannot encode a null BitSet
• cannot encode a null byte[]

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/cb7b5efb4baf8592. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/coders/coder_impl.py:538

      stream.write_byte(NAMED_TUPLE_TYPE)
      self.encode_type(type(value), stream)
      try:
        self.iterable_coder_impl.encode_to_stream(value, stream, True)
      except Exception as e:
        raise TypeError(self._deterministic_encoding_error_msg(value)) from e
    elif isinstance(value, enum.Enum):
      stream.write_byte(ENUM_TYPE)
      self.encode_type(type(value), stream)
      # Enum values can be of any type.
      try:
        self.encode_to_stream(value.value, stream, True)
      except Exception as e:
        raise TypeError(self._deterministic_encoding_error_msg(value)) from e
    elif (hasattr(value, "__getstate__") and
          # https://github.com/apache/beam/issues/33020
          type(value).__reduce__ == object.__reduce__):
      if not hasattr(value, "__setstate__"):
        raise TypeError(
            "Unable to deterministically encode '%s' of type '%s', "
            "for the input of '%s'. The object defines __getstate__ but not "
            "__setstate__." %
            (value, type(value), self.requires_deterministic_step_label))
      stream.write_byte(NESTED_STATE_TYPE)
      self.encode_type(type(value), stream)
      state_value = value.__getstate__()
      try:
        self.encode_to_stream(state_value, stream, True)
      except Exception as e:
        raise TypeError(self._deterministic_encoding_error_msg(value)) from e
    else:
      raise TypeError(self._deterministic_encoding_error_msg(value))

  def _deterministic_encoding_error_msg(self, value):
    return (
        "Unable to deterministically encode '%s' of type '%s', "
        "please provide a type hint for the input of '%s'" %

View on GitHub (pinned to 12126d8942)