apache/beam · error · TypeError

Unable to deterministically order keys of dict for

Error message

Unable to deterministically order keys of dict for '%s'

What it means

When a dict coder requires deterministic encoding (requires_deterministic_step_label set), encode_to_stream sorts the dict items before writing. If the keys cannot be compared/sorted (mixed or unorderable key types), it raises TypeError naming the step label, chained from the original exception.

Solutions

Make all dict keys the same comparable type (e.g. all strings) before the encoding boundary.
Replace dict keys with a canonical string representation (str(key) or json.dumps) if types must vary.
Convert the dict into a sorted list of key-value pairs of uniform type upstream of the coder.
Inspect the step label in the message to locate the producing transform and fix its output types.

Example fix

// before
def to_dict(row):
    return {row['id']: row['value']}  # id may be int or str
// after
def to_dict(row):
    return {str(row['id']): row['value']}  # uniform, sortable keys

Defensive patterns

Strategy: type-guard

Validate before calling

def dict_keys_sortable(d):
    try:
        sorted(d.keys()); return True
    except TypeError:
        return False

Type guard

def has_uniform_keys(d):
    return bool(d) and len({type(k) for k in d}) == 1 and None not in d

Try / catch

try:
    result = p.run()
except TypeError as e:
    if 'Unable to deterministically order keys of dict' in str(e):
        normalize_keys_upstream()  # cast all keys to str

Prevention

Enforce uniform, non-None key types in transforms producing dicts.
Sort/normalize dict keys (str()) at the boundary of any determinism-required stage.
Add unit tests encoding sample elements with DirectRunner determinism checks.

When it happens

Trigger: Encoding a PCollection containing dicts with heterogeneous or unorderable keys (e.g. {1: 'a', 'b': 2} or keys containing None) under a deterministic-coding requirement (side inputs, GBK keys, deterministic output requirement).

Common situations: Dictionaries with mixed int/str keys built from JSON-ish data; keys that are None; sets/dicts flowing into stages where Beam enforces determinism for correctness (stateful DoFns, cross-language, streaming).

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/10c57a6fa52d3bc1. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/coders/coder_impl.py:455

      stream.write_byte(LIST_TYPE if t is list else TUPLE_TYPE)
      stream.write_var_int64(len(value))
      for e in value:
        self.encode_to_stream(e, stream, True)
    elif t is bool:
      stream.write_byte(BOOL_TYPE)
      stream.write_byte(value)
    elif t in _ITERABLE_LIKE_TYPES:
      stream.write_byte(ITERABLE_LIKE_TYPE)
      self.iterable_coder_impl.encode_to_stream(value, stream, nested)
    elif t is dict:
      dict_value = value  # for typing
      stream.write_byte(DICT_TYPE)
      stream.write_var_int64(len(dict_value))
      if self.requires_deterministic_step_label is not None:
        try:
          ordered_kvs = sorted(dict_value.items())
        except Exception as exn:
          raise TypeError(
              "Unable to deterministically order keys of dict for '%s'" %
              self.requires_deterministic_step_label) from exn
        for k, v in ordered_kvs:
          self.encode_to_stream(k, stream, True)
          self.encode_to_stream(v, stream, True)
      else:
        # Loop over dict.items() is optimized by Cython.
        for k, v in dict_value.items():
          self.encode_to_stream(k, stream, True)
          self.encode_to_stream(v, stream, True)
    elif t is set:
      stream.write_byte(SET_TYPE)
      stream.write_var_int64(len(value))
      if self.requires_deterministic_step_label is not None:
        try:
          value = sorted(value)
        except Exception as exn:
          raise TypeError(

View on GitHub (pinned to 12126d8942)