apache/beam · error · ValueError

Encountered unknown type {other!r}

Error message

Encountered unknown type {other!r}

What it means

IndexingIsSubpartitioningOf.is_subpartitioning_of compares the current partitioning against another partitioning object; if the other object is not one of the known partitioning types (DefaultIndexing/Order/Arbitrary/JoinIndex/Hashing etc.), it raises ValueError because no containment rule is defined.

Source

Thrown at sdks/python/apache_beam/dataframe/partitionings.py:105

    if self._levels:
      return hash(tuple(sorted(self._levels)))
    else:
      return hash(type(self))

  def is_subpartitioning_of(self, other):
    if isinstance(other, Singleton):
      return True
    elif isinstance(other, Index):
      if self._levels is None:
        return True
      elif other._levels is None:
        return False
      else:
        return all(level in self._levels for level in other._levels)
    elif isinstance(other, (Arbitrary, JoinIndex)):
      return False
    else:
      raise ValueError(f"Encountered unknown type {other!r}")

  def _hash_index(self, df):
    if self._levels is None:
      levels = list(range(df.index.nlevels))
    else:
      levels = self._levels
    return sum(
        pd.util.hash_array(np.asarray(df.index.get_level_values(level)))
        for level in levels)

  def partition_fn(self, df, num_partitions):
    hashes = self._hash_index(df)
    for key in range(num_partitions):
      yield key, df[hashes % num_partitions == key]

  def check(self, dfs):
    # Drop empty DataFrames
    dfs = [df for df in dfs if len(df)]

View on GitHub (pinned to 12126d8942)

Solutions

1. Ensure the custom partitioning subclasses one of apache_beam.dataframe.partitionings.Partitioning's known types (e.g. Arbitrary, Hashing, JoinIndex)
2. Implement is_subpartitioning_of on the custom class so it handles comparisons instead of falling through
3. Inspect repr(other) in the message to identify the unknown object and add handling for it upstream

Example fix

// before
class MyPartitioning:
  ...
// after
from apache_beam.dataframe.partitionings import Hashing
class MyPartitioning(Hashing):
  def is_subpartitioning_of(self, other): ...

from apache_beam.dataframe import partitionings
if not isinstance(other, partitionings.Partitioning):
    raise TypeError(f'Expected a partitionings.Partitioning, got {type(other)!r}')

def is_known_partitioning(p) -> bool:
    from apache_beam.dataframe import partitionings
    return isinstance(p, partitionings.Partitioning)

When it happens

Trigger: A custom Partitioning implementation is passed into Beam DataFrame internals (e.g. via partitionings or stage hints) that is not a recognized subclass; internal API misuse when extending the DataFrame API.

Common situations: Developers writing custom partitionings for the Beam DataFrame API and forgetting to subclass one of the built-in types or register it with the partitioning-aware stages.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

• Proxy '{proxy}' has unsupported type '{type(proxy)}'
• Proxy '{proxy}' has unsupported type '{type(proxy)}'
• Unable to convert objects of type %s to a PCollection
• Scalar expression %s of type %s partitoned by non-singleton
• Cannot specify both 'labels' and 'index'/'columns'

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/44cf5139326713e3. Report an issue: GitHub.