apache/beam · error · TypeError

batch type must be np.ndarray or beam.typehints.batch.NumpyA

Error message

batch type must be np.ndarray or beam.typehints.batch.NumpyArray[..]

What it means

NumpyBatchConverter.from_typehints requires the batch type to be np.ndarray or a NumpyArray[...] typehint (optionally with a shape/partition dimension); the supplied batch_type is neither, so no numpy-based batching applies. This converter returning None/raising lets BatchConverter pick the right implementation.

Source

Thrown at sdks/python/apache_beam/typehints/batch.py:196

      partition_dimension=0):
    super().__init__(batch_type, element_type)
    self.dtype = np.dtype(dtype)
    self.element_shape = element_shape
    self.partition_dimension = partition_dimension

  @staticmethod
  @BatchConverter.register(name="numpy")
  def from_typehints(element_type,
                     batch_type) -> Optional['NumpyBatchConverter']:
    if not isinstance(element_type, NumpyTypeHint.NumpyTypeConstraint):
      try:
        element_type = NumpyArray[element_type, ()]
      except TypeError as e:
        raise TypeError("Element type is not a dtype") from e

    if not isinstance(batch_type, NumpyTypeHint.NumpyTypeConstraint):
      if not batch_type == np.ndarray:
        raise TypeError(
            "batch type must be np.ndarray or "
            "beam.typehints.batch.NumpyArray[..]")
      batch_type = NumpyArray[element_type.dtype, (N, )]

    if not batch_type.dtype == element_type.dtype:
      raise TypeError(
          "batch type and element type must have equivalent dtypes "
          f"(batch={batch_type.dtype}, element={element_type.dtype})")

    computed_element_shape = list(batch_type.shape)
    partition_dimension = computed_element_shape.index(N)
    computed_element_shape.pop(partition_dimension)
    if not tuple(computed_element_shape) == element_type.shape:
      raise TypeError(
          "Failed to align batch type's batch dimension with element type. "
          f"(batch type dimensions: {batch_type.shape}, element type "
          f"dimenstions: {element_type.shape}")

View on GitHub (pinned to 12126d8942)

Solutions

1. Pass batch_type = np.ndarray or NumpyArray[dtype, shape]
2. Wrap the batch type as beam.typehints.batch.NumpyArray[dtype, (N,)]
3. Use the list converter if batches are Python lists

Example fix

// before
BatchConverter.from_typehints(np.int64, List[int])
// after
BatchConverter.from_typehints(np.int64, np.ndarray)

import numpy as np
from apache_beam.typehints.batch import NumpyArray, NumpyTypeHint
assert isinstance(batch_type, NumpyTypeHint.NumpyTypeConstraint) or batch_type == np.ndarray

import numpy as np
def is_numpy_batch_type(bt):
    return bt == np.ndarray or isinstance(getattr(bt, 'dtype', None), np.dtype)

When it happens

Trigger: from_typehints(np.int64, List[int]) or from_typehints(np.int64, np.matrix) — batch type is neither a NumpyArray[..] hint nor np.ndarray.

Common situations: Mixing list and numpy batching hints; passing array subclasses instead of np.ndarray.

Understand the failure class

Background: "is not a compatible type" / "cannot merge" errors: when a value's type doesn't match what the library requires — this error's family across 65 libraries.

Related errors

• Unable to find BatchConverter for element_type={element_type
• batch type must be List[T] for element type T
• Element type is not a dtype
• batch type and element type must have equivalent dtypes (bat
• Failed to align batch type's batch dimension with element ty

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/1c3c805e75ea7244. Report an issue: GitHub.