apache/beam · error · TypeError

Batch {batch!r} is not an instance of ndarray

Error message

Batch {batch!r} is not an instance of ndarray

What it means

NumpyTypeConstraint.type_check is the runtime type guard for NumpyArray[...] hints during pipeline execution: the batched value flowing through is not a numpy.ndarray, so the declared constraint is violated.

Source

Thrown at sdks/python/apache_beam/typehints/batch.py:251

    return np.size(batch, axis=self.partition_dimension)

  def estimate_byte_size(self, batch):
    return batch.nbytes


# numpy is starting to add typehints, which we should support
# https://numpy.org/doc/stable/reference/typing.html for now they don't allow
# specifying shape, seems to be coming after
# https://www.python.org/dev/peps/pep-0646/
class NumpyTypeHint():
  class NumpyTypeConstraint(typehints.TypeConstraint):
    def __init__(self, dtype, shape=()):
      self.dtype = np.dtype(dtype)
      self.shape = shape

    def type_check(self, batch):
      if not isinstance(batch, np.ndarray):
        raise TypeError(f"Batch {batch!r} is not an instance of ndarray")
      if not np.issubdtype(batch.dtype, self.dtype):
        raise TypeError(
            f"Batch {batch!r} does not have expected dtype: {self.dtype!r}")

      for dim in range(len(self.shape)):
        if not self.shape[dim] == N and not batch.shape[dim] == self.shape[dim]:
          raise TypeError(
              f"Batch {batch!r} does not have expected shape: {self.shape!r}")

    def _consistent_with_check_(self, sub):
      # TODO Check sub against batch type, and element type
      return True

    def __key(self):
      return (self.dtype, self.shape)

    def __eq__(self, other) -> bool:
      if isinstance(other, NumpyTypeHint.NumpyTypeConstraint):

View on GitHub (pinned to 12126d8942)

Solutions

1. Ensure batched outputs are np.ndarray instances (np.asarray(list) as needed)
2. Change the declared batch type to List[T] if you actually produce lists
3. Use a BatchElements transform with the matching converter to produce batches

Example fix

// before
return list(elements)  # with numpy batch type declared
// after
return np.asarray(list(elements), dtype=np.int64)

import numpy as np
assert isinstance(batch, np.ndarray), 'batch must be np.ndarray'

import numpy as np
def is_ndarray(x):
    return isinstance(x, np.ndarray)

try:
    check_batch(batch)
except TypeError as e:
    batch = np.asarray(batch)  # coerce array-likes

When it happens

Trigger: Emitting batches that are plain Python lists from a DoFn while the declared batch type is NumpyArray/np.ndarray, then beam runtime type-checking rejects them.

Common situations: Switching a pipeline from list batching to numpy batching without changing produce_batch/output code; returning array-likes that aren't np.ndarray.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

• Transform "{full_label}" was applied to the output of an obj
• Transform '{full_label}' expects a PCollection as input. Got
• Input to _GroupByKeyOnly must be a PCollection of windowed k
• Input to GroupByKey must be a PCollection with elements comp
• Element type is not a dtype

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/29b6345bcb7b0f73. Report an issue: GitHub.