apache/beam · error · TypeError

batch type and element type must have equivalent dtypes (bat

Error message

batch type and element type must have equivalent dtypes (batch={batch_type.dtype}, element={element_type.dtype})

What it means

NumpyBatchConverter.from_typehints found the batch typehint and the element typehint resolve to different numpy dtypes (shown in the message); element-wise batching requires both to share one dtype so arrays can be stacked/reshaped losslessly.

Source

Thrown at sdks/python/apache_beam/typehints/batch.py:202

  @staticmethod
  @BatchConverter.register(name="numpy")
  def from_typehints(element_type,
                     batch_type) -> Optional['NumpyBatchConverter']:
    if not isinstance(element_type, NumpyTypeHint.NumpyTypeConstraint):
      try:
        element_type = NumpyArray[element_type, ()]
      except TypeError as e:
        raise TypeError("Element type is not a dtype") from e

    if not isinstance(batch_type, NumpyTypeHint.NumpyTypeConstraint):
      if not batch_type == np.ndarray:
        raise TypeError(
            "batch type must be np.ndarray or "
            "beam.typehints.batch.NumpyArray[..]")
      batch_type = NumpyArray[element_type.dtype, (N, )]

    if not batch_type.dtype == element_type.dtype:
      raise TypeError(
          "batch type and element type must have equivalent dtypes "
          f"(batch={batch_type.dtype}, element={element_type.dtype})")

    computed_element_shape = list(batch_type.shape)
    partition_dimension = computed_element_shape.index(N)
    computed_element_shape.pop(partition_dimension)
    if not tuple(computed_element_shape) == element_type.shape:
      raise TypeError(
          "Failed to align batch type's batch dimension with element type. "
          f"(batch type dimensions: {batch_type.shape}, element type "
          f"dimenstions: {element_type.shape}")

    return NumpyBatchConverter(
        batch_type,
        element_type,
        batch_type.dtype,
        element_type.shape,
        partition_dimension)

View on GitHub (pinned to 12126d8942)

Solutions

1. Make element and batch dtypes identical, e.g. NumpyArray[np.int64, (N,)] for np.int64 elements
2. Explicitly set the dtype when creating arrays in the pipeline
3. Compare dtypes with np.dtype(...) equality before calling

Example fix

// before
BatchConverter.from_typehints(np.float32, NumpyArray[np.int64, (N,)])
// after
BatchConverter.from_typehints(np.float32, NumpyArray[np.float32, (N,)])

import numpy as np
assert np.dtype(batch_dtype) == np.dtype(element_dtype), 'element and batch dtypes must match'

import numpy as np
def dtypes_match(elem_t, batch_t):
    return np.dtype(elem_t.dtype) == np.dtype(batch_t.dtype)

When it happens

Trigger: from_typehints(np.float32, NumpyArray[np.int64, (N,)]) — dtype of the NumpyArray batch hint differs from element dtype.

Common situations: Specifying np.float64 elements but batching into float32 arrays (or vice versa); numpy default dtype (float64) colliding with explicitly declared int element types.

Related errors

• Element type is not a dtype
• Batch {batch!r} does not have expected dtype: {self.dtype!r}
• batch type must be np.ndarray or beam.typehints.batch.NumpyA
• Failed to align batch type's batch dimension with element ty
• Batch {batch!r} is not an instance of ndarray

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/bc79c0bfe3807eeb. Report an issue: GitHub.