{"record":{"id":"bc79c0bfe3807eeb","repo":"apache/beam","slug":"batch-type-and-element-type-must-have-equivalent-dtypes","errorCode":null,"errorMessage":"batch type and element type must have equivalent dtypes (batch={batch_type.dtype}, element={element_type.dtype})","messagePattern":"batch type and element type must have equivalent dtypes \\(batch=(.+?), element=(.+?)\\)","errorType":"validation","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/typehints/batch.py","lineNumber":202,"sourceCode":"  @staticmethod\n  @BatchConverter.register(name=\"numpy\")\n  def from_typehints(element_type,\n                     batch_type) -> Optional['NumpyBatchConverter']:\n    if not isinstance(element_type, NumpyTypeHint.NumpyTypeConstraint):\n      try:\n        element_type = NumpyArray[element_type, ()]\n      except TypeError as e:\n        raise TypeError(\"Element type is not a dtype\") from e\n\n    if not isinstance(batch_type, NumpyTypeHint.NumpyTypeConstraint):\n      if not batch_type == np.ndarray:\n        raise TypeError(\n            \"batch type must be np.ndarray or \"\n            \"beam.typehints.batch.NumpyArray[..]\")\n      batch_type = NumpyArray[element_type.dtype, (N, )]\n\n    if not batch_type.dtype == element_type.dtype:\n      raise TypeError(\n          \"batch type and element type must have equivalent dtypes \"\n          f\"(batch={batch_type.dtype}, element={element_type.dtype})\")\n\n    computed_element_shape = list(batch_type.shape)\n    partition_dimension = computed_element_shape.index(N)\n    computed_element_shape.pop(partition_dimension)\n    if not tuple(computed_element_shape) == element_type.shape:\n      raise TypeError(\n          \"Failed to align batch type's batch dimension with element type. \"\n          f\"(batch type dimensions: {batch_type.shape}, element type \"\n          f\"dimenstions: {element_type.shape}\")\n\n    return NumpyBatchConverter(\n        batch_type,\n        element_type,\n        batch_type.dtype,\n        element_type.shape,\n        partition_dimension)","sourceCodeStart":184,"sourceCodeEnd":220,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/typehints/batch.py#L184-L220","documentation":"NumpyBatchConverter.from_typehints found the batch typehint and the element typehint resolve to different numpy dtypes (shown in the message); element-wise batching requires both to share one dtype so arrays can be stacked/reshaped losslessly.","triggerScenarios":"from_typehints(np.float32, NumpyArray[np.int64, (N,)]) — dtype of the NumpyArray batch hint differs from element dtype.","commonSituations":"Specifying np.float64 elements but batching into float32 arrays (or vice versa); numpy default dtype (float64) colliding with explicitly declared int element types.","solutions":["Make element and batch dtypes identical, e.g. NumpyArray[np.int64, (N,)] for np.int64 elements","Explicitly set the dtype when creating arrays in the pipeline","Compare dtypes with np.dtype(...) equality before calling"],"exampleFix":"// before\nBatchConverter.from_typehints(np.float32, NumpyArray[np.int64, (N,)])\n// after\nBatchConverter.from_typehints(np.float32, NumpyArray[np.float32, (N,)])","handlingStrategy":"validation","validationCode":"import numpy as np\nassert np.dtype(batch_dtype) == np.dtype(element_dtype), 'element and batch dtypes must match'","typeGuard":"import numpy as np\ndef dtypes_match(elem_t, batch_t):\n    return np.dtype(elem_t.dtype) == np.dtype(batch_t.dtype)","tryCatchPattern":null,"preventionTips":["Declare both dtypes explicitly and identically","Never rely on numpy's default dtype inference for batched arrays","Add a dtype assertion before constructing the converter"],"tags":["python","apache-beam","numpy","dtype"],"backgroundTag":"dtype-mismatch","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T21:17:11.552Z"}