{"record":{"id":"cbc20871b584b758","repo":"apache/beam","slug":"batch-type-and-element-type-must-have-equivalent-dtypes-cbc208","errorCode":null,"errorMessage":"batch type and element type must have equivalent dtypes (batch={batch_type.dtype}, element={element_type.dtype})","messagePattern":"batch type and element type must have equivalent dtypes \\(batch=(.+?), element=(.+?)\\)","errorType":"validation","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/typehints/pytorch_type_compatibility.py","lineNumber":55,"sourceCode":"    self.element_shape = element_shape\n    self.partition_dimension = partition_dimension\n\n  @staticmethod\n  @BatchConverter.register(name=\"pytorch\")\n  def from_typehints(element_type,\n                     batch_type) -> Optional['PytorchBatchConverter']:\n    if not isinstance(element_type, PytorchTypeHint.PytorchTypeConstraint):\n      element_type = PytorchTensor[element_type, ()]\n\n    if not isinstance(batch_type, PytorchTypeHint.PytorchTypeConstraint):\n      if not batch_type == torch.Tensor:\n        raise TypeError(\n            \"batch type must be torch.Tensor or \"\n            \"beam.typehints.pytorch_type_compatibility.PytorchTensor[..]\")\n      batch_type = PytorchTensor[element_type.dtype, (N, )]\n\n    if not batch_type.dtype == element_type.dtype:\n      raise TypeError(\n          \"batch type and element type must have equivalent dtypes \"\n          f\"(batch={batch_type.dtype}, element={element_type.dtype})\")\n    computed_element_shape = list(batch_type.shape)\n    partition_dimension = computed_element_shape.index(N)\n    computed_element_shape.pop(partition_dimension)\n    if not tuple(computed_element_shape) == element_type.shape:\n      raise TypeError(\n          \"Could not align batch type's batch dimension with element type. \"\n          f\"(batch type dimensions: {batch_type.shape}, element type \"\n          f\"dimenstions: {element_type.shape}\")\n\n    return PytorchBatchConverter(\n        batch_type,\n        element_type,\n        batch_type.dtype,\n        element_type.shape,\n        partition_dimension)\n","sourceCodeStart":37,"sourceCodeEnd":73,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/typehints/pytorch_type_compatibility.py#L37-L73","documentation":"Apache Beam's PytorchBatchConverter.from_typehints builds a batch converter from an element type hint and a batch type hint. When batch_type was given explicitly as a PytorchTensor[...] constraint, Beam verifies that its declared dtype matches the element type's dtype; a mismatch means batches and the elements they contain would claim incompatible dtypes, so Beam refuses to construct the converter.","triggerScenarios":"Calling from_typehints (e.g. via @with_batch_type or registering a 'pytorch' BatchConverter) where batch_type=PytorchTensor[torch.float64, (N, 128)] but element_type resolves to a different dtype such as torch.float32 — typically because the PytorchTensor dtype was written explicitly while the element type was inferred from data.","commonSituations":"Copy-pasting a PytorchTensor hint with a dtype copied from a model checkpoint (float64/float32 mismatch), changing tensor dtype in a DoFn (e.g. .double()) without updating the batch hint, or mixing hints defined in different places of the pipeline.","solutions":["Make the PytorchTensor batch type dtype identical to the element type dtype, e.g. PytorchTensor[torch.float32, (N, 128)] with element hint PytorchTensor[torch.float32, (128,)].","Pass batch_type=torch.Tensor so Beam derives the batch dtype from the element type automatically (PytorchTensor[element_type.dtype, (N,)]).","Check for tensor casts in your pipeline (.float(), .double(), .half()) that changed the runtime dtype away from the declared hint.","Print both hints (repr) and compare dtype fields before registering the converter."],"exampleFix":"// before\nwith element_type(PytorchTensor[torch.float32, (128,)]) and \\\n     batch_type(PytorchTensor[torch.float64, (N, 128)]):\n    ...\n// after\nwith element_type(PytorchTensor[torch.float32, (128,)]) and \\\n     batch_type(PytorchTensor[torch.float32, (N, 128)]):  # dtype matches element\n    ...","handlingStrategy":"validation","validationCode":"def check_batch_dtype(batch_type, element_type):\n    bt = batch_type if hasattr(batch_type, 'dtype') else PytorchTensor[element_type.dtype, (N,)]\n    if bt.dtype != element_type.dtype:\n        raise TypeError(f'dtype mismatch: batch={bt.dtype}, element={element_type.dtype}')","typeGuard":"def is_dtype_consistent(batch_type, element_type) -> bool:\n    return getattr(batch_type, 'dtype', None) == getattr(element_type, 'dtype', None)","tryCatchPattern":null,"preventionTips":["Pass plain torch.Tensor as batch_type and let Beam derive the dtype from the element type.","Never hand-write the dtype in a PytorchTensor batch hint; copy it from the element hint constant.","Watch for accidental .double()/.half()/.to(dtype) calls in DoFns.","Add a unit test asserting the registered converter's dtype before running the pipeline."],"tags":["apache-beam","pytorch","typehints","dtype"],"backgroundTag":"dtype-mismatch","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}