apache/beam · error · TypeCheckError

Runtime type violation detected within ParDo(%s): %s

Error message

Runtime type violation detected within ParDo(%s): %s

What it means

TypeCheckWrapperDoFn's wrapper catches a TypeCheckError raised while running the wrapped DoFn and re-raises it prefixed with the ParDo label, identifying which transform violated the runtime type check. Beam's runtime type checker validates DoFn inputs/outputs against declared hints when type checking is enabled.

Source

Thrown at sdks/python/apache_beam/typehints/typecheck.py:102

    return self.dofn.teardown()


class OutputCheckWrapperDoFn(AbstractDoFnWrapper):
  """A DoFn that verifies against common errors in the output type."""
  def __init__(self, dofn, full_label):
    super().__init__(dofn)
    self.full_label = full_label

  def wrapper(self, method, args, kwargs):
    try:
      result = method(*args, **kwargs)
    except TypeCheckError as e:
      # TODO(BEAM-10710): Remove the 'ParDo' prefix for the label name
      error_msg = (
          'Runtime type violation detected within ParDo(%s): '
          '%s' % (self.full_label, e))
      _, _, tb = sys.exc_info()
      raise TypeCheckError(error_msg).with_traceback(tb)
    else:
      return self._check_type(result)

  @staticmethod
  def _check_type(output):
    if output is None:
      return output

    elif isinstance(output, (dict, bytes, str)):
      object_type = type(output).__name__
      raise TypeCheckError(
          'Returning a %s from a ParDo or FlatMap is '
          'discouraged. Please use list("%s") if you really '
          'want this behavior.' % (object_type, output))
    elif not isinstance(output, abc.Iterable):
      raise TypeCheckError(
          'FlatMap and ParDo must return an '
          'iterable. %s was returned instead.' % type(output))

View on GitHub (pinned to 12126d8942)

Solutions

1. Read the wrapped TypeCheckError message to find the offending element and expected type, then fix the DoFn to emit the hinted type.
2. Convert elements explicitly (e.g. str(x), int(x)) before emitting.
3. Correct the declared type hints if the actual data shape is the intended one.
4. Add a validation/filter step upstream to drop or coerce bad records.

Example fix

// before
class F(beam.DoFn):
    def process(self, x):
        yield len(x)  # hinted output str
// after
class F(beam.DoFn):
    def process(self, x):
        yield str(len(x))

def check_output(el, expected_type):
    if not isinstance(el, expected_type):
        raise TypeError(f'{el!r} is not {expected_type}')
    return el

def is_str_list(xs) -> bool:
    return all(isinstance(x, str) for x in xs)

try:
    result = pipeline.run()
except TypeCheckError as e:
    log.error('Runtime type violation: %s', e)
    raise

When it happens

Trigger: Running a pipeline with runtime type checking enabled (--runtime_type_check or TypeCheckWrapperDoFn) where a DoFn's output element fails its declared output type hint, or input elements fail input hints during process().

Common situations: Emitting elements of the wrong type from a DoFn (e.g. emitting ints where str was hinted); data arriving from an external source with unexpected types; hints declared on the PTransform not matching actual runtime data.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

• Transform '{full_label}' expects a PCollection as input. Got
• Transform "{full_label}" was applied to the output of "{prod
• Pipeline type checking is enabled, however no output type-hi
• Returning a %s from a ParDo or FlatMap is not allowed. Pleas
• Expected a CombineFn or callable, got %r

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/c072c5d8b2940b99. Report an issue: GitHub.