apache/beam · error · RuntimeError

Transform node %r was not replaced as expected.

Error message

Transform node %r was not replaced as expected.

What it means

During pipeline construction Beam applies PTransformOverride replacements and then verifies (via _check_replacement) that no transform matching the override still exists in the graph. If a matching node survives, its replacement silently failed, so a RuntimeError is raised naming the node.

Source

Thrown at sdks/python/apache_beam/pipeline.py:530

          side_input_replacements[transform_node] = new_side_inputs

    self.visit(InputOutputUpdater(self))

    for transform, output_replacement in output_replacements.items():
      for tag, output in output_replacement:
        transform.replace_output(output, tag=tag)

    for transform, input_replacement in input_replacements.items():
      transform.replace_inputs(input_replacement)

    for transform, side_input_replacement in side_input_replacements.items():
      transform.replace_side_inputs(side_input_replacement)

  def _check_replacement(self, override: 'PTransformOverride') -> None:
    class ReplacementValidator(PipelineVisitor):
      def visit_transform(self, transform_node: AppliedPTransform) -> None:
        if override.matches(transform_node):
          raise RuntimeError(
              'Transform node %r was not replaced as expected.' %
              transform_node)

    self.visit(ReplacementValidator())

  def replace_all(self, replacements: Iterable['PTransformOverride']) -> None:
    """ Dynamically replaces PTransforms in the currently populated hierarchy.

    Currently this only works for replacements where input and output types
    are exactly the same.

    TODO: Update this to also work for transform overrides where input and
    output types are different.

    Args:
      replacements (list[~apache_beam.pipeline.PTransformOverride]): a list of
        :class:`~apache_beam.pipeline.PTransformOverride` objects.
    """

View on GitHub (pinned to 12126d8942)

Solutions

  1. Ensure your PTransformOverride.replace() returns a transform that no longer matches matches().
  2. Narrow matches() so the replacement output isn't matched again.
  3. Check for duplicate/overlapping overrides in the replacement list.
  4. If it comes from Beam internals, upgrade/downgrade the SDK — this often indicates an internal replacement bug; report with the transform detail.

Example fix

// before
class BadOverride(PTransformOverride):
  def matches(self, applied):
    return isinstance(applied.transform, MyTransform)
  def replace(self, applied):
    return applied.transform  # no-op replacement
// after
class GoodOverride(PTransformOverride):
  def matches(self, applied):
    return isinstance(applied.transform, MyTransform)
  def replace(self, applied):
    return MyReplacementTransform()  # different transform
Defensive patterns

Strategy: try-catch

Try / catch

try:
    pipeline.replace_all(replacements)
except RuntimeError as e:
    if 'was not replaced' in str(e):
        # review override matches/replace pair
        raise

Prevention

When it happens

Trigger: A custom PTransformOverride whose replace() doesn't actually replace the matched node (e.g. returns the same transform, or the replacement re-matches the override pattern); incorrect matches() implementation that matches its own output.

Common situations: Writing custom overrides for portable/foreign-language transforms; Beam-internal replacement machinery failing after an SDK version change; overrides whose replacement transform still matches the same predicate.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/2f4871d9c8bc2855. Report an issue: GitHub.