apache/beam · warning · ValueError

Could not translate the internal step name %r.

Error message

Could not translate the internal step name %r.

What it means

After consulting both the proto transform map and job proto steps, if no user step name was found for the internal step name, DataflowMetrics raises ValueError('Could not translate the internal step name %r.').

Source

Thrown at sdks/python/apache_beam/runners/dataflow/dataflow_metrics.py:118

          'Could not translate the internal step name %r since job graph is '
          'not available.' % internal_name)
    user_step_name = None
    if (self._job_graph and internal_name
        in self._job_graph.proto_pipeline.components.transforms.keys()):
      # Dataflow Portable Runner with portable job submission uses proto transform map
      # IDs for step names. Also PTransform.unique_name maps to user step names.
      # Hence we lookup user step names based on the proto.
      user_step_name = self._job_graph.proto_pipeline.components.transforms[
          internal_name].unique_name
    else:
      try:
        step = _get_match(
            self._job_graph.proto.steps, lambda x: x.name == internal_name)
        user_step_name = step.properties.get('user_name')
      except ValueError:
        pass  # Exception is handled below.
    if not user_step_name:
      raise ValueError(
          'Could not translate the internal step name %r.' % internal_name)
    return user_step_name

  def _get_metric_key(self, metric):
    """Populate the MetricKey object for a queried metric result."""
    step = ""
    name = metric.name.name  # Always extract a name
    labels = {}
    try:  # Try to extract the user step name.
      # If ValueError is thrown within this try-block, it is because of
      # one of the following:
      # 1. Unable to translate the step name. Only happening with improperly
      #   formatted job graph (unlikely), or step name not being the internal
      #   step name (only happens for unstructured-named metrics).
      # 2. Unable to unpack [step] or [namespace]; which should only happen
      #   for unstructured names.
      step = metric.name.context['step']
      step = self._translate_step_name(step)

View on GitHub (pinned to 12126d8942)

Solutions

  1. Wrap metric queries in try/except ValueError and skip or label untranslatable steps.
  2. Make sure the job graph belongs to the same job (same job_id/revision) as the metrics.
  3. Filter out system/internal metrics before translation.

Example fix

# before
for key in result.keys():
    user_name = metrics._translate_step_name(key.step)
# after
for key in result.keys():
    try:
        user_name = metrics._translate_step_name(key.step)
    except ValueError:
        user_name = key.step  # keep internal name
Defensive patterns

Strategy: try-catch

Validate before calling

known = set(job_graph.proto_pipeline.components.transforms) | {s.name for s in job_graph.proto.steps}
if internal_name not in known:
    skip_translation(internal_name)

Try / catch

try:
    user_name = metrics._translate_step_name(internal_name)
except ValueError:
    user_name = internal_name  # untranslated internal step

Prevention

When it happens

Trigger: Querying a metric whose internal step name exists neither in job_graph.proto_pipeline.components.transforms nor as a step name in job_graph.proto.steps (or the step lacks a 'user_name' property).

Common situations: Job graph and metrics out of sync: querying metrics from an updated/relaunched job using an older graph, or system steps without user_name labels.

Understand the failure class

Background: Record Not Found Errors: "not found", RecordNotFound, and "was not found" — what they mean and how to fix them — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/f21a82e11b4bd6dd. Report an issue: GitHub.