apache/beam · warning · ValueError

Could not translate the internal step name %r since job grap

Error message

Could not translate the internal step name %r since job graph is not available.

What it means

DataflowMetrics translates internal Dataflow step names (like 's1') to user-facing names using the job graph; if the graph was not provided (self._job_graph is falsy), translation is impossible and it raises ValueError.

Source

Thrown at sdks/python/apache_beam/runners/dataflow/dataflow_metrics.py:99

    self._cached_metrics = None
    self._job_graph = job_graph

  @staticmethod
  def _is_counter(metric_result):
    return isinstance(metric_result.attempted, numbers.Number)

  @staticmethod
  def _is_distribution(metric_result):
    return isinstance(metric_result.attempted, DistributionResult)

  @staticmethod
  def _is_string_set(metric_result):
    return isinstance(metric_result.attempted, set)

  def _translate_step_name(self, internal_name):
    """Translate between internal step names (e.g. "s1") and user step names."""
    if not self._job_graph:
      raise ValueError(
          'Could not translate the internal step name %r since job graph is '
          'not available.' % internal_name)
    user_step_name = None
    if (self._job_graph and internal_name
        in self._job_graph.proto_pipeline.components.transforms.keys()):
      # Dataflow Portable Runner with portable job submission uses proto transform map
      # IDs for step names. Also PTransform.unique_name maps to user step names.
      # Hence we lookup user step names based on the proto.
      user_step_name = self._job_graph.proto_pipeline.components.transforms[
          internal_name].unique_name
    else:
      try:
        step = _get_match(
            self._job_graph.proto.steps, lambda x: x.name == internal_name)
        user_step_name = step.properties.get('user_name')
      except ValueError:
        pass  # Exception is handled below.
    if not user_step_name:

View on GitHub (pinned to 12126d8942)

Solutions

Pass the job graph when constructing the metrics helper (DataflowMetrics(..., job_graph)).
Fetch the job graph from the Dataflow API before querying if it wasn't retained.
Accept internal step names in results if translation is unavailable (catch ValueError and use the raw name).

Example fix

# before
metrics = DataflowMetrics(job_result, dataflow_client)  # no graph
# after
metrics = DataflowMetrics(job_result, dataflow_client, job_graph=job_graph)

Defensive patterns

Strategy: validation

Validate before calling

if metrics._job_graph is None:
    raise RuntimeError('Provide job_graph to DataflowMetrics for step-name translation')

Type guard

def has_job_graph(metrics) -> bool:
    return getattr(metrics, '_job_graph', None) is not None

Try / catch

try:
    name = metrics._translate_step_name(internal)
except ValueError as e:
    if 'job graph is not available' in str(e):
        name = internal
    else:
        raise

Prevention

Always pass the job graph when constructing DataflowMetrics.
Persist the job graph alongside the job result for later queries.
Fall back to raw internal names when translation is unavailable.

When it happens

Trigger: Creating a DataflowMetrics/query object without a job_graph and then querying metrics whose step names need translation — e.g. DataflowMetrics(job_result, dataflow_client) with no graph arg, then reading metric keys.

Common situations: Using the Dataflow metrics API programmatically (google.cloud.dataflow client) without passing the job graph retrieved at submission time.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/84ccefd77e9a10bb. Report an issue: GitHub.