apache/beam · warning · ValueError

Too many matches

Error message

Too many matches

What it means

Internal helper _get_match in the Dataflow metrics reader found more protos matching the query than the caller's single-match expectation allows (e.g. a step-name translation matching multiple job steps), making the result ambiguous; despite the docstring promising the first match, this strict check raises.

Source

Thrown at sdks/python/apache_beam/runners/dataflow/dataflow_metrics.py:54

from apache_beam.metrics.metric import MetricResults
from apache_beam.metrics.metricbase import MetricName
from apache_beam.options.pipeline_options import GoogleCloudOptions
from apache_beam.options.pipeline_options import PipelineOptions

_LOGGER = logging.getLogger(__name__)


def _get_match(proto, filter_fn):
  """Finds and returns the first element that matches a query.

  If no element matches the query, it throws ValueError.
  If more than one element matches the query, it returns only the first.
  """
  query = [elm for elm in proto if filter_fn(elm)]
  if len(query) == 0:
    raise ValueError('Could not find element')
  elif len(query) > 1:
    raise ValueError('Too many matches')

  return query[0]


# V1b3 MetricStructuredName keys to accept and copy to the MetricKey labels.
STRUCTURED_NAME_LABELS = set(
    ['execution_step', 'original_name', 'output_user_name'])


class DataflowMetrics(MetricResults):
  """Implementation of MetricResults class for the Dataflow runner."""
  def __init__(self, dataflow_client=None, job_result=None, job_graph=None):
    """Initialize the Dataflow metrics object.

    Args:
      dataflow_client: apiclient.DataflowApplicationClient to interact with the
        dataflow service.
      job_result: DataflowPipelineResult with the state and id information of

View on GitHub (pinned to 12126d8942)

Solutions

1. Ensure unique step/transform names when building the pipeline (avoid reusing names in composites).
2. Catch ValueError in query/translation code and skip translation for that key.
3. Fix or regenerate the job graph so proto step names are unique.

Example fix

# before
user_step = metrics._translate_step_name(internal_name)
# after
try:
    user_step = metrics._translate_step_name(internal_name)
except ValueError:
    user_step = internal_name  # ambiguous or unknown step

from collections import Counter
c = Counter(s.name for s in job_graph.proto.steps)
dups = [n for n, k in c.items() if k > 1]
if dups:
    raise ValueError(f'Duplicate step names: {dups}')

try:
    user_name = metrics._translate_step_name(internal_name)
except ValueError as e:
    if 'Too many matches' in str(e):
        user_name = internal_name  # ambiguous
    raise

When it happens

Trigger: `_translate_step_name` searches job_graph.proto.steps for x.name == internal_name and the job graph contains duplicate step names — possible with duplicated transform names in the submitted pipeline proto.

Common situations: Pipelines with repeated composite naming leading to identical internal step names in the proto, causing step-name translation to blow up instead of returning the first match.

Related errors

• Could not find element
• Could not translate the internal step name %r since job grap
• Could not translate the internal step name %r.
• Can not query metrics. Job id is unknown.
• NotImplementedError

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/f560290fd3b8fd72. Report an issue: GitHub.