{"record":{"id":"de61408b0c6e24d0","repo":"apache/beam","slug":"could-not-find-element","errorCode":null,"errorMessage":"Could not find element","messagePattern":"Could not find element","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"warning","filePath":"sdks/python/apache_beam/runners/dataflow/dataflow_metrics.py","lineNumber":52,"sourceCode":"from apache_beam.metrics.execution import MetricKey\nfrom apache_beam.metrics.execution import MetricResult\nfrom apache_beam.metrics.metric import MetricResults\nfrom apache_beam.metrics.metricbase import MetricName\nfrom apache_beam.options.pipeline_options import GoogleCloudOptions\nfrom apache_beam.options.pipeline_options import PipelineOptions\n\n_LOGGER = logging.getLogger(__name__)\n\n\ndef _get_match(proto, filter_fn):\n  \"\"\"Finds and returns the first element that matches a query.\n\n  If no element matches the query, it throws ValueError.\n  If more than one element matches the query, it returns only the first.\n  \"\"\"\n  query = [elm for elm in proto if filter_fn(elm)]\n  if len(query) == 0:\n    raise ValueError('Could not find element')\n  elif len(query) > 1:\n    raise ValueError('Too many matches')\n\n  return query[0]\n\n\n# V1b3 MetricStructuredName keys to accept and copy to the MetricKey labels.\nSTRUCTURED_NAME_LABELS = set(\n    ['execution_step', 'original_name', 'output_user_name'])\n\n\nclass DataflowMetrics(MetricResults):\n  \"\"\"Implementation of MetricResults class for the Dataflow runner.\"\"\"\n  def __init__(self, dataflow_client=None, job_result=None, job_graph=None):\n    \"\"\"Initialize the Dataflow metrics object.\n\n    Args:\n      dataflow_client: apiclient.DataflowApplicationClient to interact with the","sourceCodeStart":34,"sourceCodeEnd":70,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/runners/dataflow/dataflow_metrics.py#L34-L70","documentation":"`_get_match` filters a protobuf sequence with a predicate and requires exactly one match; zero matches raises ValueError('Could not find element'). In dataflow_metrics it's used to look up pipeline steps by name when translating internal step names.","triggerScenarios":"`_translate_step_name` calls `_get_match(job_graph.proto.steps, lambda x: x.name == internal_name)` and no step in the job graph has that internal name (e.g. stale metric keys from a different job, or job graph mismatch).","commonSituations":"Querying Dataflow metrics for a job whose graph doesn't correspond to the metric step names — e.g. metrics from a retried/updated job or when the harness emits step names absent from the submitted proto.","solutions":["Catch ValueError from _get_match; dataflow_metrics itself catches and falls through to 'Could not translate' handling in the caller.","Refresh/re-fetch the job graph and metrics from the same job_id so names match.","Check that you're querying the correct job's metrics object (DataflowMetrics for this specific job result)."],"exampleFix":"# before\nstep = metrics._translate_step_name(name)\n# after\ntry:\n    step = metrics._translate_step_name(name)\nexcept ValueError as e:\n    logging.warning('step name lookup failed: %s', e)\n    step = name  # fall back to internal name","handlingStrategy":"try-catch","validationCode":"step_names = {s.name for s in job_graph.proto.steps}\nassert internal_name in step_names, f'{internal_name} not in job graph'","typeGuard":null,"tryCatchPattern":"try:\n    user_name = metrics._translate_step_name(internal_name)\nexcept ValueError:\n    user_name = internal_name","preventionTips":["Query metrics from the same job result that produced the graph.","Handle per-key translation failures when walking all metric keys.","Log the internal name and job id to diagnose graph/metric mismatch."],"tags":["python","apache-beam","dataflow","metrics"],"backgroundTag":"record-not-found","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}