{"record":{"id":"6963789c4e790f6d","repo":"apache/beam","slug":"can-not-query-metrics-job-id-is-unknown","errorCode":null,"errorMessage":"Can not query metrics. Job id is unknown.","messagePattern":"Can not query metrics\\. Job id is unknown\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/runners/dataflow/dataflow_metrics.py","lineNumber":224,"sourceCode":"      dist_min = int(metric.distribution['min'])\n      dist_max = int(metric.distribution['max'])\n      dist_sum = int(metric.distribution['sum'])\n      return DistributionResult(\n          DistributionData(dist_sum, dist_count, dist_min, dist_max))\n      #TODO(https://github.com/apache/beam/issues/31788) support StringSet after\n      #  re-generate apiclient\n    else:\n      return None\n\n  def _get_metrics_from_dataflow(self, job_id=None):\n    \"\"\"Return cached metrics or query the dataflow service.\"\"\"\n    if not job_id:\n      try:\n        job_id = self.job_result.job_id()\n      except AttributeError:\n        job_id = None\n    if not job_id:\n      raise ValueError('Can not query metrics. Job id is unknown.')\n\n    if self._cached_metrics:\n      return self._cached_metrics\n\n    job_metrics = self._dataflow_client.get_job_metrics(job_id)\n    # If we cannot determine that the job has terminated,\n    # then metrics will not change and we can cache them.\n    if self.job_result and self.job_result.is_in_terminal_state():\n      self._cached_metrics = job_metrics\n    return job_metrics\n\n  def all_metrics(self, job_id=None):\n    \"\"\"Return all user and system metrics from the dataflow service.\"\"\"\n    metric_results = []\n    response = self._get_metrics_from_dataflow(job_id=job_id)\n    self._populate_metrics(response, metric_results, user_metrics=True)\n    self._populate_metrics(response, metric_results, user_metrics=False)\n    return metric_results","sourceCodeStart":206,"sourceCodeEnd":242,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/runners/dataflow/dataflow_metrics.py#L206-L242","documentation":"DataflowMetrics queries the Dataflow API by job id; before calling get_job_metrics it resolves the job id (argument or job_result.job_id()). If none is available it raises ValueError because metrics cannot be fetched without identifying the job.","triggerScenarios":"Calling all_metrics() or query() on a DataflowMetrics whose job_result has no job_id yet — e.g. metrics object built before job submission completed, or a failed/dry-run result that never received an id.","commonSituations":"Accessing pipeline.result.metrics() when the job launch failed or when using a mock/custom job result without job_id(), or querying before the Dataflow job was actually submitted.","solutions":["Ensure the pipeline run completed/submitted successfully and job_result.job_id() returns a real id before querying metrics.","Check the job launch result for errors; retry submission if the job never started.","Only construct/use DataflowMetrics after the job id is known (e.g. after wait_until_finish())."],"exampleFix":"# before\nresult = pipeline.run()\nprint(DataflowMetrics(result, client).all_metrics())\n# after\nresult = pipeline.run()\nresult.wait_until_finish()\nif result.job_id():\n    print(result.metrics().all_metrics())","handlingStrategy":"validation","validationCode":"job_id = getattr(job_result, 'job_id', lambda: None)()\nif not job_id:\n    raise RuntimeError('No Dataflow job id yet; run/wait for submission first')","typeGuard":"def has_job_id(job_result) -> bool:\n    try:\n        return bool(job_result.job_id())\n    except AttributeError:\n        return False","tryCatchPattern":"try:\n    m = metrics.all_metrics()\nexcept ValueError as e:\n    if 'Job id is unknown' in str(e):\n        m = None  # job never submitted\n    else:\n        raise","preventionTips":["Only query metrics after the job was submitted successfully.","Call wait_until_finish() before reading metrics.","Check launch errors when job_id() is absent."],"tags":["python","apache-beam","dataflow","metrics","job-id"],"backgroundTag":"missing-required-argument","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}