apache/beam · error · ValueError

No running job found with name

Error message

No running job found with name '%s'

What it means

ValueError from job_id_for_name when it pages through all jobs visible to the project and finds none whose name matches and whose current state is RUNNING or DRAINING. The lookup only considers running/draining jobs, so completed, failed, or cancelled jobs are invisible to it.

Solutions

1. Verify the job is currently RUNNING or DRAINING in the Dataflow console; use a different lookup (list all jobs) if it finished.
2. Check the job name spelling and case exactly matches the Dataflow job name.
3. Confirm the client's project/region options point at where the job runs.

Example fix

// before
job_id = client.job_id_for_name('my_job')  # job already finished -> ValueError
// after
# guard: ensure the job exists and is active, or handle the error
try:
  job_id = client.job_id_for_name('my_job')
except ValueError:
  job_id = None  # job not running (yet/anymore)

active = [j for j in client._jobs_client.list_jobs(project=proj).jobs
          if j.name == job_name and j.current_state in ('JOB_STATE_RUNNING','JOB_STATE_DRAINING')]
if not active:
    logging.warning('No active job named %s', job_name)

try:
    job_id = client.job_id_for_name(name)
except ValueError as e:
    if 'No running job found' in str(e):
        job_id = None  # job finished or never existed
    else:
        raise

When it happens

Trigger: Calling job_id_for_name(name) when no job with that exact name is currently running or draining — the job finished, was cancelled, the name is misspelled, or the job lives in a different project/region.

Common situations: Looking up a job after it already completed; case-sensitive name mismatch; job name vs job id confusion; running against the wrong GCP project.

Understand the failure class

Background: Record Not Found Errors: "not found", RecordNotFound, and "was not found" — what they mean and how to fix them — this error's family across 28 libraries.

Related errors

• Can not query metrics. Job id is unknown.
• Coder for the GroupByKey operation
• CombineFn.setup and CombineFn.teardown are not supported…
• Could not find element
• Could not translate the internal step name %r.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/127e4bb29972df6a. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/runners/dataflow/internal/apiclient.py:1094

    response = self._messages_client.list_job_messages(request=request)
    return response.job_messages, response.next_page_token

  def job_id_for_name(self, job_name):
    token = None
    while True:
      request = dataflow.ListJobsRequest(
          project_id=self.google_cloud_options.project,
          location=self.google_cloud_options.region,
          page_token=token)
      response = self._jobs_client.list_jobs(request)
      for job in response:
        if (job.name == job_name and
            job.current_state in [dataflow.JobState.JOB_STATE_RUNNING,
                                  dataflow.JobState.JOB_STATE_DRAINING]):
          return job.id
      token = response.next_page_token
      if token is None:
        raise ValueError("No running job found with name '%s'" % job_name)


class MetricUpdateTranslators(object):
  """Translators between accumulators and dataflow metric updates."""
  @staticmethod
  def translate_boolean(
      accumulator, metric_update_proto: dataflow.MetricUpdate):
    metric_update_proto.scalar = accumulator.value

  @staticmethod
  def translate_scalar_mean_int(
      accumulator, metric_update_proto: dataflow.MetricUpdate):
    if accumulator.count:
      metric_update_proto.kind = 'Mean'
      metric_update_proto.mean_sum = accumulator.sum
      metric_update_proto.mean_count = accumulator.count
    else:
      metric_update_proto.kind = None  # type: ignore

View on GitHub (pinned to 12126d8942)