{"record":{"id":"a59dd430d7c7f89c","repo":"apache/beam","slug":"there-is-already-active-job-named-s-with-id-s-if-you-want-to","errorCode":null,"errorMessage":"There is already active job named %s with id: %s. If you want to submit a second job, try again by setting a different name using --job_name.","messagePattern":"There is already active job named (.+?) with id: (.+?)\\. If you want to submit a second job, try again by setting a different name using --job_name\\.","errorType":"exception","errorClass":"DataflowJobAlreadyExistsError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/runners/dataflow/internal/apiclient.py","lineNumber":919,"sourceCode":"      response = self._jobs_client.create_job(request=request)\n    except exceptions.GoogleAPICallError as e:\n      _LOGGER.error(\n          'HTTP status %d trying to create job'\n          ' at dataflow service endpoint %s',\n          e.code,\n          self.google_cloud_options.dataflow_endpoint)\n      _LOGGER.fatal('details of server error: %s', e)\n      raise\n\n    if response.client_request_id and \\\n        response.client_request_id != job.proto.client_request_id:\n      if self.google_cloud_options.update:\n        raise DataflowJobAlreadyExistsError(\n            \"The job named %s with id: %s has already been updated into job \"\n            \"id: %s and cannot be updated again.\" %\n            (response.name, job.proto.replace_job_id, response.id))\n      else:\n        raise DataflowJobAlreadyExistsError(\n            'There is already active job named %s with id: %s. If you want to '\n            'submit a second job, try again by setting a different name using '\n            '--job_name.' % (response.name, response.id))\n\n    _LOGGER.info('Create job: %s', response)\n    # The response is a Job proto with the id for the new job.\n    _LOGGER.info('Created job with id: [%s]', response.id)\n    _LOGGER.info('Submitted job: %s', response.id)\n    _LOGGER.info(\n        'To access the Dataflow monitoring console, please navigate to '\n        'https://console.cloud.google.com/dataflow/jobs/%s/%s?project=%s',\n        self.google_cloud_options.region,\n        response.id,\n        self.google_cloud_options.project)\n\n    return response\n\n  @retry.with_exponential_backoff()  # Using retry defaults from utils/retry.py","sourceCodeStart":901,"sourceCodeEnd":937,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py#L901-L937","documentation":"DataflowJobAlreadyExistsError thrown at submission when an active job with the same name already exists and --update was not requested. Dataflow requires job names to be unique among active jobs, so a plain submit colliding with a running job is rejected. Beam raises this with guidance to change the name or use --update.","triggerScenarios":"Submitting a pipeline whose --job_name matches an existing RUNNING/DRAINING Dataflow job while the update option is off.","commonSituations":"Hardcoded job_name in a script run twice; scheduled jobs where the previous run is still draining; two developers launching the same named pipeline.","solutions":["Pick a unique job name, e.g. append a timestamp: --job_name=my-job-$(date +%Y%m%d-%H%M%S).","Pass --update to update the existing job instead of creating a new one.","Cancel or drain the existing job before resubmitting with the same name."],"exampleFix":"// before\npython -m my_pipeline --job_name=my-job   # my-job already active\n// after\npython -m my_pipeline --job_name=my-job-$(date +%Y%m%d-%H%M%S)\n# or add --update to replace the existing job","handlingStrategy":"try-catch","validationCode":"jobs = dataflow_client.list_jobs(project=proj, location=loc).jobs\nif any(j.name == job_name and j.current_state in ('JOB_STATE_RUNNING','JOB_STATE_DRAINING') for j in jobs):\n    raise RuntimeError(f'Job {job_name} is already active')","typeGuard":null,"tryCatchPattern":"try:\n    result = pipeline.run()\nexcept DataflowJobAlreadyExistsError as e:\n    logging.error('Job name collision: %s', e)\n    sys.exit(2)  # caller retries with a fresh name","preventionTips":["Generate job names with a timestamp or run id suffix.","Cancel/drain old jobs before reusing names in scheduled pipelines.","Avoid hardcoded job names shared across teams."],"tags":["python","apache-beam","dataflow","name-conflict"],"backgroundTag":"file-already-exists","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}