apache/beam · error · RuntimeError

--pid_file must be specified with --stop

Error message

--pid_file must be specified with --stop

What it means

local_job_service_main's run() validates CLI options: --stop requires --pid_file to know which process to kill. Passing --stop without --pid_file is a command-line usage error, so it raises RuntimeError immediately after parsing.

Source

Thrown at sdks/python/apache_beam/runners/portability/local_job_service_main.py:80

      '--background',
      action='store_true',
      help='Start the server up as a background process.'
      ' Will fail if pid_file already exists, unless --stop is also specified.')
  parser.add_argument(
      '--stderr_file',
      help='Where to write stderr (if not specified, merged with stdout).')
  parser.add_argument(
      '--stdout_file', help='Where to write stdout for background job service.')
  parser.add_argument(
      '--stop',
      action='store_true',
      help='Stop the existing process, if any, specified in pid_file.'
      ' Will not start up a new service unless --background is specified.')
  options = parser.parse_args(argv)

  if options.stop:
    if not options.pid_file:
      raise RuntimeError('--pid_file must be specified with --stop')
    if os.path.exists(options.pid_file):
      with open(options.pid_file) as fin:
        pid = int(fin.read())
      print('Killing process at', pid)
      try:
        os.kill(pid, signal.SIGTERM)
      except Exception:
        print('Process', pid, 'already killed.')
      os.unlink(options.pid_file)
    else:
      print('Process id file', options.pid_file, 'already removed.')
    if not options.background:
      return

  if options.background:
    if not options.pid_file:
      raise RuntimeError('--pid_file must be specified with --start')
    if options.stop:

View on GitHub (pinned to 12126d8942)

Solutions

1. Add --pid_file=/path/to/pid pointing at the file created when the service started.
2. Restart the stop command exactly as used previously, including the same --pid_file path.
3. If the pid file is lost, find the process manually (pgrep -f local_job_service) and kill it directly.

Example fix

// before
python -m apache_beam.runners.portability.local_job_service_main --stop
// after
python -m apache_beam.runners.portability.local_job_service_main --stop --pid_file=/tmp/beam_service.pid

if args.stop and not args.pid_file:
    parser.error('--pid_file must be specified with --stop')

try:
    main(argv)
except RuntimeError as e:
    if 'pid_file' in str(e): print_usage_and_exit(2)

When it happens

Trigger: Invoking `python -m apache_beam.runners.portability.local_job_service_main --stop` without also supplying --pid_file pointing at the file holding the service's PID.

Common situations: Copy-pasting only part of a documented stop command; forgetting that the pid_file was written to a custom path when the service was started; scripts calling --stop without persisting the pid file first.

Understand the failure class

Background: "--flag is required" and "must specify" CLI errors: how missing-required-flag validation works and how to fix it — this error's family across 20 libraries.

Related errors

• Unrecognized value for stable unique names:
• Expecting exactly one field, found
• %s expects a single %s tagged PCollection<Row> input
• smallBatchTag and largeBatchTag must not be null
• Unable to rename unequal number of sources and destinations

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/ab39cd2618ebb013. Report an issue: GitHub.