{"record":{"id":"d4ada54b91605efc","repo":"apache/beam","slug":"you-must-specify-a-job-endpoint-when-using-runner","errorCode":null,"errorMessage":"You must specify a --job_endpoint when using --runner=PortableRunner. Alternatively, you may specify which portable runner you intend to use, such as --runner=FlinkRunner or --runner=SparkRunner.","messagePattern":"You must specify a --job_endpoint when using --runner=PortableRunner\\. Alternatively, you may specify which portable runner you intend to use, such as --runner=FlinkRunner or --runner=SparkRunner\\.","errorType":"validation","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/runners/portability/portable_runner.py","lineNumber":268,"sourceCode":"class PortableRunner(runner.PipelineRunner):\n  \"\"\"\n    Experimental: No backward compatibility guaranteed.\n    A BeamRunner that executes Python pipelines via the Beam Job API.\n\n    This runner is a stub and does not run the actual job.\n    This runner schedules the job on a job service. The responsibility of\n    running and managing the job lies with the job service used.\n  \"\"\"\n  def __init__(self):\n    self._dockerized_job_server: Optional[job_server.JobServer] = None\n\n  @staticmethod\n  def _create_environment(options: PipelineOptions) -> environments.Environment:\n    return environments.Environment.from_options(\n        options.view_as(PortableOptions))\n\n  def default_job_server(self, options):\n    raise NotImplementedError(\n        'You must specify a --job_endpoint when using --runner=PortableRunner. '\n        'Alternatively, you may specify which portable runner you intend to '\n        'use, such as --runner=FlinkRunner or --runner=SparkRunner.')\n\n  def create_job_service_handle(self, job_service, options) -> JobServiceHandle:\n    return JobServiceHandle(job_service, options)\n\n  def create_job_service(self, options: PipelineOptions) -> JobServiceHandle:\n    \"\"\"\n    Start the job service and return a `JobServiceHandle`\n    \"\"\"\n    job_endpoint = options.view_as(PortableOptions).job_endpoint\n    if job_endpoint:\n      if job_endpoint == 'embed':\n        server: job_server.JobServer = job_server.EmbeddedJobServer()\n      else:\n        job_server_timeout = options.view_as(PortableOptions).job_server_timeout\n        server = job_server.ExternalJobServer(job_endpoint, job_server_timeout)","sourceCodeStart":250,"sourceCodeEnd":286,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/runners/portability/portable_runner.py#L250-L286","documentation":"PortableRunner has no built-in job server; it needs a --job_endpoint pointing at an already-running job service. default_job_server() raises this NotImplementedError because the base PortableRunner cannot start a server itself, unlike concrete runners (FlinkRunner, SparkRunner) that override it.","triggerScenarios":"Running a pipeline with --runner=PortableRunner and no --job_endpoint, so create_job_service -> default_job_server hits the base-class NotImplementedError.","commonSituations":"Testing against a local Beam job service but forgetting the endpoint flag; switching a pipeline from DirectRunner to PortableRunner without adding job-service configuration; tutorials that assume FlinkRunner's default endpoint.","solutions":["Start a job service and pass --job_endpoint=<host:port> (e.g. localhost:8099).","Use a concrete portable runner instead: --runner=FlinkRunner or --runner=SparkRunner.","If you only want local execution, use --runner=DirectRunner rather than PortableRunner."],"exampleFix":"// before\n--runner=PortableRunner\n// after\n--runner=PortableRunner --job_endpoint=localhost:8099","handlingStrategy":"validation","validationCode":"opts = PipelineOptions(flags)\nportable = opts.view_as(PortableOptions)\nif portable.runner_type == 'PortableRunner' and not portable.job_endpoint:\n    raise SystemExit('--runner=PortableRunner requires --job_endpoint (or use FlinkRunner/SparkRunner)')","typeGuard":null,"tryCatchPattern":"try:\n    result = pipeline.run()\n    result.wait_until_finish()\nexcept NotImplementedError as e:\n    if 'job_endpoint' in str(e):\n        sys.exit('Set --job_endpoint or choose FlinkRunner/SparkRunner')\n    raise","preventionTips":["Never select PortableRunner without also setting --job_endpoint.","Prefer concrete runners (FlinkRunner, SparkRunner) which supply default job servers.","Validate pipeline options in your launcher script before submitting."],"tags":["configuration","portable-runner","missing-endpoint"],"backgroundTag":"missing-required-config-field","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}