{"record":{"id":"be5a2671305fdef7","repo":"apache/beam","slug":"unable-to-parse-jar-url-s-if-using-a-full-url-make-sure-the-be5a26","errorCode":null,"errorMessage":"Unable to parse jar URL \"%s\". If using a full URL, make sure the scheme is specified. If using a local file path, make sure the file exists; you may have to first build the job server using `./gradlew runners:spark:3:job-server:shadowJar`.","messagePattern":"Unable to parse jar URL \"(.+?)\"\\. If using a full URL, make sure the scheme is specified\\. If using a local file path, make sure the file exists; you may have to first build the job server using `\\./gradlew runners:spark:3:job-server:shadowJar`\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/runners/portability/spark_runner.py","lineNumber":104,"sourceCode":"\nclass SparkJarJobServer(job_server.JavaJarJobServer):\n  def __init__(self, options):\n    super().__init__(options)\n    options = options.view_as(pipeline_options.SparkRunnerOptions)\n    self._jar = options.spark_job_server_jar\n    self._master_url = options.spark_master_url\n    self._spark_version = options.spark_version\n    self._jvm_properties = list(self._jvm_properties)\n    for arg in SPARK_JAR_JOB_SERVER_JVM_ARGS:\n      if arg not in self._jvm_properties:\n        self._jvm_properties.append(arg)\n\n  def path_to_jar(self):\n    if self._jar:\n      if not os.path.exists(self._jar):\n        url = urllib.parse.urlparse(self._jar)\n        if not url.scheme:\n          raise ValueError(\n              'Unable to parse jar URL \"%s\". If using a full URL, make sure '\n              'the scheme is specified. If using a local file path, make sure '\n              'the file exists; you may have to first build the job server '\n              'using `./gradlew runners:spark:3:job-server:shadowJar`.' %\n              self._jar)\n      return self._jar\n    else:\n      if self._spark_version == '2':\n        raise ValueError('Support for Spark 2 was dropped.')\n      return self.path_to_beam_jar(':runners:spark:3:job-server:shadowJar')\n\n  def java_arguments(\n      self, job_port, artifact_port, expansion_port, artifacts_dir):\n    return [\n        '--spark-master-url',\n        self._master_url,\n        '--artifacts-dir',\n        artifacts_dir,","sourceCodeStart":86,"sourceCodeEnd":122,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/runners/portability/spark_runner.py#L86-L122","documentation":"SparkRunner.path_to_jar validates the user-supplied job server jar. If the given path does not exist as a local file and the string has no URL scheme (http://, file://, gs://, etc.), Beam cannot classify it as either a file or a URL and raises ValueError with remediation hints.","triggerScenarios":"Passing --spark_job_server_jar (or constructing SparkRunner with _jar) set to a string that is neither an existing file nor a scheme-qualified URL, e.g. a typo'd path or a bare hostname/path.","commonSituations":"Developer forgot to build the job server shadowJar with ./gradlew runners:spark:3:job-server:shadowJar; jar was moved/deleted; path typed relative to the wrong working directory; copied a URL but dropped the http(s):// prefix.","solutions":["Build the jar: ./gradlew runners:spark:3:job-server:shadowJar, then pass the full path to the produced artifact.","Fix the path so the file actually exists at the given location (check with ls / os.path.exists).","If it is a remote jar, prefix the scheme, e.g. gs://bucket/spark_job_server.jar or https://host/jar.","Confirm the path is absolute or relative to the directory from which the pipeline is launched."],"exampleFix":"// before\n--spark_job_server_jar=/builds/job-server.jar   # file does not exist\n// after\n./gradlew runners:spark:3:job-server:shadowJar\n--spark_job_server_jar=$(pwd)/runners/spark/3/job-server/build/libs/beam-runners-spark-3-job-server-*.jar","handlingStrategy":"validation","validationCode":"import os, urllib.parse\njar = options.spark_job_server_jar\nif not (os.path.exists(jar) or urllib.parse.urlparse(jar).scheme):\n    raise SystemExit(f'Job server jar {jar!r} is not an existing file or a scheme-qualified URL')","typeGuard":null,"tryCatchPattern":"try:\n    runner.default_job_server(options)\nexcept ValueError as e:\n    if 'Unable to parse jar URL' in str(e):\n        build_or_rebuild_jar()  # ./gradlew runners:spark:3:job-server:shadowJar\n    else:\n        raise","preventionTips":["Build the shadowJar before first run and store its path in one config place","Use absolute paths for local jars","Always include a scheme for remote jars"],"tags":["apache-beam","spark","jar","path"],"backgroundTag":"invalid-url-format","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}