apache/beam · error · ValueError

Unable to parse jar URL

Error message

Unable to parse jar URL "%s". If using a full URL, make sure the scheme is specified. If using a local file path, make sure the file exists; you may have to first build the job server using `./gradlew runners:spark:3:job-server:shadowJar`.

What it means

SparkRunner.path_to_jar validates the user-supplied job server jar. If the given path does not exist as a local file and the string has no URL scheme (http://, file://, gs://, etc.), Beam cannot classify it as either a file or a URL and raises ValueError with remediation hints.

Solutions

1. Build the jar: ./gradlew runners:spark:3:job-server:shadowJar, then pass the full path to the produced artifact.
2. Fix the path so the file actually exists at the given location (check with ls / os.path.exists).
3. If it is a remote jar, prefix the scheme, e.g. gs://bucket/spark_job_server.jar or https://host/jar.
4. Confirm the path is absolute or relative to the directory from which the pipeline is launched.

Example fix

// before
--spark_job_server_jar=/builds/job-server.jar   # file does not exist
// after
./gradlew runners:spark:3:job-server:shadowJar
--spark_job_server_jar=$(pwd)/runners/spark/3/job-server/build/libs/beam-runners-spark-3-job-server-*.jar

import os, urllib.parse
jar = options.spark_job_server_jar
if not (os.path.exists(jar) or urllib.parse.urlparse(jar).scheme):
    raise SystemExit(f'Job server jar {jar!r} is not an existing file or a scheme-qualified URL')

try:
    runner.default_job_server(options)
except ValueError as e:
    if 'Unable to parse jar URL' in str(e):
        build_or_rebuild_jar()  # ./gradlew runners:spark:3:job-server:shadowJar
    else:
        raise

When it happens

Trigger: Passing --spark_job_server_jar (or constructing SparkRunner with _jar) set to a string that is neither an existing file nor a scheme-qualified URL, e.g. a typo'd path or a bare hostname/path.

Common situations: Developer forgot to build the job server shadowJar with ./gradlew runners:spark:3:job-server:shadowJar; jar was moved/deleted; path typed relative to the wrong working directory; copied a URL but dropped the http(s):// prefix.

Understand the failure class

Background: "Invalid URL" errors: why new URL(), URI.parse, and reqwest::Url reject your string — missing scheme, whitespace, and bad path format — this error's family across 39 libraries.

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• f'Invalid path or url
• Property "version" not found in…
• Unable to parse jar URL
• Cannot create a temporary directory for root path prefix
• error in creating relative path

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/be5a2671305fdef7. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/runners/portability/spark_runner.py:104

class SparkJarJobServer(job_server.JavaJarJobServer):
  def __init__(self, options):
    super().__init__(options)
    options = options.view_as(pipeline_options.SparkRunnerOptions)
    self._jar = options.spark_job_server_jar
    self._master_url = options.spark_master_url
    self._spark_version = options.spark_version
    self._jvm_properties = list(self._jvm_properties)
    for arg in SPARK_JAR_JOB_SERVER_JVM_ARGS:
      if arg not in self._jvm_properties:
        self._jvm_properties.append(arg)

  def path_to_jar(self):
    if self._jar:
      if not os.path.exists(self._jar):
        url = urllib.parse.urlparse(self._jar)
        if not url.scheme:
          raise ValueError(
              'Unable to parse jar URL "%s". If using a full URL, make sure '
              'the scheme is specified. If using a local file path, make sure '
              'the file exists; you may have to first build the job server '
              'using `./gradlew runners:spark:3:job-server:shadowJar`.' %
              self._jar)
      return self._jar
    else:
      if self._spark_version == '2':
        raise ValueError('Support for Spark 2 was dropped.')
      return self.path_to_beam_jar(':runners:spark:3:job-server:shadowJar')

  def java_arguments(
      self, job_port, artifact_port, expansion_port, artifacts_dir):
    return [
        '--spark-master-url',
        self._master_url,
        '--artifacts-dir',
        artifacts_dir,

View on GitHub (pinned to 12126d8942)