apache/beam · error · ValueError

Unable to parse jar URL

Error message

Unable to parse jar URL "%s". If using a full URL, make sure the scheme is specified. If using a local file path, make sure the file exists; you may have to first build the job server using `./gradlew runners:spark:3:job-server:shadowJar`.

What it means

SparkUberJarJobServer.executable_jar validates the uber jar location the same way SparkRunner does: a value that is neither an existing file nor a scheme-qualified URL is ambiguous, so Beam raises ValueError with instructions to build the job server or fix the URL.

Solutions

Build the uber jar with ./gradlew runners:spark:3:job-server:shadowJar and pass the real, existing path.
Correct the path so the file exists where the driver runs.
Add a scheme if it is remote: gs://..., https://..., file:///... .
Log/verify os.path.exists(jar) and urllib.parse.urlparse(jar).scheme before launching the pipeline.

Example fix

// before
SparkUberJarJobServer('build/libs/job-server.jar', options)  # missing file, no scheme
// after
SparkUberJarJobServer('gs://my-bucket/beam-runners-spark-3-job-server.jar', options)

Defensive patterns

Strategy: validation

Validate before calling

import os, urllib.parse
jar = uber_jar_path
assert os.path.exists(jar) or urllib.parse.urlparse(jar).scheme, f'bad jar location: {jar}'

Try / catch

try:
    server.create_beam_job(...)
except ValueError as e:
    if 'Unable to parse jar URL' in str(e):
        upload_or_build_jar_and_retry()
    else:
        raise

Prevention

Verify jar existence from the same host/CWD that submits
Pre-upload remote jars and reference them by scheme
Fail fast with your own existence check

When it happens

Trigger: Constructing SparkUberJarJobServer with an executable jar string that does not exist on disk and lacks a scheme (e.g. '/missing/dir/server.jar' that was deleted, or 'myhost/jar' without http://); reached via create_beam_job during pipeline launch.

Common situations: Uber jar built on another machine and never copied over; relative path resolved from a different CWD; URL pasted without its scheme; CI artifact not uploaded before the job ran.

Understand the failure class

Background: "Invalid URL" errors: why new URL(), URI.parse, and reqwest::Url reject your string — missing scheme, whitespace, and bad path format — this error's family across 39 libraries.

Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/78e189c9c92b1d32. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/runners/portability/spark_uber_jar_job_server.py:69

        options.view_as(pipeline_options.JobServerOptions).artifact_port)
    self._temp_dir = tempfile.mkdtemp(prefix='apache-beam-spark')
    spark_options = options.view_as(pipeline_options.SparkRunnerOptions)
    self._executable_jar = spark_options.spark_job_server_jar
    self._spark_version = spark_options.spark_version
    self._user_agent = options.view_as(pipeline_options.SetupOptions).user_agent

  def start(self):
    return self

  def stop(self):
    pass

  def executable_jar(self):
    if self._executable_jar:
      if not os.path.exists(self._executable_jar):
        parsed = urllib.parse.urlparse(self._executable_jar)
        if not parsed.scheme:
          raise ValueError(
              'Unable to parse jar URL "%s". If using a full URL, make sure '
              'the scheme is specified. If using a local file path, make sure '
              'the file exists; you may have to first build the job server '
              'using `./gradlew runners:spark:3:job-server:shadowJar`.' %
              self._executable_jar)
      url = self._executable_jar
    else:
      if self._spark_version == '2':
        raise ValueError('Support for Spark 2 was dropped.')
      else:
        url = job_server.JavaJarJobServer.path_to_beam_jar(
            ':runners:spark:3:job-server:shadowJar')
    return job_server.JavaJarJobServer.local_jar(
        url, user_agent=self._user_agent)

  def create_beam_job(self, job_id, job_name, pipeline, options):
    return SparkBeamJob(
        self._rest_url,

View on GitHub (pinned to 12126d8942)