apache/beam · error · ValueError

Property "version" not found in…

Error message

Property "version" not found in spark-version-info.properties.

What it means

To stay compatible with the client's Spark version, Beam reads spark-version-info.properties from inside the user's uber jar looking for a 'version' entry. If the property file is missing the key (or it is blank), Beam raises ValueError because it cannot determine which Spark job server artifact to fetch.

Solutions

1. Rebuild the uber jar with the standard Spark/Gradle build so spark-version-info.properties contains version=<x.y.z>.
2. Verify the entry: unzip -p your.jar spark-version-info.properties and confirm a non-empty version= line.
3. If you control packaging, ensure resource filtering/merging does not blank out the version key.
4. Rely on _get_client_spark_version's fallback only intentionally — fix the jar rather than depending on exception handling.

Example fix

// before (broken resource)
spark-version-info.properties: version=
// after
spark-version-info.properties: version=3.5.0

import zipfile
with zipfile.ZipFile(jar_path) as z:
    props = z.read('spark-version-info.properties').decode()
    version = dict(
        line.split('=', 1) for line in props.splitlines() if '=' in line
    ).get('version', '').strip()
assert version, f'{jar_path} lacks a non-empty version in spark-version-info.properties'

When it happens

Trigger: The uber jar supplied to SparkUberJarJobServer was built without (or with an empty) spark-version-info.properties, so the zip entry exists but no line matches key 'version' with a non-empty value; _get_client_spark_version then, after this error, falls back to its except path.

Common situations: Hand-assembled or repackaged uber jar that dropped the properties file contents; custom Spark builds where version metadata was stripped; jars produced by shading tools that overwrite resource files.

Related errors

• Unable to parse jar URL
• Unable to parse jar URL
• f'Invalid path or url
• Failed to build package from
• File not found.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/5f8a1adac4f819af. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/runners/portability/spark_uber_jar_job_server.py:156

  def delete(self, path, **kwargs):
    return self.request(requests.delete, path, **kwargs)

  def _get_server_spark_version(self):
    # Spark REST API doesn't seem to offer a dedicated endpoint for getting the
    # version, but it does include the version in all responses, even errors.
    return self.get('', expected_status=400)['serverSparkVersion']

  def _get_client_spark_version_from_properties(self, jar):
    """Parse Spark version from spark-version-info.properties file in the jar.
    https://github.com/apache/spark/blob/dddfeca175bdce5294debe00d4a993daef92ca60/build/spark-build-info#L30
    """
    with zipfile.ZipFile(jar, 'a', compression=zipfile.ZIP_DEFLATED) as z:
      with z.open('spark-version-info.properties') as fin:
        for line in fin.read().decode('utf-8').splitlines():
          split = list(map(lambda s: s.strip(), line.split('=')))
          if len(split) == 2 and split[0] == 'version' and split[1] != '':
            return split[1]
        raise ValueError(
            'Property "version" not found in spark-version-info.properties.')

  def _get_client_spark_version(self, jar):
    try:
      return self._get_client_spark_version_from_properties(jar)
    except Exception as e:
      _LOGGER.debug(e)
      server_version = self._get_server_spark_version()
      _LOGGER.warning(
          'Unable to parse Spark version from '
          'spark-version-info.properties. Defaulting to %s' % server_version)
      return server_version

  def _create_submission_request(self, jar, job_name):
    jar_url = "file:%s" % jar
    return {
        "action": "CreateSubmissionRequest",
        "appArgs": [],

View on GitHub (pinned to 12126d8942)