{"record":{"id":"5f8a1adac4f819af","repo":"apache/beam","slug":"property-version-not-found-in-spark-version-info-properties","errorCode":null,"errorMessage":"Property \"version\" not found in spark-version-info.properties.","messagePattern":"Property \"version\" not found in spark-version-info\\.properties\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/runners/portability/spark_uber_jar_job_server.py","lineNumber":156,"sourceCode":"  def delete(self, path, **kwargs):\n    return self.request(requests.delete, path, **kwargs)\n\n  def _get_server_spark_version(self):\n    # Spark REST API doesn't seem to offer a dedicated endpoint for getting the\n    # version, but it does include the version in all responses, even errors.\n    return self.get('', expected_status=400)['serverSparkVersion']\n\n  def _get_client_spark_version_from_properties(self, jar):\n    \"\"\"Parse Spark version from spark-version-info.properties file in the jar.\n    https://github.com/apache/spark/blob/dddfeca175bdce5294debe00d4a993daef92ca60/build/spark-build-info#L30\n    \"\"\"\n    with zipfile.ZipFile(jar, 'a', compression=zipfile.ZIP_DEFLATED) as z:\n      with z.open('spark-version-info.properties') as fin:\n        for line in fin.read().decode('utf-8').splitlines():\n          split = list(map(lambda s: s.strip(), line.split('=')))\n          if len(split) == 2 and split[0] == 'version' and split[1] != '':\n            return split[1]\n        raise ValueError(\n            'Property \"version\" not found in spark-version-info.properties.')\n\n  def _get_client_spark_version(self, jar):\n    try:\n      return self._get_client_spark_version_from_properties(jar)\n    except Exception as e:\n      _LOGGER.debug(e)\n      server_version = self._get_server_spark_version()\n      _LOGGER.warning(\n          'Unable to parse Spark version from '\n          'spark-version-info.properties. Defaulting to %s' % server_version)\n      return server_version\n\n  def _create_submission_request(self, jar, job_name):\n    jar_url = \"file:%s\" % jar\n    return {\n        \"action\": \"CreateSubmissionRequest\",\n        \"appArgs\": [],","sourceCodeStart":138,"sourceCodeEnd":174,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/runners/portability/spark_uber_jar_job_server.py#L138-L174","documentation":"To stay compatible with the client's Spark version, Beam reads spark-version-info.properties from inside the user's uber jar looking for a 'version' entry. If the property file is missing the key (or it is blank), Beam raises ValueError because it cannot determine which Spark job server artifact to fetch.","triggerScenarios":"The uber jar supplied to SparkUberJarJobServer was built without (or with an empty) spark-version-info.properties, so the zip entry exists but no line matches key 'version' with a non-empty value; _get_client_spark_version then, after this error, falls back to its except path.","commonSituations":"Hand-assembled or repackaged uber jar that dropped the properties file contents; custom Spark builds where version metadata was stripped; jars produced by shading tools that overwrite resource files.","solutions":["Rebuild the uber jar with the standard Spark/Gradle build so spark-version-info.properties contains version=<x.y.z>.","Verify the entry: unzip -p your.jar spark-version-info.properties and confirm a non-empty version= line.","If you control packaging, ensure resource filtering/merging does not blank out the version key.","Rely on _get_client_spark_version's fallback only intentionally — fix the jar rather than depending on exception handling."],"exampleFix":"// before (broken resource)\nspark-version-info.properties: version=\n// after\nspark-version-info.properties: version=3.5.0","handlingStrategy":"validation","validationCode":"import zipfile\nwith zipfile.ZipFile(jar_path) as z:\n    props = z.read('spark-version-info.properties').decode()\n    version = dict(\n        line.split('=', 1) for line in props.splitlines() if '=' in line\n    ).get('version', '').strip()\nassert version, f'{jar_path} lacks a non-empty version in spark-version-info.properties'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Use the official Spark/Gradle build to produce uber jars","Never strip resource files when repackaging","Verify jar metadata as part of artifact intake"],"tags":["apache-beam","spark","jar","packaging"],"backgroundTag":"missing-config-key","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}