{"record":{"id":"398dc2daf47713b0","repo":"apache/beam","slug":"the-kafka-streams-runner-is-experimental-and-is-not-part-of","errorCode":null,"errorMessage":"The Kafka Streams runner is experimental and is not part of any Apache Beam release, so there is no published job server jar to download. Build one from a Beam source tree with\n  ./gradlew -Pwith-kafka-streams-runner :runners:kafka-streams:job-server:shadowJar\nand pass it as --kafka_streams_job_server_jar, or point that option at a jar you already have. The runner is opt-in at build time, so the -Pwith-kafka-streams-runner flag is required; without it the runner is not part of the build at all.","messagePattern":"The Kafka Streams runner is experimental and is not part of any Apache Beam release, so there is no published job server jar to download\\. Build one from a Beam source tree with\n  \\./gradlew -Pwith-kafka-streams-runner :runners:kafka-streams:job-server:shadowJar\nand pass it as --kafka_streams_job_server_jar, or point that option at a jar you already have\\. The runner is opt-in at build time, so the -Pwith-kafka-streams-runner flag is required; without it the runner is not part of the build at all\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/runners/portability/kafka_streams_runner.py","lineNumber":115,"sourceCode":"        url = urllib.parse.urlparse(self._jar)\n        if not url.scheme:\n          raise ValueError(\n              'Unable to parse jar URL \"%s\". If using a full URL, make sure '\n              'the scheme is specified. If using a local file path, make sure '\n              'the file exists; you may have to first build the job server '\n              'using `./gradlew -Pwith-kafka-streams-runner '\n              'runners:kafka-streams:job-server:shadowJar`.' % self._jar)\n      return self._jar\n\n    # No jar was given, so look for one built from this source tree. The base\n    # class would fall back to Maven Central, but the job server is not\n    # published for any Beam release, so that download always fails and says\n    # nothing useful about why.\n    local_jar = subprocess_server.JavaJarServer.path_to_dev_beam_jar(\n        ':runners:kafka-streams:job-server:shadowJar')\n    if os.path.exists(local_jar):\n      return local_jar\n    raise RuntimeError(\n        'The Kafka Streams runner is experimental and is not part of any '\n        'Apache Beam release, so there is no published job server jar to '\n        'download. Build one from a Beam source tree with\\n'\n        '  ./gradlew -Pwith-kafka-streams-runner '\n        ':runners:kafka-streams:job-server:shadowJar\\n'\n        'and pass it as --kafka_streams_job_server_jar, or point that option '\n        'at a jar you already have. The runner is opt-in at build time, so '\n        'the -Pwith-kafka-streams-runner flag is required; without it the '\n        'runner is not part of the build at all.')\n\n  def java_arguments(\n      self, job_port, artifact_port, expansion_port, artifacts_dir):\n    return [\n        '--artifacts-dir',\n        artifacts_dir,\n        '--job-port',\n        job_port,\n        '--artifact-port',","sourceCodeStart":97,"sourceCodeEnd":133,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/runners/portability/kafka_streams_runner.py#L97-L133","documentation":"The Kafka Streams runner is experimental and not part of any released Beam artifact, so the normal 'download the Beam job server jar' path always fails. When no locally built shadowJar exists either, path_to_jar raises this RuntimeError explaining that the jar must be built from source with the opt-in Gradle flag.","triggerScenarios":"Running a Kafka Streams pipeline without --kafka_streams_job_server_jar, where the dev jar path produced by JavaJarServer.path_to_dev_beam_jar(':runners:kafka-streams:job-server:shadowJar') does not exist on disk.","commonSituations":"Using pip-installed Apache Beam (no Beam source tree, so no dev jar); building Beam without -Pwith-kafka-streams-runner so the jar was never produced; assuming a published job server exists for releases.","solutions":["Build from a Beam source tree: ./gradlew -Pwith-kafka-streams-runner :runners:kafka-streams:job-server:shadowJar.","Pass the built jar explicitly with --kafka_streams_job_server_jar=/path/to/shadowJar.","Alternatively point the option at any Kafka Streams job server jar you already have."],"exampleFix":"// before\nbeam run --runner=KafkaStreamsRunner ...\n// after\n./gradlew -Pwith-kafka-streams-runner :runners:kafka-streams:job-server:shadowJar\nbeam run ... --kafka_streams_job_server_jar=runners/kafka-streams/job-server/build/libs/beam-runners-kafka-streams-job-server-<ver>-SNAPSHOT.jar","handlingStrategy":"validation","validationCode":"local = subprocess_server.JavaJarServer.path_to_dev_beam_jar(\n    ':runners:kafka-streams:job-server:shadowJar')\nif not (options.kafka_streams_job_server_jar or os.path.exists(local)):\n    raise SystemExit('build the kafka-streams job server jar first')","typeGuard":null,"tryCatchPattern":"try:\n    jar = runner.path_to_jar()\nexcept RuntimeError:\n    subprocess.run(['./gradlew', '-Pwith-kafka-streams-runner',\n                    ':runners:kafka-streams:job-server:shadowJar'], check=True)","preventionTips":["Remember the runner requires a source build with -Pwith-kafka-streams-runner","Pass --kafka_streams_job_server_jar explicitly in CI","Cache the built jar"],"tags":["kafka","build","missing-dependency","experimental"],"backgroundTag":"missing-dependency","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}