{"record":{"id":"52d8cf5779576921","repo":"mlflow/mlflow","slug":"failed-to-read-scala-version","errorCode":null,"errorMessage":"Failed to read scala version.","messagePattern":"Failed to read scala version\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"mlflow/utils/_spark_utils.py","lineNumber":76,"sourceCode":"        return os.environ[\"SPARK_SCALA_VERSION\"]\n\n    if spark := _get_active_spark_session():\n        return _get_spark_scala_version_from_spark_session(spark)\n\n    result_queue = multiprocessing.Queue()\n\n    # If we need to create a new spark local session for reading scala version,\n    # we have to create the temporal spark session in a child process,\n    # if we create the temporal spark session in current process,\n    # after terminating the temporal spark session, creating another spark session\n    # with \"spark.jars.packages\" configuration doesn't work.\n    proc = multiprocessing.Process(\n        target=_get_spark_scala_version_child_proc_target, args=(result_queue,)\n    )\n    proc.start()\n    proc.join()\n    if proc.exitcode != 0:\n        raise RuntimeError(\"Failed to read scala version.\")\n\n    return result_queue.get()\n\n\ndef _create_local_spark_session_for_loading_spark_model():\n    from pyspark.sql import SparkSession\n\n    return (\n        SparkSession.builder\n        .config(\"spark.python.worker.reuse\", \"true\")\n        # The config is a workaround for avoiding databricks delta cache issue when loading\n        # some specific model such as ALSModel.\n        .config(\"spark.databricks.io.cache.enabled\", \"false\")\n        # In Spark 3.1 and above, we need to set this conf explicitly to enable creating\n        # a SparkSession on the workers\n        .config(\"spark.executor.allowSparkContext\", \"true\")\n        # Binding \"spark.driver.host\" to 127.0.0.1 helps avoiding some local hostname\n        # related issues (e.g. https://github.com/mlflow/mlflow/issues/5733).","sourceCodeStart":58,"sourceCodeEnd":94,"githubUrl":"https://github.com/mlflow/mlflow/blob/6a27f2decc0b76eb1b54af31849784addb357dbc/mlflow/utils/_spark_utils.py#L58-L94","documentation":"MLflow determines the Scala version of the installed PySpark by launching a child process and reading the result from a queue. If the child process exits with a non-zero exit code (crashed, JVM failed to start, fork unsupported), MLflow raises this RuntimeError because it cannot read the Scala version needed for Spark model loading.","triggerScenarios":"Calling mlflow.spark.load_model (or any code path invoking _get_spark_scala_version, e.g. creating a local Spark session for loading a Spark model) when the spawned multiprocessing child fails before it can put the Scala version into result_queue.","commonSituations":"Environments where multiprocessing fork of a JVM is unreliable (macOS spawn default, Windows, containers without /proc, restricted sandboxes), broken or mismatched JAVA_HOME, PySpark installed without a working JVM, or memory limits killing the forked JVM.","solutions":["Verify Java is installed and JAVA_HOME points to a valid JDK 8/11/17 compatible with the installed PySpark","Run `python -c \"import pyspark; pyspark.SparkContext.getOrCreate()\"` to confirm Spark itself starts; fix any JVM errors reported","Try setting the multiprocessing start method, e.g. multiprocessing.set_start_method('spawn') before importing mlflow, or run in an environment where fork works","As a workaround, set the Scala version explicitly if supported by your MLflow version, or load Spark models inside a real Spark cluster job instead of locally"],"exampleFix":"// before\nimport mlflow\nmodel = mlflow.spark.load_model(\"runs:/abc/model\")  # RuntimeError: Failed to read scala version.\n// after\nimport multiprocessing\nmultiprocessing.set_start_method(\"spawn\", force=True)\nimport mlflow\nmodel = mlflow.spark.load_model(\"runs:/abc/model\")","handlingStrategy":"fallback","validationCode":"import shutil\nif not (shutil.which(\"java\") or __import__(\"os\").environ.get(\"JAVA_HOME\")):\n    raise SystemExit(\"Install a JDK and set JAVA_HOME before loading Spark models\")","typeGuard":null,"tryCatchPattern":"try:\n    model = mlflow.spark.load_model(uri)\nexcept RuntimeError as e:\n    if \"Failed to read scala version\" in str(e):\n        model = load_spark_model_in_external_session(uri)\n    else:\n        raise","preventionTips":["Pin a JDK (8/11/17) and JAVA_HOME in all environments that load Spark models","Smoke-test SparkSession creation in CI before model loading","Avoid loading Spark models in sandboxed/serverless runtimes that block multiprocessing"],"tags":["pyspark","jvm","multiprocessing","model-loading"],"backgroundTag":"jvm-startup-failure","analyzedSha":"6a27f2decc0b76eb1b54af31849784addb357dbc","analyzedAt":"2026-08-29T20:54:51.419Z","schemaVersion":2},"datasetVersion":"2026-08-29T22:17:34.462Z"}