{"record":{"id":"2c691abd4f440a83","repo":"prestodb/presto","slug":"sparkconf-must-be-initialized-with-prestosparkconf","errorCode":null,"errorMessage":"SparkConf must be initialized with PrestoSparkConfInitializer.initialize before creating SparkContext","messagePattern":"SparkConf must be initialized with PrestoSparkConfInitializer\\.initialize before creating SparkContext","errorType":"exception","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"presto-spark-classloader-interface/src/main/java/com/facebook/presto/spark/classloader_interface/PrestoSparkConfInitializer.java","lineNumber":52,"sourceCode":"    private static void registerKryoClasses(SparkConf sparkConf)\n    {\n        sparkConf.registerKryoClasses(new Class[] {\n                MutablePartitionId.class,\n                PrestoSparkMutableRow.class,\n                PrestoSparkSerializedPage.class,\n                SerializedPrestoSparkTaskDescriptor.class,\n                SerializedTaskInfo.class,\n                PrestoSparkShuffleStats.class,\n                PrestoSparkStorageHandle.class,\n                SerializedPrestoSparkTaskSource.class\n        });\n    }\n\n    public static void checkInitialized(SparkContext sparkContext)\n    {\n        SparkConf sparkConf = sparkContext.getConf();\n        if (sparkConf.get(INITIALIZED_MARKER, null) == null) {\n            throw new IllegalArgumentException(\"SparkConf must be initialized with PrestoSparkConfInitializer.initialize before creating SparkContext\");\n        }\n    }\n}\n","sourceCodeStart":34,"sourceCodeEnd":56,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-spark-classloader-interface/src/main/java/com/facebook/presto/spark/classloader_interface/PrestoSparkConfInitializer.java#L34-L56","documentation":"Presto requires SparkConf objects to have gone through PrestoSparkConfInitializer.initialize (which registers Kryo classes and sets an INITIALIZED_MARKER). checkInitialized validates a SparkContext's conf after creation and throws IllegalArgumentException if the marker is absent, meaning SparkContext was created with an uninitialized conf.","triggerScenarios":"Creating a SparkContext/SparkSession with a raw new SparkConf() (or a conf loaded from a system properties default) that never passed PrestoSparkConfInitializer.initialize.","commonSituations":"Standard Spark application bootstrap not adapted for Presto on Spark; rebuilding the conf in tests or tools bypassing the initializer; Spark creating a default conf internally.","solutions":["Call PrestoSparkConfInitializer.initialize(sparkConf) before constructing the SparkContext","Ensure the same conf object that was initialized is passed to the SparkContext constructor","In tests/tools, replicate the Presto launch path instead of new SparkConf()"],"exampleFix":"// before\nSparkConf conf = new SparkConf().setAppName(\"presto\");\nSparkContext ctx = new SparkContext(conf);\n// after\nSparkConf conf = new SparkConf().setAppName(\"presto\");\nPrestoSparkConfInitializer.initialize(conf);\nSparkContext ctx = new SparkContext(conf);","handlingStrategy":"validation","validationCode":"Preconditions.checkState(sparkConf.get(\"__presto_conf_initialized__\", null) != null, \"call PrestoSparkConfInitializer.initialize first\");","typeGuard":null,"tryCatchPattern":"try { PrestoSparkConfInitializer.checkInitialized(sparkContext); } catch (IllegalArgumentException e) { log.error(\"SparkContext created without Presto init; recreate with initialized conf\"); throw e; }","preventionTips":["Route all SparkConf creation through PrestoSparkConfInitializer.initialize","Avoid new SparkConf() defaults in Presto launch paths and tests","Run checkInitialized early in application startup to fail fast"],"tags":["java","spark","configuration","initialization"],"backgroundTag":"missing-initialization","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}