prestodb/presto · error · IllegalArgumentException

SparkConf is already initialized

Error message

SparkConf is already initialized

What it means

PrestoSparkConfInitializer.initialize registers required Kryo classes and stamps an INITIALIZED_MARKER into the SparkConf. If the marker is already present, the conf was initialized more than once, and initialize throws IllegalArgumentException to prevent duplicate Kryo registrations.

Source

Thrown at presto-spark-classloader-interface/src/main/java/com/facebook/presto/spark/classloader_interface/PrestoSparkConfInitializer.java:28

 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package com.facebook.presto.spark.classloader_interface;

import org.apache.spark.SparkConf;
import org.apache.spark.SparkContext;

public class PrestoSparkConfInitializer
{
    private static final String INITIALIZED_MARKER = "presto.spark.conf.initialized";

    private PrestoSparkConfInitializer() {}

    public static void initialize(SparkConf sparkConf)
    {
        if (sparkConf.get(INITIALIZED_MARKER, null) != null) {
            throw new IllegalArgumentException("SparkConf is already initialized");
        }
        registerKryoClasses(sparkConf);
        sparkConf.set(INITIALIZED_MARKER, "true");
    }

    private static void registerKryoClasses(SparkConf sparkConf)
    {
        sparkConf.registerKryoClasses(new Class[] {
                MutablePartitionId.class,
                PrestoSparkMutableRow.class,
                PrestoSparkSerializedPage.class,
                SerializedPrestoSparkTaskDescriptor.class,
                SerializedTaskInfo.class,
                PrestoSparkShuffleStats.class,
                PrestoSparkStorageHandle.class,
                SerializedPrestoSparkTaskSource.class
        });
    }

View on GitHub (pinned to 55bb57d202)

Solutions

1. Call initialize exactly once per SparkConf before creating the SparkContext
2. Create a fresh SparkConf instead of re-initializing an existing one
3. Guard with a check of sparkConf.get(INITIALIZED_MARKER) before calling initialize

Example fix

// before
PrestoSparkConfInitializer.initialize(sparkConf);
PrestoSparkConfInitializer.initialize(sparkConf);
// after
PrestoSparkConfInitializer.initialize(sparkConf); // once only, then reuse

if (sparkConf.get("__presto_conf_initialized__", null) == null) { PrestoSparkConfInitializer.initialize(sparkConf); }

try { PrestoSparkConfInitializer.initialize(sparkConf); } catch (IllegalArgumentException e) { /* already initialized: safe to proceed */ }

When it happens

Trigger: Calling PrestoSparkConfInitializer.initialize on the same SparkConf instance twice, or on a conf that was already stamped (e.g. shared between driver tasks or a retry path).

Common situations: Application bootstrap code that re-runs initialization after a failed launch; copy/reuse of a SparkConf across multiple SparkContext creations; duplicate plugin/launcher invocation.

Related errors

• SparkConf must be initialized with PrestoSparkConfInitialize
• UNSUPPORTED_STORAGE_TYPE
• File doesn't exist %s
• INVALID_RETRY_EXECUTION_STRATEGY
• ACCUMULO_TABLE_EXISTS

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/1363093235dd9d19. Report an issue: GitHub.