apache/flink · error · RuntimeException

Unable to instantiate the hadoop output format

Error message

Unable to instantiate the hadoop output format

What it means

Thrown by HadoopOutputFormatBase.readFields (configure phase) when Class.forName(hadoopOutputFormatName).newInstance() fails. The Hadoop OutputFormat class name is read from the serialized JobConf; if the class is absent from the classpath, is abstract, lacks a public no-arg constructor, or throws during construction, this RuntimeException wraps the root cause.

Solutions

1. Add the JAR containing the Hadoop OutputFormat class to the Flink job classpath.
2. Verify the class name is a concrete, instantiable class with a public no-arg constructor.
3. Inspect getCause() for ClassNotFoundException, InstantiationException, or IllegalAccessException details.
4. Ensure Hadoop version compatibility between the OutputFormat JAR and the Flink Hadoop compatibility layer.

// Validate the OutputFormat class is loadable and instantiable before submitting
String className = hadoopOutputFormatClass.getName();
try {
    Class<?> clazz = Class.forName(className);
    if (clazz.isInterface() || java.lang.reflect.Modifier.isAbstract(clazz.getModifiers())) {
        throw new IllegalArgumentException("OutputFormat class must be concrete: " + className);
    }
    clazz.getDeclaredConstructor().newInstance();
} catch (Exception e) {
    throw new IllegalArgumentException("Cannot instantiate OutputFormat: " + className, e);
}

When it happens

Trigger: Deserializing a HadoopOutputFormat whose class name is not resolvable on the task classpath, or whose class cannot be instantiated.

Common situations: Missing Hadoop OutputFormat JAR in the Flink job classpath; class renamed or removed between Hadoop versions; classloader isolation hiding the OutputFormat class; constructor requiring configuration that is not available.

Related errors

• Unable to find key class.
• Unable to find value class.
• Unable to instantiate the hadoop input format
• Could not load class for serialization config
• Task id too large.

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/d3d52750e52a1639. Report an issue: GitHub.

Appendix: source

Thrown at flink-connectors/flink-hadoop-compatibility/src/main/java/org/apache/flink/api/java/hadoop/mapred/HadoopOutputFormatBase.java:220

    @SuppressWarnings("unchecked")
    private void readObject(ObjectInputStream in) throws IOException, ClassNotFoundException {
        super.read(in);
        String hadoopOutputFormatName = in.readUTF();
        if (jobConf == null) {
            jobConf = new JobConf();
        }
        jobConf.readFields(in);
        try {
            this.mapredOutputFormat =
                    (org.apache.hadoop.mapred.OutputFormat<K, V>)
                            Class.forName(
                                            hadoopOutputFormatName,
                                            true,
                                            Thread.currentThread().getContextClassLoader())
                                    .newInstance();
        } catch (Exception e) {
            throw new RuntimeException("Unable to instantiate the hadoop output format", e);
        }
        ReflectionUtils.setConf(mapredOutputFormat, jobConf);

        jobConf.getCredentials().addAll(this.credentials);
        Credentials currentUserCreds = getCredentialsFromUGI(UserGroupInformation.getCurrentUser());
        if (currentUserCreds != null) {
            jobConf.getCredentials().addAll(currentUserCreds);
        }
    }
}

View on GitHub (pinned to 2f3c205e92)