apache/flink · error · RuntimeException

Unable to instantiate the hadoop output format

Error message

Unable to instantiate the hadoop output format

What it means

Thrown by HadoopOutputFormatBase.readFields (configure phase) when Class.forName(hadoopOutputFormatName).newInstance() fails. The Hadoop OutputFormat class name is read from the serialized JobConf; if the class is absent from the classpath, is abstract, lacks a public no-arg constructor, or throws during construction, this RuntimeException wraps the root cause.

Source

Thrown at flink-connectors/flink-hadoop-compatibility/src/main/java/org/apache/flink/api/java/hadoop/mapred/HadoopOutputFormatBase.java:220

    @SuppressWarnings("unchecked")
    private void readObject(ObjectInputStream in) throws IOException, ClassNotFoundException {
        super.read(in);
        String hadoopOutputFormatName = in.readUTF();
        if (jobConf == null) {
            jobConf = new JobConf();
        }
        jobConf.readFields(in);
        try {
            this.mapredOutputFormat =
                    (org.apache.hadoop.mapred.OutputFormat<K, V>)
                            Class.forName(
                                            hadoopOutputFormatName,
                                            true,
                                            Thread.currentThread().getContextClassLoader())
                                    .newInstance();
        } catch (Exception e) {
            throw new RuntimeException("Unable to instantiate the hadoop output format", e);
        }
        ReflectionUtils.setConf(mapredOutputFormat, jobConf);

        jobConf.getCredentials().addAll(this.credentials);
        Credentials currentUserCreds = getCredentialsFromUGI(UserGroupInformation.getCurrentUser());
        if (currentUserCreds != null) {
            jobConf.getCredentials().addAll(currentUserCreds);
        }
    }
}

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Add the JAR containing the Hadoop OutputFormat class to the Flink job classpath.
2. Verify the class name is a concrete, instantiable class with a public no-arg constructor.
3. Inspect getCause() for ClassNotFoundException, InstantiationException, or IllegalAccessException details.
4. Ensure Hadoop version compatibility between the OutputFormat JAR and the Flink Hadoop compatibility layer.

// Validate the OutputFormat class is loadable and instantiable before submitting
String className = hadoopOutputFormatClass.getName();
try {
    Class<?> clazz = Class.forName(className);
    if (clazz.isInterface() || java.lang.reflect.Modifier.isAbstract(clazz.getModifiers())) {
        throw new IllegalArgumentException("OutputFormat class must be concrete: " + className);
    }
    clazz.getDeclaredConstructor().newInstance();
} catch (Exception e) {
    throw new IllegalArgumentException("Cannot instantiate OutputFormat: " + className, e);
}

When it happens

Trigger: Deserializing a HadoopOutputFormat whose class name is not resolvable on the task classpath, or whose class cannot be instantiated.

Common situations: Missing Hadoop OutputFormat JAR in the Flink job classpath; class renamed or removed between Hadoop versions; classloader isolation hiding the OutputFormat class; constructor requiring configuration that is not available.

Related errors

• Unable to instantiate the hadoop input format
• Unable to find key class.
• Unable to find value class.
• The program's entry point class '{className}' could not be l
• Task id too large.

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/d3d52750e52a1639. Report an issue: GitHub.