apache/flink · critical · RuntimeException

Incompatible versions of the Hadoop Compatibility classes fo

Error message

Incompatible versions of the Hadoop Compatibility classes found.

What it means

Thrown by TypeExtractor.createHadoopWritableTypeInfo when the HadoopWritableTypeInfo class is found on the classpath but its constructor cannot be invoked — the expected constructor HadoopWritableTypeInfo(Class) is missing, inaccessible, or the class cannot be instantiated. This indicates an incompatible version of flink-hadoop-compatibility.

Source

Thrown at flink-core/src/main/java/org/apache/flink/api/java/typeutils/TypeExtractor.java:2474

                    Class.forName(
                            HADOOP_WRITABLE_TYPEINFO_CLASS,
                            false,
                            Thread.currentThread().getContextClassLoader());
        } catch (ClassNotFoundException e) {
            throw new RuntimeException(
                    "Could not load the TypeInformation for the class '"
                            + HADOOP_WRITABLE_CLASS
                            + "'. You may be missing the 'flink-hadoop-compatibility' dependency.");
        }

        try {
            Constructor<?> constr = typeInfoClass.getConstructor(Class.class);

            @SuppressWarnings("unchecked")
            TypeInformation<T> typeInfo = (TypeInformation<T>) constr.newInstance(clazz);
            return typeInfo;
        } catch (NoSuchMethodException | IllegalAccessException | InstantiationException e) {
            throw new RuntimeException(
                    "Incompatible versions of the Hadoop Compatibility classes found.");
        } catch (InvocationTargetException e) {
            throw new RuntimeException(
                    "Cannot create Hadoop WritableTypeInfo.", e.getTargetException());
        }
    }

    // visible for testing
    static void validateIfWritable(TypeInformation<?> typeInfo, Type type) {
        try {
            // try to load the writable type info

            Class<?> writableTypeInfoClass =
                    Class.forName(
                            HADOOP_WRITABLE_TYPEINFO_CLASS,
                            false,
                            typeInfo.getClass().getClassLoader());

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Ensure all Flink modules use the same version (align flink-core and flink-hadoop-compatibility versions in pom.xml/gradle).
2. Run mvn dependency:tree to find and exclude conflicting transitive versions of flink-hadoop-compatibility.
3. Use the Flink Maven dependency enforcer or BOM to keep versions consistent.
4. If shading, ensure shading rules don't relocate internal Flink classes incompatibly.

Example fix

// before
<dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-core</artifactId>
  <version>1.18.0</version>
</dependency>
<dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-hadoop-compatibility_2.12</artifactId>
  <version>1.15.0</version> <!-- mismatched! -->
</dependency>

// after (aligned versions)
<dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-hadoop-compatibility_2.12</artifactId>
  <version>1.18.0</version>
</dependency>

// Verify the constructor exists and is accessible
Class<?> cls = Class.forName("org.apache.flink.hadoopcompatibility.wrappers.HadoopWritableTypeInfo");
try {
    cls.getConstructor(Class.class);
    // compatible version
} catch (NoSuchMethodException e) {
    // incompatible version; align Flink versions
}

static boolean hadoopWritableTypeInfoCompatible() {
    try {
        Class<?> cls = Class.forName(
            "org.apache.flink.hadoopcompatibility.wrappers.HadoopWritableTypeInfo");
        cls.getConstructor(Class.class);
        return true;
    } catch (Exception e) {
        return false;
    }
}

try {
    TypeInformation<?> ti = TypeExtractor.createHadoopWritableTypeInfo(MyWritable.class);
} catch (RuntimeException e) {
    if (e.getMessage().contains("Incompatible versions")) {
        // align all Flink module versions
    } else throw e;
}

When it happens

Trigger: Called during createHadoopWritableTypeInfo after successfully loading HadoopWritableTypeInfo, but typeInfoClass.getConstructor(Class.class).newInstance(clazz) throws NoSuchMethodException, IllegalAccessException, or InstantiationException. This means the loaded version of the class has a different API than expected.

Common situations: Mixing Flink versions (e.g., core from 1.18 but hadoop-compatibility from 1.15). The HadoopWritableTypeInfo class changed its constructor signature between versions. Classloader isolation issues where a different/incompatible version is loaded by a parent classloader. Shading/dependency conflicts where multiple versions are on the classpath.

Related errors

• Could not load the TypeInformation for the class '{}'. You m
• InputSplit must implement Writable interface.
• Cannot create Comparator for {typeClass.getCanonicalName()}.
• The given class is no subclass of {Writable.class.getName()}
• Cannot create Hadoop WritableTypeInfo.

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/c1debbbf7083a638. Report an issue: GitHub.