apache/flink · critical · RuntimeException

Could not load the TypeInformation for the class '{}'. You m

Error message

Could not load the TypeInformation for the class '{}'. You may be missing the 'flink-hadoop-compatibility' dependency.

What it means

Thrown by TypeExtractor.createHadoopWritableTypeInfo when trying to load the HadoopWritableTypeInfo class via Class.forName but the class is not found on the classpath. This means the flink-hadoop-compatibility module (which provides Writable type support) is not available as a dependency.

Source

Thrown at flink-core/src/main/java/org/apache/flink/api/java/typeutils/TypeExtractor.java:2461

        Class<?> superclass = clazz.getSuperclass();
        return superclass != null
                && alreadySeen.add(superclass)
                && hasHadoopWritableInterface(superclass, alreadySeen);
    }

    // visible for testing
    public static <T> TypeInformation<T> createHadoopWritableTypeInfo(Class<T> clazz) {
        checkNotNull(clazz);

        Class<?> typeInfoClass;
        try {
            typeInfoClass =
                    Class.forName(
                            HADOOP_WRITABLE_TYPEINFO_CLASS,
                            false,
                            Thread.currentThread().getContextClassLoader());
        } catch (ClassNotFoundException e) {
            throw new RuntimeException(
                    "Could not load the TypeInformation for the class '"
                            + HADOOP_WRITABLE_CLASS
                            + "'. You may be missing the 'flink-hadoop-compatibility' dependency.");
        }

        try {
            Constructor<?> constr = typeInfoClass.getConstructor(Class.class);

            @SuppressWarnings("unchecked")
            TypeInformation<T> typeInfo = (TypeInformation<T>) constr.newInstance(clazz);
            return typeInfo;
        } catch (NoSuchMethodException | IllegalAccessException | InstantiationException e) {
            throw new RuntimeException(
                    "Incompatible versions of the Hadoop Compatibility classes found.");
        } catch (InvocationTargetException e) {
            throw new RuntimeException(
                    "Cannot create Hadoop WritableTypeInfo.", e.getTargetException());
        }

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Add the flink-hadoop-compatibility dependency to your project (Maven/Gradle).
2. Ensure the dependency is shaded/bundled or available on the Flink cluster's lib directory.
3. Verify the Flink version matches between your project and the cluster to avoid class relocation issues.
4. If you don't need Hadoop Writable support, avoid using Writable types and convert to POJOs or Flink-native types.

Example fix

// before (missing dependency)
// pom.xml lacks flink-hadoop-compatibility

// after
// pom.xml
<dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-hadoop-compatibility_2.12</artifactId>
  <version>${flink.version}</version>
</dependency>

// Check if HadoopWritableTypeInfo is on the classpath before using Writable types
try {
    Class.forName("org.apache.flink.hadoopcompatibility.wrappers.HadoopWritableTypeInfo",
        false, Thread.currentThread().getContextClassLoader());
    // dependency present
} catch (ClassNotFoundException e) {
    // flink-hadoop-compatibility is missing; add it or avoid Writable types
}

static boolean hadoopCompatibilityAvailable() {
    try {
        Class.forName("org.apache.flink.hadoopcompatibility.wrappers.HadoopWritableTypeInfo",
            false, Thread.currentThread().getContextClassLoader());
        return true;
    } catch (ClassNotFoundException e) {
        return false;
    }
}

try {
    TypeInformation<?> ti = TypeExtractor.createHadoopWritableTypeInfo(MyWritable.class);
} catch (RuntimeException e) {
    if (e.getMessage().contains("flink-hadoop-compatibility")) {
        // add dependency or switch to non-Writable type
        ti = TypeInformation.of(MyPojo.class);
    } else throw e;
}

When it happens

Trigger: Called during createHadoopWritableTypeInfo when a Hadoop Writable type is detected but the HADOOP_WRITABLE_TYPEINFO_CLASS constant (referencing org.apache.flink.hadoopcompatibility.wrappers.HadoopWritableTypeInfo) cannot be loaded via Class.forName with the context class loader. This happens when the flink-hadoop-compatibility JAR is missing from the classpath.

Common situations: Using Hadoop Writable types (e.g., Text, IntWritable, LongWritable) in Flink jobs without including the flink-hadoop-compatibility dependency. Running in environments where the classpath does not include all Flink modules (e.g., thin JARs, minimal Flink distributions). Version mismatches where the dependency exists but the class was moved or renamed.

Related errors

• Incompatible versions of the Hadoop Compatibility classes fo
• InputSplit must implement Writable interface.
• Cannot create Comparator for {typeClass.getCanonicalName()}.
• The given class is no subclass of {Writable.class.getName()}
• Cannot create Hadoop WritableTypeInfo.

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/1fe96edbd83fe113. Report an issue: GitHub.