apache/flink · error · InvalidTypesException

org.apache.hadoop.io.Writable type expected.

Error message

org.apache.hadoop.io.Writable type expected.

What it means

Thrown by TypeExtractor.validateIfWritable when a WritableTypeInfo is expected (the writable type info class is assignable from the given typeInfo) but the reflected type is not a Class that implements org.apache.hadoop.io.Writable. This means a Hadoop Writable type was declared but the actual extracted type is not a Writable.

Source

Thrown at flink-core/src/main/java/org/apache/flink/api/java/typeutils/TypeExtractor.java:2497

        }
    }

    // visible for testing
    static void validateIfWritable(TypeInformation<?> typeInfo, Type type) {
        try {
            // try to load the writable type info

            Class<?> writableTypeInfoClass =
                    Class.forName(
                            HADOOP_WRITABLE_TYPEINFO_CLASS,
                            false,
                            typeInfo.getClass().getClassLoader());

            if (writableTypeInfoClass.isAssignableFrom(typeInfo.getClass())) {
                // this is actually a writable type info
                // check if the type is a writable
                if (!(type instanceof Class && isHadoopWritable((Class<?>) type))) {
                    throw new InvalidTypesException(HADOOP_WRITABLE_CLASS + " type expected.");
                }

                // check writable type contents
                Class<?> clazz = (Class<?>) type;
                if (typeInfo.getTypeClass() != clazz) {
                    throw new InvalidTypesException(
                            "Writable type '"
                                    + typeInfo.getTypeClass().getCanonicalName()
                                    + "' expected but was '"
                                    + clazz.getCanonicalName()
                                    + "'.");
                }
            }
        } catch (ClassNotFoundException e) {
            // class not present at all, so cannot be that type info
            // ignore
        }
    }

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Ensure the UDF signature uses the actual Hadoop Writable subclass (e.g., org.apache.hadoop.io.Text, not String).
2. Provide explicit TypeInformation matching the actual type via .returns().
3. If the type is genuinely not a Writable, do not use a WritableTypeInfo; use the appropriate TypeInformation instead.
4. Add the flink-hadoop-compatibility dependency and ensure the Writable class is on the classpath.

Example fix

// before
public String map(MyRecord r) { return r.getText(); }
.returns(WritableTypeInfo.of(Text.class)) // mismatch

// after
public Text map(MyRecord r) { return new Text(r.getText()); }
.returns(WritableTypeInfo.of(Text.class))

// Check if the type is a Hadoop Writable before extraction
if (type instanceof Class<?>) {
    Class<?> clazz = (Class<?>) type;
    // check implements org.apache.hadoop.io.Writable
    boolean isWritable = org.apache.hadoop.io.Writable.class.isAssignableFrom(clazz);
    if (!isWritable) {
        // provide correct TypeInformation
    }
}

static boolean isHadoopWritableType(Type t) {
    if (!(t instanceof Class<?>)) return false;
    try {
        Class<?> writable = Class.forName("org.apache.hadoop.io.Writable");
        return writable.isAssignableFrom((Class<?>) t);
    } catch (ClassNotFoundException e) {
        return false;
    }
}

try {
    validateIfWritable(typeInfo, type);
} catch (InvalidTypesException e) {
    // provide correct non-Writable TypeInformation
    ti = TypeInformation.of((Class<?>) type);
}

When it happens

Trigger: Called during validateIfWritable when writableTypeInfoClass.isAssignableFrom(typeInfo.getClass()) is true (meaning we have a WritableTypeInfo expectation) but the type is not a Class instance or isHadoopWritable((Class<?>) type) returns false. Occurs when a function declares a Hadoop Writable return/output type but the actual generic type resolves to a non-Writable class.

Common situations: Declaring a UDF output as a Hadoop Writable (e.g., Text) but actually using a String or POJO. Generic function signatures where the type parameter erases to a non-Writable class. Missing flink-hadoop-compatibility at validation time but the type info was constructed with it present (cross-module or classloader boundary).

Related errors

• Writable type '{}' expected but was '{}'.
• InputSplit must implement Writable interface.
• Cannot create Comparator for {typeClass.getCanonicalName()}.
• The given class is no subclass of {Writable.class.getName()}
• POJO type '{}' expected but was '{}'.

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/9e0de3ca447f8aac. Report an issue: GitHub.