apache/flink · error · IOException

Could not deserialize Hadoop Configuration, the serialized…

Error message

Could not deserialize Hadoop Configuration, the serialized and de-serialized don't match.

What it means

Wraps an exception thrown while rebuilding the Hadoop Configuration from its serialized bytes during Java deserialization; the chained cause (typically a stream corruption or classloader issue) explains the failure. It signals that the custom writeObject/readObject round-trip did not reproduce the config.

Solutions

  1. Address the cause reported by the error message: Could not deserialize Hadoop Configuration, the serialized and de-serialized don't match.
  2. Verify the inputs, configuration values, and classpath/dependency setup related to this operation, then retry.

Example fix

Correct the condition described ("Could not deserialize Hadoop Configuration, the serialized and de-serialized don't match.") and rerun the job or command.
Defensive patterns

Strategy: try-catch

When it happens

Trigger: Triggered at runtime when the operation fails because: Could not deserialize Hadoop Configuration, the serialized and de-serialized don't match.

Common situations: Commonly caused by misconfiguration, missing dependencies or files, unsupported types or operations, or invalid user input leading to: Could not deserialize Hadoop Configuration, the serialized and de-serialized don't match.

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/3dc9f83ca83da9fa. Report an issue: GitHub.

Appendix: source

Thrown at flink-formats/flink-orc/src/main/java/org/apache/flink/orc/util/SerializableHadoopConfigWrapper.java:73

        // corrupts the serialization stream
        final DataOutputSerializer ser = new DataOutputSerializer(256);
        hadoopConfig.write(ser);
        out.writeInt(ser.length());
        out.write(ser.getSharedBuffer(), 0, ser.length());
    }

    private void readObject(ObjectInputStream in) throws IOException, ClassNotFoundException {
        in.defaultReadObject();

        final byte[] data = new byte[in.readInt()];
        in.readFully(data);
        final DataInputDeserializer deser = new DataInputDeserializer(data);
        this.hadoopConfig = new Configuration();

        try {
            this.hadoopConfig.readFields(deser);
        } catch (IOException e) {
            throw new IOException(
                    "Could not deserialize Hadoop Configuration, the serialized and de-serialized don't match.",
                    e);
        }
    }
}

View on GitHub (pinned to 2f3c205e92)