{"record":{"id":"7f89cdcea2f255a1","repo":"apache/flink","slug":"unable-to-instantiate-the-hadoop-input-format-7f89cd","errorCode":null,"errorMessage":"Unable to instantiate the hadoop input format","messagePattern":"Unable to instantiate the hadoop input format","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"critical","filePath":"flink-connectors/flink-hadoop-compatibility/src/main/java/org/apache/flink/api/java/hadoop/mapreduce/HadoopInputFormatBase.java","lineNumber":323,"sourceCode":"\n        org.apache.hadoop.conf.Configuration configuration =\n                new org.apache.hadoop.conf.Configuration();\n        configuration.readFields(in);\n\n        if (this.configuration == null) {\n            this.configuration = configuration;\n        }\n\n        try {\n            this.mapreduceInputFormat =\n                    (org.apache.hadoop.mapreduce.InputFormat<K, V>)\n                            Class.forName(\n                                            hadoopInputFormatClassName,\n                                            true,\n                                            Thread.currentThread().getContextClassLoader())\n                                    .newInstance();\n        } catch (Exception e) {\n            throw new RuntimeException(\"Unable to instantiate the hadoop input format\", e);\n        }\n        try {\n            this.keyClass =\n                    (Class<K>)\n                            Class.forName(\n                                    keyClassName,\n                                    true,\n                                    Thread.currentThread().getContextClassLoader());\n        } catch (Exception e) {\n            throw new RuntimeException(\"Unable to find key class.\", e);\n        }\n        try {\n            this.valueClass =\n                    (Class<V>)\n                            Class.forName(\n                                    valueClassName,\n                                    true,\n                                    Thread.currentThread().getContextClassLoader());","sourceCodeStart":305,"sourceCodeEnd":341,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-connectors/flink-hadoop-compatibility/src/main/java/org/apache/flink/api/java/hadoop/mapreduce/HadoopInputFormatBase.java#L305-L341","documentation":"Thrown during Java deserialization of HadoopInputFormatBase to a TaskManager. readObject() reconstructs the wrapped InputFormat by name using Class.forName(className, true, contextClassLoader).newInstance(). If the class is missing, abstract, non-instantiable, or lacks an accessible no-arg constructor, the RuntimeException aborts deserialization and the task cannot start.","triggerScenarios":"Triggered on the TaskManager when the serialized HadoopInputFormat is rebuilt and Class.forName(hadoopInputFormatClassName, true, contextClassLoader).newInstance() fails — e.g. the InputFormat class is not in the user jar shipped to the cluster, has no public no-arg constructor, is abstract/interface, or the context classloader cannot resolve it.","commonSituations":"The Hadoop InputFormat implementation (e.g. a custom or connector InputFormat) is not included in the submitted user-code jar; class relocated/shaded differently than at build time; Hadoop version on cluster differs and a class was removed/renamed; SecurityManager or access rules block reflective instantiation.","solutions":["Package the concrete Hadoop InputFormat class (and its dependencies) into the user-code jar submitted with the job.","Confirm the InputFormat class has a public no-arg constructor required by newInstance().","Verify the cluster's Hadoop version exposes the same class name as at build time (no relocation/removal).","If shading, ensure the className written at serialization time matches the resolvable name on the TaskManager."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// Before submitting, confirm the InputFormat class is loadable + instantiable on the runtime classpath\nString className = myInputFormat.getClass().getName();\ntry {\n    Class<?> c = Class.forName(className, true, Thread.currentThread().getContextClassLoader());\n    if (java.lang.reflect.Modifier.isAbstract(c.getModifiers()) || c.isInterface())\n        throw new IllegalStateException(className + \" is abstract/interface — cannot instantiate\");\n    c.getDeclaredConstructor().newInstance(); // mirrors readObject's newInstance()\n} catch (Exception e) {\n    throw new IllegalStateException(\"InputFormat class \" + className\n        + \" cannot be instantiated on the cluster (missing jar / no public no-arg ctor)\", e);\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Build a fat user-code jar containing the InputFormat and all dependencies.","Ensure the InputFormat has a public no-arg constructor.","Keep class names consistent under any shading.","Match Hadoop versions between build and cluster."],"tags":["hadoop","mapreduce","serialization","classloader","input-format"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}