{"record":{"id":"514a4235a151ac12","repo":"apache/beam","slug":"unable-to-create-configuration-e","errorCode":null,"errorMessage":"Unable to create configuration: ${e}","messagePattern":"Unable to create configuration: (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/hadoop-common/src/main/java/org/apache/beam/sdk/io/hadoop/SerializableConfiguration.java","lineNumber":99,"sourceCode":"    out.write(serializationCache);\n  }\n\n  @Override\n  public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException {\n    confMutated = true;\n    String className = in.readUTF();\n    try {\n      conf =\n          Class.forName(className)\n              .asSubclass(Configuration.class)\n              .getDeclaredConstructor()\n              .newInstance();\n      conf.readFields(in);\n    } catch (InstantiationException\n        | IllegalAccessException\n        | NoSuchMethodException\n        | InvocationTargetException e) {\n      throw new IOException(\"Unable to create configuration: \" + e);\n    }\n  }\n\n  /** Returns new configured {@link Job} object. */\n  public static Job newJob(@Nullable SerializableConfiguration conf) throws IOException {\n    if (conf == null) {\n      return Job.getInstance();\n    } else {\n      // Don't reading configuration from slave thread, but only from master thread.\n      Job job = Job.getInstance(new Configuration(false));\n      for (Map.Entry<String, String> entry : conf.get()) {\n        job.getConfiguration().set(entry.getKey(), entry.getValue());\n      }\n      return job;\n    }\n  }\n\n  /** Returns a new configuration instance using provided flags. */","sourceCodeStart":81,"sourceCodeEnd":117,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/hadoop-common/src/main/java/org/apache/beam/sdk/io/hadoop/SerializableConfiguration.java#L81-L117","documentation":"SerializableConfiguration.readExternal reconstructs a Hadoop Configuration from a serialized stream. If instantiating the Configuration class reflectively fails (no default constructor, inaccessible class, constructor exception), it wraps the problem in this IOException. It means the deserialized configuration object could not be created.","triggerScenarios":"Deserializing a SerializableConfiguration (e.g. from a worker state or pipeline serialization) when Configuration cannot be instantiated: wrong Hadoop version, Configuration class not on classpath, or reflective constructor failure.","commonSituations":"Running Beam Hadoop IO connectors on runners where the Hadoop classpath differs from the one used at submission time, or shaded/missing hadoop-common dependency causing class initialization errors.","solutions":["Ensure hadoop-common and its dependencies are on the worker classpath at the same version as submission time","Check the wrapped `e` in the message for the true reflective failure (InstantiationException/NoSuchMethodException etc.)","Verify no shading/relocation breaks org.apache.hadoop.conf.Configuration's no-arg constructor","Test Configuration c = new Configuration() directly on the worker to confirm the class loads"],"exampleFix":"// before (broken classpath)\nconf = new SerializableConfiguration();\n// after — ensure dependency present\n<dependency>\n  <groupId>org.apache.hadoop</groupId>\n  <artifactId>hadoop-common</artifactId>\n  <scope>provided</scope>\n</dependency>","handlingStrategy":"try-catch","validationCode":"try {\n  Configuration probe = new Configuration();\n  probe.get(\"fs.defaultFS\");\n} catch (Throwable t) {\n  throw new IllegalStateException(\"hadoop Configuration unavailable on classpath\", t);\n}","typeGuard":null,"tryCatchPattern":"try {\n  serConf.readExternal(in);\n} catch (IOException e) {\n  if (e.getMessage().startsWith(\"Unable to create configuration\")) {\n    // check hadoop-common on worker classpath\n  }\n  throw e;\n}","preventionTips":["Pin the same hadoop-common version at submit and worker time","Run a smoke test instantiating Configuration on worker nodes","Check shaded-jar relocation rules for hadoop packages"],"tags":["java","hadoop","serialization","classpath","deserialization"],"backgroundTag":"deserialization-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}