{"record":{"id":"44fe058f3dbdb108","repo":"apache/hadoop","slug":"failed-to-instantiate-custom-class-name","errorCode":null,"errorMessage":"Failed to instantiate custom class \" + name + \" \" + e","messagePattern":"Failed to instantiate custom class \" \\+ name \\+ \" \" \\+ e","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/streams/StreamIntegration.java","lineNumber":187,"sourceCode":"   * @throws RuntimeException any binding/loading/instantiation problem\n   */\n  static ObjectInputStreamFactory loadCustomFactory(Configuration conf) {\n\n    // make sure the classname option is actually set\n    final String name = conf.getTrimmed(INPUT_STREAM_CUSTOM_FACTORY, \"\");\n    checkArgument(!isEmpty(name), E_EMPTY_CUSTOM_CLASSNAME);\n\n    final Class<? extends ObjectInputStreamFactory> factoryClass =\n        conf.getClass(INPUT_STREAM_CUSTOM_FACTORY,\n            null,\n            ObjectInputStreamFactory.class);\n\n    try {\n      final Constructor<? extends ObjectInputStreamFactory> ctor =\n          factoryClass.getConstructor();\n      return ctor.newInstance();\n    } catch (Exception e) {\n      throw new RuntimeException(\"Failed to instantiate custom class \"\n          + name + \" \" + e, e);\n    }\n  }\n\n  /**\n   * Populates the configurations related to vectored IO operations.\n   * The context is still mutable at this point.\n   * @param conf configuration object.\n   * @return VectoredIOContext.\n   */\n  public static VectoredIOContext populateVectoredIOContext(Configuration conf) {\n    final int minSeekVectored = (int) longBytesOption(conf, AWS_S3_VECTOR_READS_MIN_SEEK_SIZE,\n        DEFAULT_AWS_S3_VECTOR_READS_MIN_SEEK_SIZE, 0);\n    final int maxReadSizeVectored =\n        (int) longBytesOption(conf, AWS_S3_VECTOR_READS_MAX_MERGED_READ_SIZE,\n            DEFAULT_AWS_S3_VECTOR_READS_MAX_MERGED_READ_SIZE, 0);\n    final int vectoredActiveRangeReads = intOption(conf,\n        AWS_S3_VECTOR_ACTIVE_RANGE_READS, DEFAULT_AWS_S3_VECTOR_ACTIVE_RANGE_READS, 1);","sourceCodeStart":169,"sourceCodeEnd":205,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/streams/StreamIntegration.java#L169-L205","documentation":"When fs.s3a.input.stream.type=Custom, StreamIntegration.loadCustomFactory() loads the class named by fs.s3a.input.stream.custom.factory and reflectively invokes its no-arg constructor. Any failure - property unset (factoryClass is null -> NPE in getConstructor()), class missing from classpath, class not implementing ObjectInputStreamFactory, missing public no-arg constructor, or constructor throwing - is wrapped in RuntimeException(\"Failed to instantiate custom class <name> <cause>\").","triggerScenarios":"Setting fs.s3a.input.stream.type=Custom without fs.s3a.input.stream.custom.factory; custom factory jar not on the classpath of the JVM opening the filesystem (client, NM container, etc.); class lacking a public no-arg constructor; constructor throwing (e.g. reading config that is absent); class not assignable to ObjectInputStreamFactory so conf.getClass returns null.","commonSituations":"Teams shipping custom S3A input streams forgetting the jar in the cluster's share directory or the job's classpath; refactoring away the no-arg constructor; typos in the class name; version skew where the factory compiled against an older hadoop-aws API no longer implements the current ObjectInputStreamFactory interface.","solutions":["Set fs.s3a.input.stream.custom.factory to the fully-qualified class name whenever fs.s3a.input.stream.type is Custom.","Give the factory a public no-arg constructor and make it implement org.apache.hadoop.fs.s3a.impl.streams.ObjectInputStreamFactory.","Ship the factory jar on every JVM that opens the s3a filesystem (client classpath, YARN containers, or fs.s3a.aws.sdk classpath-style mechanisms).","Test instantiation standalone: Class.forName(name).getConstructor().newInstance() before shipping config."],"exampleFix":"// before\n<property><name>fs.s3a.input.stream.type</name><value>Custom</value></property>\n<!-- factory property missing -> NPE wrapped in RuntimeException -->\n\n// after\n<property><name>fs.s3a.input.stream.type</name><value>Custom</value></property>\n<property><name>fs.s3a.input.stream.custom.factory</name>\n  <value>com.example.MyObjectInputStreamFactory</value></property>\n\n// required shape of the class:\npublic class MyObjectInputStreamFactory implements ObjectInputStreamFactory {\n  public MyObjectInputStreamFactory() { }\n  // ... interface methods\n}","handlingStrategy":"validation","validationCode":"String name = conf.getTrimmed(\"fs.s3a.input.stream.custom.factory\", \"\");\nif (\"Custom\".equalsIgnoreCase(conf.getTrimmed(\"fs.s3a.input.stream.type\", \"\"))) {\n  if (name.isEmpty()) throw new IllegalArgumentException(\"fs.s3a.input.stream.custom.factory is required\");\n  Class<?> c = Class.forName(name);                       // ClassNotFoundException if jar missing\n  if (!org.apache.hadoop.fs.s3a.impl.streams.ObjectInputStreamFactory.class.isAssignableFrom(c))\n    throw new IllegalArgumentException(name + \" does not implement ObjectInputStreamFactory\");\n  c.getConstructor();                                      // NoSuchMethodException if no no-arg ctor\n}","typeGuard":null,"tryCatchPattern":"try {\n  FileSystem fs = path.getFileSystem(conf);\n} catch (RuntimeException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"Failed to instantiate custom class\")) {\n    // report classpath/jar/constructor issue; do not retry with same config\n  } else { throw e; }\n}","preventionTips":["Smoke-test Class.forName(name).getConstructor().newInstance() on every node type that opens s3a://.","Keep a public no-arg constructor on custom factories.","Recompile custom factories against the hadoop-aws version you deploy; the interface evolves."],"tags":["hadoop-aws","configuration","reflection","classloader","custom-stream"],"backgroundTag":"class-instantiation-failure","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}