apache/beam · critical · IOException

Unable to create InputFormat object:

Error message

Unable to create InputFormat object: 

What it means

HadoopFormatIO reflectively instantiates the configured InputFormat class (via the job configuration). If reflection fails — class missing, instantiation error, illegal access, wrong constructor, or constructor threw — it wraps the cause in this IOException.

Solutions

1. Confirm the InputFormat class name in the configuration is correct and fully qualified
2. Ensure the Hadoop InputFormat and its dependencies are bundled in the classpath (shadowJar / job staging)
3. Verify the class is concrete and has a public no-arg constructor
4. Check for Hadoop version conflicts (mvn dependency:tree) and align hadoop-client versions

Example fix

// before
conf.set("mapreduce.job.inputformat.class", "com.example.MyIF");
// after
conf.set("mapreduce.job.inputformat.class", "com.example.MyInputFormat"); // verify class is on classpath

try {
  Class<?> cls = Class.forName(conf.get("mapreduce.job.inputformat.class"));
  cls.getDeclaredConstructor().newInstance();
} catch (ReflectiveOperationException e) {
  throw new IllegalStateException("InputFormat not loadable/instantiable: " + e, e);
}

static boolean isValidInputFormatClass(String name) {
  try {
    Class<?> c = Class.forName(name);
    return InputFormat.class.isAssignableFrom(c)
        && !c.isInterface()
        && !java.lang.reflect.Modifier.isAbstract(c.getModifiers())
        && java.util.Arrays.stream(c.getConstructors()).anyMatch(ctor -> ctor.getParameterCount() == 0);
  } catch (ClassNotFoundException e) { return false; }
}

try {
  pipeline.run().waitUntilFinish();
} catch (Exception e) {
  if (e.getCause() instanceof IOException && e.getCause().getMessage().startsWith("Unable to create InputFormat object")) {
    throw new IllegalStateException("Check InputFormat class name and classpath bundling", e);
  }
  throw e;
}

When it happens

Trigger: The `inputFormatClass` configuration points to a class that is not on the classpath, is abstract/an interface, has no no-arg constructor, or whose constructor throws; triggered from computeSplitsIfNecessary(), createReader(), or testReadingWithConfigurableInputFormat().

Common situations: Hadoop client jars not shaded/ bundled in the Beam pipeline; typo in the fully-qualified InputFormat class name; InputFormat lacking a public no-arg constructor; conflicting Hadoop versions (e.g. class exists in two jars with incompatible signatures).

Related errors

• Could not find class
• ${e}
• Error in computing splits, getSplits() returns a empty list
• Error in computing splits, getSplits() returns null.
• Error in computing splits, split is null in InputSplits…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/4918b77d57cf45f1. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/hadoop-format/src/main/java/org/apache/beam/sdk/io/hadoop/format/HadoopFormatIO.java:843

                      .getClassByName(conf.get().get("mapreduce.job.inputformat.class"))
                      .getConstructor()
                      .newInstance();
          /*
           * If InputFormat explicitly implements interface {@link Configurable}, then setConf()
           * method of {@link Configurable} needs to be explicitly called to set all the
           * configuration parameters. For example: InputFormat classes which implement Configurable
           * are {@link org.apache.hadoop.mapreduce.lib.db.DBInputFormat DBInputFormat}, {@link
           * org.apache.hadoop.hbase.mapreduce.TableInputFormat TableInputFormat}, etc.
           */
          if (Configurable.class.isAssignableFrom(inputFormatObj.getClass())) {
            ((Configurable) inputFormatObj).setConf(conf.get());
          }
        } catch (InstantiationException
            | IllegalAccessException
            | ClassNotFoundException
            | NoSuchMethodException
            | InvocationTargetException e) {
          throw new IOException("Unable to create InputFormat object: ", e);
        }
      }
    }

    @VisibleForTesting
    InputFormat<?, ?> getInputFormat() {
      return inputFormatObj;
    }

    @VisibleForTesting
    void setInputFormatObj(InputFormat<?, ?> inputFormatObj) {
      this.inputFormatObj = inputFormatObj;
    }

    @Override
    public Coder<KV<K, V>> getOutputCoder() {
      return KvCoder.of(keyCoder, valueCoder);
    }

View on GitHub (pinned to 12126d8942)