apache/hadoop · error · IOException

Unable to instantiate " + copyListingClassName

Error message

Unable to instantiate " + copyListingClassName

What it means

CopyListing.getCopyListing picks the listing implementation — from distcp.copy.listing.class when set, otherwise GlobbedCopyListing or FileBasedCopyListing depending on whether a source file listing was supplied — and reflectively constructs it via its (Configuration, Credentials) constructor. Any reflective failure is rethrown as IOException('Unable to instantiate <class name>') with the cause attached.

Source

Thrown at hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/CopyListing.java:304

    Class<? extends CopyListing> copyListingClass;
    try {
      if (! copyListingClassName.isEmpty()) {
        copyListingClass = configuration.getClass(DistCpConstants.
            CONF_LABEL_COPY_LISTING_CLASS, GlobbedCopyListing.class,
            CopyListing.class);
      } else {
        if (context.getSourceFileListing() == null) {
            copyListingClass = GlobbedCopyListing.class;
        } else {
            copyListingClass = FileBasedCopyListing.class;
        }
      }
      copyListingClassName = copyListingClass.getName();
      Constructor<? extends CopyListing> constructor = copyListingClass.
          getDeclaredConstructor(Configuration.class, Credentials.class);
      return constructor.newInstance(configuration, credentials);
    } catch (Exception e) {
      throw new IOException("Unable to instantiate " + copyListingClassName, e);
    }
  }

  static class DuplicateFileException extends RuntimeException {
    public DuplicateFileException(String message) {
      super(message);
    }
  }

  static class InvalidInputException extends RuntimeException {
    public InvalidInputException(String message) {
      super(message);
    }

    public InvalidInputException(String message, Throwable cause) {
      super(message, cause);
    }
  }

View on GitHub (pinned to 2add963021)

Solutions

1. Ensure the class extends CopyListing with a public constructor taking exactly (Configuration, Credentials)
2. Ship it on the distcp classpath for both client and tasks (e.g. -libjars)
3. Read the cause: ClassNotFoundException -> classpath, NoSuchMethodException -> signature, InvocationTargetException -> constructor threw
4. Unset distcp.copy.listing.class to fall back to the default listing and isolate the issue

Example fix

// before: wrong constructor signature
public class MyListing extends CopyListing {
  public MyListing(Configuration conf) { super(conf); }
}
// -> IOException: Unable to instantiate com.example.MyListing

// after
public class MyListing extends CopyListing {
  public MyListing(Configuration conf, Credentials credentials) {
    super(conf, credentials);
  }
}

String cls = conf.get(DistCpConstants.CONF_LABEL_COPY_LISTING_CLASS, "");
if (!cls.isEmpty()) {
  Class<?> c = Class.forName(cls);
  if (!CopyListing.class.isAssignableFrom(c)) {
    throw new IllegalArgumentException(cls + " must extend CopyListing");
  }
  c.getConstructor(Configuration.class, Credentials.class); // fail fast
}

Catch IOException around DistCp.execute(); when the message starts with 'Unable to instantiate' plus the listing class name, inspect the cause to distinguish classpath (ClassNotFoundException), constructor signature (NoSuchMethodException), or constructor failure (InvocationTargetException), then fix and rerun.

When it happens

Trigger: Configuring distcp.copy.listing.class to a custom class whose constructor is not exactly (Configuration, Credentials) or is non-public; the class missing from the client/task classpath; or the constructor throwing during instantiation.

Common situations: Custom CopyListing implementations drifting from the required constructor signature across versions; jar packaging differences between client and MR tasks; class-name typos in the configuration.

Related errors

• Unable to instantiate " + filtersClassName
• Errors on getting mount table loader class. The fs.viewfs.mo
• Error creating plugin: {}
• Failed to get method
• Unable to create the Decommission monitor from {cls}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/8140828b044645b7. Report an issue: GitHub.