apache/hadoop · error · RuntimeException

Unable to instantiate " + filtersClassName

Error message

Unable to instantiate " + filtersClassName

What it means

DistCp's CopyFilter.getCopyFilter reflectively loads the class named by distcp.filters.class, requires it to subclass CopyFilter with a constructor taking exactly Configuration, and instantiates it. Any failure — class not on the classpath, not a CopyFilter subclass, missing/non-public constructor, or the constructor itself throwing — is wrapped in a RuntimeException prefixed with 'Unable to instantiate <classname>' (DistCpConstants.CLASS_INSTANTIATION_ERROR_MSG), and the cause is both logged via LOG.error and attached.

Source

Thrown at hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/CopyFilter.java:71

   * @param conf DistCp configuration
   * @return An instance of the appropriate CopyFilter
   */
  public static CopyFilter getCopyFilter(Configuration conf) {
    String filtersClassName = conf
            .get(DistCpConstants.CONF_LABEL_FILTERS_CLASS);
    if (filtersClassName != null) {
      try {
        Class<? extends CopyFilter> filtersClass = conf
                .getClassByName(filtersClassName)
                .asSubclass(CopyFilter.class);
        filtersClassName = filtersClass.getName();
        Constructor<? extends CopyFilter> constructor = filtersClass
                .getDeclaredConstructor(Configuration.class);
        return constructor.newInstance(conf);
      } catch (Exception e) {
        LOG.error(DistCpConstants.CLASS_INSTANTIATION_ERROR_MSG +
                filtersClassName, e);
        throw new RuntimeException(
                DistCpConstants.CLASS_INSTANTIATION_ERROR_MSG +
                        filtersClassName, e);
      }
    } else {
      return getDefaultCopyFilter(conf);
    }
  }

  private static CopyFilter getDefaultCopyFilter(Configuration conf) {
    String filtersFilename = conf.get(DistCpConstants.CONF_LABEL_FILTERS_FILE);

    if (filtersFilename == null) {
      return new TrueCopyFilter();
    } else {
      String filterFilename = conf.get(
          DistCpConstants.CONF_LABEL_FILTERS_FILE);
      return new RegexCopyFilter(filterFilename);
    }

View on GitHub (pinned to 2add963021)

Solutions

1. Make the class public, extend org.apache.hadoop.tools.CopyFilter, and expose a public constructor taking exactly Configuration
2. Ship the filter jar where both the client and the MR tasks load it (distcp -libjars or the cluster share dir)
3. Read the LOG.error line and the attached cause: ClassNotFoundException means classpath, NoSuchMethodException means signature, InvocationTargetException means the constructor threw
4. Temporarily unset distcp.filters.class to confirm the default (regex) filter path works, isolating the problem

Example fix

// before: only a no-arg constructor
public class MyFilter extends CopyFilter {
  public MyFilter() { }
}
// -> RuntimeException: Unable to instantiate com.example.MyFilter

// after
public class MyFilter extends CopyFilter {
  public MyFilter(Configuration conf) { super(conf); }
}

String cls = conf.get(DistCpConstants.CONF_LABEL_FILTERS_CLASS);
if (cls != null) {
  Class<?> c = Class.forName(cls);
  if (!CopyFilter.class.isAssignableFrom(c)
      || !Modifier.isPublic(c.getModifiers())) {
    throw new IllegalArgumentException(
        cls + " must be a public CopyFilter subclass");
  }
  c.getConstructor(Configuration.class); // NoSuchMethodException -> fail fast
}

Catch RuntimeException around DistCp execution; when the message starts with 'Unable to instantiate' plus your filter class name, inspect the cause — ClassNotFoundException means classpath, NoSuchMethodException means the (Configuration) constructor is missing, InvocationTargetException means the constructor threw — fix accordingly and rerun.

When it happens

Trigger: Setting distcp.filters.class to a custom filter class that (a) is not on the client and MR-task classpath, (b) does not extend CopyFilter, (c) lacks a public constructor with the exact signature CopyFilter(Configuration), or (d) whose constructor throws (e.g. it reads a config key that is absent).

Common situations: Custom filter jar not shipped to cluster nodes (works locally, fails in the MR job); refactoring changed the constructor signature; typo in the class name; filter logic requiring a filters file that was not provided.

Related errors

• Unable to instantiate " + copyListingClassName
• Errors on getting mount table loader class. The fs.viewfs.mo
• Error creating plugin: {}
• Failed to get method
• Unable to create the Decommission monitor from {cls}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/27326f2840c5e8f5. Report an issue: GitHub.