apache/hadoop · error · RuntimeException

Class not found: {}

Error message

Class not found: {}

What it means

StreamInputFormat.getRecordReader() loads the class named by the job property stream.recordreader.class (used for non-standard readers such as StreamXmlRecordReader) via StreamUtil.goodClassOrNull(). If the class cannot be loaded from the job classpath, it returns null and the code throws RuntimeException 'Class not found: <classname>'. The next failure mode after this (NoSuchMethodException wrapper) requires a constructor (FSDataInputStream, FileSplit, Reporter, JobConf, FileSystem).

Source

Thrown at hadoop-tools/hadoop-streaming/src/main/java/org/apache/hadoop/streaming/StreamInputFormat.java:60

      return super.getRecordReader(genericSplit, job, reporter);
    }

    // handling non-standard record reader (likely StreamXmlRecordReader) 
    FileSplit split = (FileSplit) genericSplit;
    LOG.info("getRecordReader start.....split=" + split);
    reporter.setStatus(split.toString());

    // Open the file and seek to the start of the split
    FileSystem fs = split.getPath().getFileSystem(job);
    FSDataInputStream in = fs.open(split.getPath());

    // Factory dispatch based on available params..
    Class readerClass;

    {
      readerClass = StreamUtil.goodClassOrNull(job, c, null);
      if (readerClass == null) {
        throw new RuntimeException("Class not found: " + c);
      }
    }

    Constructor ctor;
    try {
      ctor = readerClass.getConstructor(new Class[] { FSDataInputStream.class,
                                                      FileSplit.class, Reporter.class, JobConf.class, FileSystem.class });
    } catch (NoSuchMethodException nsm) {
      throw new RuntimeException(nsm);
    }

    RecordReader<Text, Text> reader;
    try {
      reader = (RecordReader<Text, Text>) ctor.newInstance(new Object[] { in, split,
                                                              reporter, job, fs });
    } catch (Exception nsm) {
      throw new RuntimeException(nsm);
    }

View on GitHub (pinned to 2add963021)

Solutions

1. Ship the reader's jar with the job: -libjars myreaders.jar (and ensure nodes can resolve it) or place it on the cluster-wide classpath
2. Verify the fully-qualified class name spelling, package and case exactly match the compiled class
3. Check the class has a public constructor (FSDataInputStream, FileSplit, Reporter, JobConf, FileSystem) or it will fail the next check
4. For XML input, use the bundled org.apache.hadoop.streaming.StreamXmlRecordReader with -inputreader StreamXmlRecordReader instead of a hand-rolled class

Example fix

# before
hadoop jar hadoop-streaming.jar -input in -output out \
  -D stream.recordreader.class=com.myco.MyXmlReader -mapper cat
# after: ship the jar containing the reader
hadoop jar hadoop-streaming.jar -input in -output out \
  -libjars myxmlreader.jar \
  -D stream.recordreader.class=com.myco.MyXmlReader -mapper cat

// before submitting: resolve exactly like StreamInputFormat will
Class<?> c = StreamUtil.goodClassOrNull(jobConf, "com.myco.MyXmlReader", null);
if (c == null) throw new IllegalStateException("stream.recordreader.class not loadable — add jar via -libjars");
try {
  c.getConstructor(FSDataInputStream.class, FileSplit.class, Reporter.class, JobConf.class, FileSystem.class);
} catch (NoSuchMethodException e) { throw new IllegalStateException("reader lacks required constructor", e); }

catch RuntimeException from job run; if the message is 'Class not found: X', fix classpath/ship the jar — no retry helps until the class resolves.

When it happens

Trigger: Setting stream.recordreader.class (typically -D stream.recordreader.class=org.apache.hadoop.streaming.StreamXmlRecordReader or a custom reader) to a class that is not on the task classpath: typo in the FQCN, jar not shipped via -libjars/-files, or custom reader compiled against a different version.

Common situations: Custom record readers developed out-of-tree and forgotten in the submit command, package renames between versions, case-sensitive class names mistyped on the command line, or the jar present on the client but never shipped to task nodes.

Related errors

• runtime classes not found: {}
• f + " is a directory"
• No KeyProviderFactory for ${uri} in ${KEY_PROVIDER_PATH}
• Could not find configured fencing method {}
• Class {} does not implement FenceMethod

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/292c88b3b8a85e51. Report an issue: GitHub.