apache/hadoop · error · IllegalArgumentException

Couldn't find listing-file at: {listingFilePath}

Error message

Couldn't find listing-file at: {listingFilePath}

What it means

Same listing-file load path, but here fs.exists() passed and opening the SequenceFile.Reader threw an IOException: the file is present yet unreadable (permissions, truncation/corruption, checksum error, connectivity). The original exception is logged with full detail, then wrapped in this IllegalArgumentException with the listing path and the cause.

Source

Thrown at hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/UniformSizeInputFormat.java:156

              : "Couldn't find listing file. Invalid input.";
    return new Path(listingFilePathString);
  }

  private SequenceFile.Reader getListingFileReader(Configuration configuration) {

    final Path listingFilePath = getListingFilePath(configuration);
    try {
      final FileSystem fileSystem = listingFilePath.getFileSystem(configuration);
      if (!fileSystem.exists(listingFilePath))
        throw new IllegalArgumentException("Listing file doesn't exist at: "
                                           + listingFilePath);

      return new SequenceFile.Reader(configuration,
                                     SequenceFile.Reader.file(listingFilePath));
    }
    catch (IOException exception) {
      LOG.error("Couldn't find listing file at: " + listingFilePath, exception);
      throw new IllegalArgumentException("Couldn't find listing-file at: "
                                         + listingFilePath, exception);
    }
  }

  /**
   * Implementation of InputFormat::createRecordReader().
   * @param split The split for which the RecordReader is sought.
   * @param context The context of the current task-attempt.
   * @return A SequenceFileRecordReader instance, (since the copy-listing is a
   * simple sequence-file.)
   * @throws IOException
   * @throws InterruptedException
   */
  @Override
  public RecordReader<Text, CopyListingFileStatus> createRecordReader(
      InputSplit split, TaskAttemptContext context)
      throws IOException, InterruptedException {
    return new SequenceFileRecordReader<Text, CopyListingFileStatus>();

View on GitHub (pinned to 2add963021)

Solutions

1. Read the wrapped cause and task log to identify the underlying filesystem error
2. Check the listing file's integrity and size (hdfs fsck <listingFilePath>); if corrupt, regenerate by re-running distcp
3. Fix read permissions so the task user can open the file
4. Always regenerate the listing fresh instead of reusing meta folders from failed attempts

// probe that the listing is fully readable, not just present
try (SequenceFile.Reader r = new SequenceFile.Reader(conf,
        SequenceFile.Reader.file(listing))) {
  // listing readable
}

When it happens

Trigger: The listing sequence file is truncated because the listing step was killed mid-write; the MapReduce user lacks read permission; the file's blocks are missing or corrupt; NameNode/DataNode problems during split generation.

Common situations: A previously failed distcp left a partial listing that is reused; listing stored on a busy or decommissioning cluster; Kerberos/permission mismatch between the submitting user and the YARN task user.

Related errors

• Listing file doesn't exist at: {listingFilePath}
• key + ": No such file or directory."
• Can't read FileStatusProto with negative size of ${size}
• Illegal buffer length " + len
• Stream data required

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/49ce2bd72650769f. Report an issue: GitHub.