{"record":{"id":"49ce2bd72650769f","repo":"apache/hadoop","slug":"couldn-t-find-listing-file-at-listingfilepath","errorCode":null,"errorMessage":"Couldn't find listing-file at: {listingFilePath}","messagePattern":"Couldn't find listing-file at: (.+?)","errorType":"exception","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/UniformSizeInputFormat.java","lineNumber":156,"sourceCode":"              : \"Couldn't find listing file. Invalid input.\";\n    return new Path(listingFilePathString);\n  }\n\n  private SequenceFile.Reader getListingFileReader(Configuration configuration) {\n\n    final Path listingFilePath = getListingFilePath(configuration);\n    try {\n      final FileSystem fileSystem = listingFilePath.getFileSystem(configuration);\n      if (!fileSystem.exists(listingFilePath))\n        throw new IllegalArgumentException(\"Listing file doesn't exist at: \"\n                                           + listingFilePath);\n\n      return new SequenceFile.Reader(configuration,\n                                     SequenceFile.Reader.file(listingFilePath));\n    }\n    catch (IOException exception) {\n      LOG.error(\"Couldn't find listing file at: \" + listingFilePath, exception);\n      throw new IllegalArgumentException(\"Couldn't find listing-file at: \"\n                                         + listingFilePath, exception);\n    }\n  }\n\n  /**\n   * Implementation of InputFormat::createRecordReader().\n   * @param split The split for which the RecordReader is sought.\n   * @param context The context of the current task-attempt.\n   * @return A SequenceFileRecordReader instance, (since the copy-listing is a\n   * simple sequence-file.)\n   * @throws IOException\n   * @throws InterruptedException\n   */\n  @Override\n  public RecordReader<Text, CopyListingFileStatus> createRecordReader(\n      InputSplit split, TaskAttemptContext context)\n      throws IOException, InterruptedException {\n    return new SequenceFileRecordReader<Text, CopyListingFileStatus>();","sourceCodeStart":138,"sourceCodeEnd":174,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/UniformSizeInputFormat.java#L138-L174","documentation":"Same listing-file load path, but here fs.exists() passed and opening the SequenceFile.Reader threw an IOException: the file is present yet unreadable (permissions, truncation/corruption, checksum error, connectivity). The original exception is logged with full detail, then wrapped in this IllegalArgumentException with the listing path and the cause.","triggerScenarios":"The listing sequence file is truncated because the listing step was killed mid-write; the MapReduce user lacks read permission; the file's blocks are missing or corrupt; NameNode/DataNode problems during split generation.","commonSituations":"A previously failed distcp left a partial listing that is reused; listing stored on a busy or decommissioning cluster; Kerberos/permission mismatch between the submitting user and the YARN task user.","solutions":["Read the wrapped cause and task log to identify the underlying filesystem error","Check the listing file's integrity and size (hdfs fsck <listingFilePath>); if corrupt, regenerate by re-running distcp","Fix read permissions so the task user can open the file","Always regenerate the listing fresh instead of reusing meta folders from failed attempts"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// probe that the listing is fully readable, not just present\ntry (SequenceFile.Reader r = new SequenceFile.Reader(conf,\n        SequenceFile.Reader.file(listing))) {\n  // listing readable\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Regenerate listings after any failed or killed distcp attempt","Give the YARN task user read access to the meta folder","Check listing file size looks complete before submitting"],"tags":["distcp","hadoop","listing","sequencefile","corruption"],"backgroundTag":"unreadable-input-file","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}