{"record":{"id":"b8d2c95db494a14a","repo":"apache/hadoop","slug":"listing-file-doesn-t-exist-at-listingfilepath","errorCode":null,"errorMessage":"Listing file doesn't exist at: {listingFilePath}","messagePattern":"Listing file doesn't exist at: (.+?)","errorType":"exception","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/UniformSizeInputFormat.java","lineNumber":148,"sourceCode":"    return splits;\n  }\n\n  private static Path getListingFilePath(Configuration configuration) {\n    final String listingFilePathString =\n            configuration.get(DistCpConstants.CONF_LABEL_LISTING_FILE_PATH, \"\");\n\n    assert !listingFilePathString.equals(\"\")\n              : \"Couldn't find listing file. Invalid input.\";\n    return new Path(listingFilePathString);\n  }\n\n  private SequenceFile.Reader getListingFileReader(Configuration configuration) {\n\n    final Path listingFilePath = getListingFilePath(configuration);\n    try {\n      final FileSystem fileSystem = listingFilePath.getFileSystem(configuration);\n      if (!fileSystem.exists(listingFilePath))\n        throw new IllegalArgumentException(\"Listing file doesn't exist at: \"\n                                           + listingFilePath);\n\n      return new SequenceFile.Reader(configuration,\n                                     SequenceFile.Reader.file(listingFilePath));\n    }\n    catch (IOException exception) {\n      LOG.error(\"Couldn't find listing file at: \" + listingFilePath, exception);\n      throw new IllegalArgumentException(\"Couldn't find listing-file at: \"\n                                         + listingFilePath, exception);\n    }\n  }\n\n  /**\n   * Implementation of InputFormat::createRecordReader().\n   * @param split The split for which the RecordReader is sought.\n   * @param context The context of the current task-attempt.\n   * @return A SequenceFileRecordReader instance, (since the copy-listing is a\n   * simple sequence-file.)","sourceCodeStart":130,"sourceCodeEnd":166,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/UniformSizeInputFormat.java#L130-L166","documentation":"UniformSizeInputFormat (the listing-based split strategy) reads the copy-listing sequence file whose path is stored in distcp.listing.file.path. During split generation, if the listing file does not exist on its filesystem, splits cannot be created and this IllegalArgumentException is thrown. The listing is normally produced by CopyListing during DistCp setup before job submission.","triggerScenarios":"The distcp meta/staging folder containing the listing was deleted or cleaned between listing generation and task execution; a hand-built job pointing at a stale or mistyped listing path; the listing sits on a filesystem currently unreachable from the tasks.","commonSituations":"Automated cleanup of _distcp* staging dirs while the job is alive; a job retried long after submission with an expired listing; cross-cluster jobs where the listing path is on a nameservice the mappers cannot reach.","solutions":["Verify the printed path exists and is readable by the job user (hdfs dfs -ls <listingFilePath>)","Re-run the distcp job so the copy-listing is regenerated","Protect the distcp meta folder from deletion for the lifetime of the MapReduce job","When building jobs manually, generate the listing with CopyListing and set distcp.listing.file.path to the result"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"Path listing = new Path(conf.get(\"distcp.listing.file.path\"));\nif (!listing.getFileSystem(conf).exists(listing)) {\n  throw new FileNotFoundException(\"copy-listing missing: \" + listing);\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Never delete a distcp staging/meta folder while its job is running","Regenerate the listing for each run instead of reusing old meta folders","Ensure tasks can reach the filesystem hosting the listing"],"tags":["distcp","hadoop","listing","inputformat","missing-file"],"backgroundTag":"missing-input-file","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}