{"record":{"id":"336f4133ea6c4b49","repo":"apache/hadoop","slug":"p-doesn-t-exist","errorCode":null,"errorMessage":"{p} doesn't exist","messagePattern":"(.+?) doesn't exist","errorType":"exception","errorClass":"InvalidInputException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/GlobbedCopyListing.java","lineNumber":84,"sourceCode":"  @Override\n  public void doBuildListing(Path pathToListingFile, DistCpContext context)\n      throws IOException {\n\n    List<Path> globbedPaths = new ArrayList<Path>();\n    if (context.getSourcePaths().isEmpty()) {\n      throw new InvalidInputException(\"Nothing to process. Source paths::EMPTY\");  \n    }\n\n    for (Path p : context.getSourcePaths()) {\n      FileSystem fs = p.getFileSystem(getConf());\n      FileStatus[] inputs = fs.globStatus(p);\n\n      if(inputs != null && inputs.length > 0) {\n        for (FileStatus onePath: inputs) {\n          globbedPaths.add(onePath.getPath());\n        }\n      } else {\n        throw new InvalidInputException(p + \" doesn't exist\");        \n      }\n    }\n\n    context.setSourcePaths(globbedPaths);\n    simpleListing.buildListing(pathToListingFile, context);\n  }\n\n  /** {@inheritDoc} */\n  @Override\n  protected long getBytesToCopy() {\n    return simpleListing.getBytesToCopy();\n  }\n\n  /** {@inheritDoc} */\n  @Override\n  protected long getNumberOfPaths() {\n    return simpleListing.getNumberOfPaths();\n  }","sourceCodeStart":66,"sourceCodeEnd":102,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/GlobbedCopyListing.java#L66-L102","documentation":"During listing, GlobbedCopyListing resolves every source path through FileSystem.globStatus(). A null or zero-length result means the pattern matched nothing on that FileSystem - a nonexistent path, a typo, or a legal glob that happens to match zero files. DistCp aborts listing, so the job never submits.","triggerScenarios":"A source glob with no matches, e.g. 'hdfs://nn/data/2026-08-22-*.csv' on a day with no data; a path on a different nameservice than the authority given; a source deleted between run scheduling and listing; a literal filename containing glob metacharacters (?, *, {, [) unescaped, so it is treated as a pattern matching nothing.","commonSituations":"date-stamped globs for periods with no data; typos in long HDFS paths; relative paths resolving against a different fs.defaultFS; viewfs/HA authorities that mount the path differently; filenames with literal glob characters.","solutions":["Run the same pattern through ls first: hadoop fs -ls 'hdfs://nn/data/2026-08-22-*.csv' - if it shows nothing, fix the pattern or paths.","Use fully-qualified URIs with the correct scheme and nameservice authority instead of relative paths.","Escape glob metacharacters that are literal parts of filenames (\\*, \\?, \\[, \\{).","If missing sources are expected, pre-filter them out of the -f listing instead of letting distcp abort."],"exampleFix":"# before: glob matches nothing, listing aborts\nhadoop distcp 'hdfs://nn/inbox/2026-08-22-*.avro' hdfs://nn2/archive/\n\n# after: verify the expansion, then run\nhadoop fs -ls 'hdfs://nn/inbox/2026-08-22-*.avro'\nhadoop distcp 'hdfs://nn/inbox/2026-08-22-*.avro' hdfs://nn2/archive/","handlingStrategy":"validation","validationCode":"// Pre-flight: every source pattern must expand to >=1 entry\nfor (Path p : context.getSourcePaths()) {\n  FileSystem fs = p.getFileSystem(conf);\n  FileStatus[] matches = fs.globStatus(p);\n  if (matches == null || matches.length == 0) {\n    throw new InvalidInputException(\"Source matches nothing: \" + p);\n  }\n}","typeGuard":null,"tryCatchPattern":"try {\n  distCp.run(args);\n} catch (InvalidInputException e) {\n  // message embeds the failing path, e.g. \"hdfs://nn/x/*.csv doesn't exist\"\n  if (e.getMessage().endsWith(\"doesn't exist\")) {\n    reportAndFixSource(e.getMessage()); // fix pattern/authority, re-run - no point retrying as-is\n  } else {\n    throw e;\n  }\n}","preventionTips":["Pre-flight `hadoop fs -ls` every source pattern exactly as distcp will see it.","Always use fully-qualified URIs with the correct nameservice authority.","Escape glob metacharacters that occur literally in filenames.","Alert when date-stamped globs match nothing - usually a sign data is missing upstream."],"tags":["distcp","glob","path-not-found","input-validation","hdfs"],"backgroundTag":"glob-pattern-no-match","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}