{"record":{"id":"b8c11db4a69d2033","repo":"apache/hadoop","slug":"resource-not-found-during-scripts-prepare-res","errorCode":null,"errorMessage":"Resource not found during scripts prepare: \" + res","messagePattern":"Resource not found during scripts prepare: \" \\+ res","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-compat-bench/src/main/java/org/apache/hadoop/fs/compat/common/HdfsCompatShellScope.java","lineNumber":141,"sourceCode":"    File logConfFile = new File(confDir, \"log4j.properties\");\n    copyResource(\"/hadoop-compat-bench-log4j.properties\", logConfFile, true);\n  }\n\n  @VisibleForTesting\n  protected void copyResource(String res, File dst) throws IOException {\n    copyResource(res, dst, false);\n  }\n\n  private void copyResource(String res, File dst, boolean overwrite)\n      throws IOException {\n    InputStream in = null;\n    try {\n      in = this.getClass().getResourceAsStream(res);\n      if (in == null) {\n        in = this.suite.getClass().getResourceAsStream(res);\n      }\n      if (in == null) {\n        throw new IOException(\"Resource not found\" +\n            \" during scripts prepare: \" + res);\n      }\n\n      if (dst.exists() && !overwrite) {\n        throw new IOException(\"Cannot overwrite existing resource file\");\n      }\n\n      Files.createDirectories(dst.getParentFile().toPath());\n\n      byte[] buf = new byte[1024];\n      try (OutputStream out = new FileOutputStream(dst)) {\n        int nRead = in.read(buf);\n        while (nRead != -1) {\n          out.write(buf, 0, nRead);\n          nRead = in.read(buf);\n        }\n      }\n    } finally {","sourceCodeStart":123,"sourceCodeEnd":159,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-compat-bench/src/main/java/org/apache/hadoop/fs/compat/common/HdfsCompatShellScope.java#L123-L159","documentation":"To run shell cases, HdfsCompatShellScope copies script resources (paths under /cases/...) into the sandbox: it first tries HdfsCompatShellScope's own class loader, then the suite's class. If neither resolves the resource, it throws IOException('Resource not found during scripts prepare: <res>'). The resource must therefore be packaged in one of those jars at the exact declared path.","triggerScenarios":"A shell case declares a script resource (e.g. /cases/my_case.sh) that is not packaged in the suite jar or the compat-bench jar; leading-slash or case mismatch between the declared name and the packaged path; shade/assembly plugins filtering out non-class files.","commonSituations":"Script placed outside src/main/resources/cases; script renamed without updating the case definition; old jar still deployed after adding a new case; aggressive maven-shade filters dropping resources.","solutions":["Put the script at src/main/resources/cases/<name> so it is packaged at /cases/<name> on the classpath","Match the declared resource path exactly — case-sensitive, leading slash included","Rebuild and redeploy, then verify: unzip -l suite.jar | grep cases/"],"exampleFix":"# before: case declares \"/cases/my_case.sh\" but the jar has\n#   src/main/resources/scripts/my_case.sh  -> Resource not found\n\n# after: move the file so the packaged path matches\nmkdir -p src/main/resources/cases\ngit mv src/main/resources/scripts/my_case.sh src/main/resources/cases/my_case.sh\nmvn package && hadoop jar target/suite.jar ... # unzip -l verifies /cases/my_case.sh","handlingStrategy":"validation","validationCode":"static boolean resourceExists(Class<?> suiteClass, String res) {\n  return HdfsCompatShellScope.class.getResource(res) != null\n      || suiteClass.getResource(res) != null;\n}\n\nList<String> missing = declaredCaseResources.stream()\n    .filter(r -> !resourceExists(suiteClass, r))\n    .collect(Collectors.toList());\nif (!missing.isEmpty()) {\n  throw new IllegalStateException(\"Missing case resources: \" + missing);\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Add a packaging test asserting every declared case resource resolves on the classpath","Derive resource paths from the same constant the case declares, so names cannot drift","Check jars after shading: unzip -l app.jar | grep cases/"],"tags":["hadoop-compat-bench","classpath","resources","packaging","shell-tests"],"backgroundTag":"missing-classpath-resource","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}