apache/hadoop · error · IOException

Resource not found during scripts prepare: " + res

Error message

Resource not found during scripts prepare: " + res

What it means

To run shell cases, HdfsCompatShellScope copies script resources (paths under /cases/...) into the sandbox: it first tries HdfsCompatShellScope's own class loader, then the suite's class. If neither resolves the resource, it throws IOException('Resource not found during scripts prepare: <res>'). The resource must therefore be packaged in one of those jars at the exact declared path.

Source

Thrown at hadoop-tools/hadoop-compat-bench/src/main/java/org/apache/hadoop/fs/compat/common/HdfsCompatShellScope.java:141

    File logConfFile = new File(confDir, "log4j.properties");
    copyResource("/hadoop-compat-bench-log4j.properties", logConfFile, true);
  }

  @VisibleForTesting
  protected void copyResource(String res, File dst) throws IOException {
    copyResource(res, dst, false);
  }

  private void copyResource(String res, File dst, boolean overwrite)
      throws IOException {
    InputStream in = null;
    try {
      in = this.getClass().getResourceAsStream(res);
      if (in == null) {
        in = this.suite.getClass().getResourceAsStream(res);
      }
      if (in == null) {
        throw new IOException("Resource not found" +
            " during scripts prepare: " + res);
      }

      if (dst.exists() && !overwrite) {
        throw new IOException("Cannot overwrite existing resource file");
      }

      Files.createDirectories(dst.getParentFile().toPath());

      byte[] buf = new byte[1024];
      try (OutputStream out = new FileOutputStream(dst)) {
        int nRead = in.read(buf);
        while (nRead != -1) {
          out.write(buf, 0, nRead);
          nRead = in.read(buf);
        }
      }
    } finally {

View on GitHub (pinned to 2add963021)

Solutions

1. Put the script at src/main/resources/cases/<name> so it is packaged at /cases/<name> on the classpath
2. Match the declared resource path exactly — case-sensitive, leading slash included
3. Rebuild and redeploy, then verify: unzip -l suite.jar | grep cases/

Example fix

# before: case declares "/cases/my_case.sh" but the jar has
#   src/main/resources/scripts/my_case.sh  -> Resource not found

# after: move the file so the packaged path matches
mkdir -p src/main/resources/cases
git mv src/main/resources/scripts/my_case.sh src/main/resources/cases/my_case.sh
mvn package && hadoop jar target/suite.jar ... # unzip -l verifies /cases/my_case.sh

static boolean resourceExists(Class<?> suiteClass, String res) {
  return HdfsCompatShellScope.class.getResource(res) != null
      || suiteClass.getResource(res) != null;
}

List<String> missing = declaredCaseResources.stream()
    .filter(r -> !resourceExists(suiteClass, r))
    .collect(Collectors.toList());
if (!missing.isEmpty()) {
  throw new IllegalStateException("Missing case resources: " + missing);
}

When it happens

Trigger: A shell case declares a script resource (e.g. /cases/my_case.sh) that is not packaged in the suite jar or the compat-bench jar; leading-slash or case mismatch between the declared name and the packaged path; shade/assembly plugins filtering out non-class files.

Common situations: Script placed outside src/main/resources/cases; script renamed without updating the case definition; old jar still deployed after adding a new case; aggressive maven-shade filters dropping resources.

Related errors

• webapps/{} not found in CLASSPATH
• Can not read resource file '{}'
• ${resource}
• Can not read resource file '{}' because class loader of the
• Can not read resource file '{}' because given class loader i

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/b8c11db4a69d2033. Report an issue: GitHub.