apache/hadoop · error · IOException

no SharedFileDescriptorFactory paths were configured.

Error message

no SharedFileDescriptorFactory paths were configured.

What it means

SharedFileDescriptorFactory.create(prefix, paths[]) builds the shared-memory descriptor directory used by HDFS short-circuit reads to pass file descriptors between client and DataNode (/dev/shm or /tmp candidates). It requires at least one candidate directory; an empty array is a configuration error and fails immediately with this IOException.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/nativeio/SharedFileDescriptorFactory.java:81

  /**
   * Create a new SharedFileDescriptorFactory.
   *
   * @param prefix       The prefix to prepend to all the file names created
   *                       by this factory.
   * @param paths        An array of paths to use.  We will try each path in 
   *                       succession, and return a factory using the first 
   *                       usable path.
   * @return             The factory.
   * @throws IOException If a factory could not be created for any reason.
   */
  public static SharedFileDescriptorFactory create(String prefix,
      String paths[]) throws IOException {
    String loadingFailureReason = getLoadingFailureReason();
    if (loadingFailureReason != null) {
      throw new IOException(loadingFailureReason);
    }
    if (paths.length == 0) {
      throw new IOException("no SharedFileDescriptorFactory paths were " +
          "configured.");
    }
    StringBuilder errors = new StringBuilder();
    String strPrefix = "";
    for (String path : paths) {
      try {
        FileInputStream fis = 
            new FileInputStream(createDescriptor0(prefix + "test", path, 1));
        fis.close();
        deleteStaleTemporaryFiles0(prefix, path);
        return new SharedFileDescriptorFactory(prefix, path);
      } catch (IOException e) {
        errors.append(strPrefix).append("Error creating file descriptor in ").
               append(path).append(": ").append(e.getMessage());
        strPrefix = ", ";
      }
    }
    throw new IOException(errors.toString());

View on GitHub (pinned to 2add963021)

Solutions

1. Set the paths property to a comma-separated candidate list, e.g. dfs.datanode.shared.file.descriptor.paths=/dev/shm,/tmp.
2. If you call the factory directly, always pass at least one existing, writable directory.
3. If you do not need short-circuit shared-memory features, disable short-circuit reads instead of blanking the paths.

Example fix

// before
factory = SharedFileDescriptorFactory.create(prefix, new String[0]);

// after
factory = SharedFileDescriptorFactory.create(prefix,
    new String[] { "/dev/shm", "/tmp" });

String[] candidates = conf.getTrimmedStrings(
    "dfs.datanode.shared.file.descriptor.paths",
    new String[] { "/dev/shm", "/tmp" });
if (candidates == null || candidates.length == 0) {
  throw new IOException("no shared-file-descriptor paths configured");
}

When it happens

Trigger: Calling create with a zero-length paths array — usually because the config that supplies the list (e.g. dfs.datanode.shared.file.descriptor.paths, default '/dev/shm,/tmp', or the client-side equivalent) was set to an empty string and parsed into no entries.

Common situations: Short-circuit local reads enabled while the shared-file-descriptor paths property was blanked in site.xml; embedded/custom code passing an empty list; deployment templates that zero out paths 'to use defaults' but actually disable them.

Related errors

• got EOF while trying to transfer the file descriptor for the
• {feature} is enabled but dfs.domain.socket.path is not set.
• Can not create a Path from a null string
• Can not create a Path from an empty string
• Error creating file descriptor in ${path}: ${e.getMessage()}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/85508150480503c1. Report an issue: GitHub.