{"record":{"id":"a69234dd646cdfd6","repo":"apache/hadoop","slug":"error-creating-file-descriptor-in-path-e-get","errorCode":null,"errorMessage":"Error creating file descriptor in ${path}: ${e.getMessage()}","messagePattern":"Error creating file descriptor in (.+?): (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/nativeio/SharedFileDescriptorFactory.java","lineNumber":99,"sourceCode":"      throw new IOException(\"no SharedFileDescriptorFactory paths were \" +\n          \"configured.\");\n    }\n    StringBuilder errors = new StringBuilder();\n    String strPrefix = \"\";\n    for (String path : paths) {\n      try {\n        FileInputStream fis = \n            new FileInputStream(createDescriptor0(prefix + \"test\", path, 1));\n        fis.close();\n        deleteStaleTemporaryFiles0(prefix, path);\n        return new SharedFileDescriptorFactory(prefix, path);\n      } catch (IOException e) {\n        errors.append(strPrefix).append(\"Error creating file descriptor in \").\n               append(path).append(\": \").append(e.getMessage());\n        strPrefix = \", \";\n      }\n    }\n    throw new IOException(errors.toString());\n  }\n\n  /**\n   * Create a SharedFileDescriptorFactory.\n   *\n   * @param prefix    Prefix to add to all file names we use.\n   * @param path      Path to use.\n   */\n  private SharedFileDescriptorFactory(String prefix, String path) {\n    this.prefix = prefix;\n    this.path = path;\n  }\n\n  public String getPath() {\n    return path;\n  }\n\n  /**","sourceCodeStart":81,"sourceCodeEnd":117,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/nativeio/SharedFileDescriptorFactory.java#L81-L117","documentation":"SharedFileDescriptorFactory.create probes each candidate directory by creating a test descriptor (createDescriptor0); on IOException it appends 'Error creating file descriptor in <path>: <cause>' and tries the next directory. If every candidate fails, the accumulated fragments are thrown as one IOException, each naming a directory and its underlying OS reason (permission denied, no such file, no space...).","triggerScenarios":"All configured dirs failing at once: /dev/shm missing or mounted tiny/full (common in containers), /tmp not writable by the daemon user, SELinux/AppArmor denials on the shm path, disk exhaustion on the fallback dir.","commonSituations":"Docker/Kubernetes containers without /dev/shm or with a small --shm-size; hardened hosts where the DataNode user cannot write the configured dir; /tmp filled by spill files.","solutions":["Read each '<path>: <message>' fragment and fix the stated cause for the first candidate (mount missing, permission, space).","Ensure /dev/shm exists, is writable by the daemon user, and is large enough (df -h /dev/shm; raise --shm-size in containers).","Extend the candidate list (comma-separated) with a guaranteed-writable fallback such as /dev/shm,/tmp,/var/tmp.","If short-circuit reads are not required, disable them (dfs.client.read.shortcircuit=false) rather than fighting the environment."],"exampleFix":"# container with missing/too-small /dev/shm\ndocker run --shm-size=1g ...\n\n# or widen candidates in hdfs-site.xml\n<property>\n  <name>dfs.datanode.shared.file.descriptor.paths</name>\n  <value>/dev/shm,/tmp</value>\n</property>","handlingStrategy":"fallback","validationCode":"for (String p : paths) {\n  File d = new File(p);\n  if (!d.isDirectory() || !d.canWrite()) {\n    LOG.warn(\"shared file descriptor path unusable: {}\", p);\n  }\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always configure multiple candidate paths ending with a guaranteed-writable directory.","Health-check /dev/shm (exists, writable, adequate size) in container entrypoints.","Monitor free space on the directories backing shared descriptors."],"tags":["short-circuit-reads","shared-memory","dev-shm","permissions","containers"],"backgroundTag":"tmp-dir-not-writable","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}