{"record":{"id":"0265e817122bae7f","repo":"apache/hadoop","slug":"wildcard-expandpath-on-opendir-s-s","errorCode":null,"errorMessage":"wildcard_expandPath: on opendir %s: %s\n","messagePattern":"wildcard_expandPath: on opendir (.+?): (.+?)\n","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/jni_helper.c","lineNumber":453,"sourceCode":"                      filename, expanded);\n#endif\n                }\n            }\n        }\n\n        if (errno != 0) {\n            fprintf(stderr, \"wildcard_expandPath: on readdir %s: %s\\n\",\n              path, strerror(errno));\n            length = -1;\n        }\n\n        if (closedir(dir) != 0) {\n            fprintf(stderr, \"wildcard_expandPath: on closedir %s: %s\\n\",\n                    path, strerror(errno));\n        }\n    } else if ((errno != EACCES) && (errno != ENOENT) && (errno != ENOTDIR)) {\n        // can not opendir due to an error we can not handle\n        fprintf(stderr, \"wildcard_expandPath: on opendir %s: %s\\n\", path,\n                strerror(errno));\n        length = -1;\n    }\n\n    if (length == 0) {\n        // either we failed to open dir due to EACCESS, ENOENT, or ENOTDIR, or\n        // we did not find any file that matches *.jar or *.JAR\n\n#ifdef _LIBHDFS_JNI_HELPER_DEBUGGING_ON_\n        fprintf(stderr, \"wildcard_expandPath: can not expand %.*s*: %s\\n\",\n                (int)(pathLength-1), path, strerror(errno));\n#endif\n\n        // in this case, the wildcard expansion is the same as the original\n        // +1 for PATH_SEPARTOR or null termination\n        length = pathLength + 1;\n        if (expanded != NULL) {\n            // pathLength includes an extra '.'","sourceCodeStart":435,"sourceCodeEnd":471,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/jni_helper.c#L435-L471","documentation":"wildcard_expandPath deliberately tolerates opendir failures EACCES, ENOENT and ENOTDIR (such entries just expand to nothing). Any other errno — ELOOP from symlink loops, EMFILE/ENFILE fd exhaustion, EIO on network mounts — prints 'on opendir <path>: <strerror>' and sets length = -1, which fails classpath setup and can surface later as a misleading JVM-creation failure.","triggerScenarios":"Symlink loops inside a wildcard classpath directory; running out of file descriptors when opendir is called; I/O errors on NFS-mounted classpath directories.","commonSituations":"Overly clever share/hadoop trees full of symlinks; containers with very low nofile limits; flaky network mounts beneath the Hadoop installation.","solutions":["Diagnose from the printed errno: ELOOP → fix the symlinks, EMFILE → raise ulimit -n, EIO → repair the mount","Pre-expand the CLASSPATH (hadoop classpath --glob) so no '*' directory is ever scanned","Ensure directories behind wildcard entries are plain readable directories"],"exampleFix":"# before\nexport CLASSPATH=\"/opt/hadoop/share/hadoop/common/lib/*\"   # dir contains a symlink loop\n\n# after\nfind /opt/hadoop -name '*.jar' -type f | tr '\\n' ':'   # real files only, then export as CLASSPATH","handlingStrategy":"fallback","validationCode":"/* preflight wildcard directories at startup, mapping errno to fixes */\nstatic int wildcards_ok(const char *classpath) {\n    /* for each ':' token containing '*', opendir it and require success or ENOENT/ENOTDIR/EACCES */\n    return 1;  /* implement per token; fail startup on ELOOP/EMFILE/EIO */\n}","typeGuard":null,"tryCatchPattern":"/* shell: eliminate symlink loops and fd pressure before launch */\n# find /opt/hadoop -name '*.jar' -type f | tr '\\n' ':'   # real files only","preventionTips":["Avoid symlink-heavy wildcard classpath trees","Raise ulimit -n in service definitions","Mount the Hadoop installation locally rather than over flaky NFS"],"tags":["hdfs","libhdfs","classpath","opendir","wildcard-expansion"],"backgroundTag":"classpath-expansion-failure","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}