{"record":{"id":"072af4d9fcc97803","repo":"apache/hadoop","slug":"wildcard-expandpath-on-readdir-s-s","errorCode":null,"errorMessage":"wildcard_expandPath: on readdir %s: %s\n","messagePattern":"wildcard_expandPath: on readdir (.+?): (.+?)\n","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/jni_helper.c","lineNumber":442,"sourceCode":"                if (expanded != NULL) {\n                    // pathLength includes an extra '.'\n                    memcpy(dest, path, pathLength - 1);\n                    dest += pathLength - 1;\n                    memcpy(dest, filename, filenameLength);\n                    dest += filenameLength;\n                    *dest = PATH_SEPARATOR;\n                    dest++;\n\n#ifdef _LIBHDFS_JNI_HELPER_DEBUGGING_ON_\n                    printf(\"wildcard_expandPath:\\t%s\\t:\\t%s\\n\",\n                      filename, expanded);\n#endif\n                }\n            }\n        }\n\n        if (errno != 0) {\n            fprintf(stderr, \"wildcard_expandPath: on readdir %s: %s\\n\",\n              path, strerror(errno));\n            length = -1;\n        }\n\n        if (closedir(dir) != 0) {\n            fprintf(stderr, \"wildcard_expandPath: on closedir %s: %s\\n\",\n                    path, strerror(errno));\n        }\n    } else if ((errno != EACCES) && (errno != ENOENT) && (errno != ENOTDIR)) {\n        // can not opendir due to an error we can not handle\n        fprintf(stderr, \"wildcard_expandPath: on opendir %s: %s\\n\", path,\n                strerror(errno));\n        length = -1;\n    }\n\n    if (length == 0) {\n        // either we failed to open dir due to EACCESS, ENOENT, or ENOTDIR, or\n        // we did not find any file that matches *.jar or *.JAR","sourceCodeStart":424,"sourceCodeEnd":460,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/jni_helper.c#L424-L460","documentation":"While expanding wildcard CLASSPATH entries, wildcard_expandPath scans each directory with opendir/readdir. If errno is non-zero after the scan loop (a readdir failed mid-iteration), it prints 'on readdir <path>: <strerror>' and forces length = -1, failing expansion of that entry and ultimately classpath setup.","triggerScenarios":"Directory contents or permissions changing while the directory is being scanned; NFS/network-mounted classpath directories returning transient errors; EMFILE from running out of file descriptors during the scan.","commonSituations":"Wildcard classpath entries over shared or network mounts; concurrent jar upgrades writing into the same Hadoop install tree; long-running processes with fd leaks.","solutions":["Check stability and read permissions of the directory named in the message","Replace wildcard classpath entries with explicit jar lists (no '*' tokens) so expansion is skipped entirely","Raise fd limits (ulimit -n) if the underlying error is EMFILE"],"exampleFix":"# before\nexport CLASSPATH=\"/opt/hadoop/share/hadoop/common/*\"\n\n# after (pre-expanded, no directory scan)\nexport CLASSPATH=\"$(/opt/hadoop/bin/hadoop classpath --glob)\"","handlingStrategy":"fallback","validationCode":"/* preflight each wildcard directory before JVM init */\nstatic int dir_scannable(const char *dir) {\n    DIR *d = opendir(dir);\n    if (!d) return 0;\n    struct dirent *e;\n    while ((e = readdir(d)) != NULL) { /* touch entries */ }\n    int failed = (e == NULL && errno != 0);\n    closedir(d);\n    return !failed;\n}","typeGuard":null,"tryCatchPattern":"/* shell-level: sidestep the scan entirely when directory access is unreliable */\n# export CLASSPATH=\"$(hadoop classpath --glob)\"","preventionTips":["Prefer pre-expanded CLASSPATHs over wildcard entries","Keep classpath directories stable and locally mounted","Watch fd usage (ulimit -n) in processes with big classpaths"],"tags":["hdfs","libhdfs","classpath","readdir","wildcard-expansion"],"backgroundTag":"classpath-expansion-failure","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}