What causes "wildcard_expandPath: on opendir %s: %s " in apache/hadoop?

Diagnose from the printed errno: ELOOP → fix the symlinks, EMFILE → raise ulimit -n, EIO → repair the mount

wildcard_expandPath: on opendir %s: %s

Error message

wildcard_expandPath: on opendir %s: %s

What it means

wildcard_expandPath deliberately tolerates opendir failures EACCES, ENOENT and ENOTDIR (such entries just expand to nothing). Any other errno — ELOOP from symlink loops, EMFILE/ENFILE fd exhaustion, EIO on network mounts — prints 'on opendir <path>: <strerror>' and sets length = -1, which fails classpath setup and can surface later as a misleading JVM-creation failure.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/jni_helper.c:453

                      filename, expanded);
#endif
                }
            }
        }

        if (errno != 0) {
            fprintf(stderr, "wildcard_expandPath: on readdir %s: %s\n",
              path, strerror(errno));
            length = -1;
        }

        if (closedir(dir) != 0) {
            fprintf(stderr, "wildcard_expandPath: on closedir %s: %s\n",
                    path, strerror(errno));
        }
    } else if ((errno != EACCES) && (errno != ENOENT) && (errno != ENOTDIR)) {
        // can not opendir due to an error we can not handle
        fprintf(stderr, "wildcard_expandPath: on opendir %s: %s\n", path,
                strerror(errno));
        length = -1;
    }

    if (length == 0) {
        // either we failed to open dir due to EACCESS, ENOENT, or ENOTDIR, or
        // we did not find any file that matches *.jar or *.JAR

#ifdef _LIBHDFS_JNI_HELPER_DEBUGGING_ON_
        fprintf(stderr, "wildcard_expandPath: can not expand %.*s*: %s\n",
                (int)(pathLength-1), path, strerror(errno));
#endif

        // in this case, the wildcard expansion is the same as the original
        // +1 for PATH_SEPARTOR or null termination
        length = pathLength + 1;
        if (expanded != NULL) {
            // pathLength includes an extra '.'

View on GitHub (pinned to 2add963021)

Solutions

Diagnose from the printed errno: ELOOP → fix the symlinks, EMFILE → raise ulimit -n, EIO → repair the mount
Pre-expand the CLASSPATH (hadoop classpath --glob) so no '*' directory is ever scanned
Ensure directories behind wildcard entries are plain readable directories

Example fix

# before
export CLASSPATH="/opt/hadoop/share/hadoop/common/lib/*"   # dir contains a symlink loop

# after
find /opt/hadoop -name '*.jar' -type f | tr '\n' ':'   # real files only, then export as CLASSPATH

Defensive patterns

Strategy: fallback

Validate before calling

/* preflight wildcard directories at startup, mapping errno to fixes */
static int wildcards_ok(const char *classpath) {
    /* for each ':' token containing '*', opendir it and require success or ENOENT/ENOTDIR/EACCES */
    return 1;  /* implement per token; fail startup on ELOOP/EMFILE/EIO */
}

Try / catch

/* shell: eliminate symlink loops and fd pressure before launch */
# find /opt/hadoop -name '*.jar' -type f | tr '\n' ':'   # real files only

Prevention

Avoid symlink-heavy wildcard classpath trees
Raise ulimit -n in service definitions
Mount the Hadoop installation locally rather than over flaky NFS

When it happens

Trigger: Symlink loops inside a wildcard classpath directory; running out of file descriptors when opendir is called; I/O errors on NFS-mounted classpath directories.

Common situations: Overly clever share/hadoop trees full of symlinks; containers with very low nofile limits; flaky network mounts beneath the Hadoop installation.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/0265e817122bae7f. Report an issue: GitHub.