apache/kafka · error · ConfigException

Could not list directory {dir}

Error message

Could not list directory {dir}

What it means

Thrown as ConfigException by DirectoryConfigProvider.get() when Files.list(dir) raises an IOException while streaming the contents of a directory that already passed the allowed-paths and isDirectory() checks. The exception is logged with full stack trace at ERROR and then rethrown without the cause so callers see a clean message naming the directory.

Source

Thrown at clients/src/main/java/org/apache/kafka/common/config/provider/DirectoryConfigProvider.java:111

        if (path != null && !path.isEmpty()) {
            Path dir = allowedPaths.parseUntrustedPath(path);
            if (dir == null) {
                log.warn("The path {} is not allowed to be accessed", path);
                return new ConfigData(map);
            }

            if (!Files.isDirectory(dir)) {
                log.warn("The path {} is not a directory", path);
            } else {
                try (Stream<Path> stream = Files.list(dir)) {
                    map = stream
                        .filter(fileFilter)
                        .collect(Collectors.toMap(
                            p -> p.getFileName().toString(),
                            p -> read(p)));
                } catch (IOException e) {
                    log.error("Could not list directory {}", dir, e);
                    throw new ConfigException("Could not list directory " + dir);
                }
            }
        }
        return new ConfigData(map);
    }

    private static String read(Path path) {
        try {
            return Files.readString(path);
        } catch (IOException e) {
            log.error("Could not read file {} for property {}", path, path.getFileName(), e);
            throw new ConfigException("Could not read file " + path + " for property " + path.getFileName());
        }
    }

}

View on GitHub (pinned to c31c9215e1)

Solutions

1. Check the ERROR log line 'Could not list directory <dir>' which carries the underlying IOException stack trace for the real cause.
2. Confirm read+execute permissions for the JVM user on the directory: ls -ld <dir> && sudo -u <kafka-user> ls <dir>.
3. For container/network mounts, verify the mount is healthy and stable during provider access.
4. Review SELinux/AppManager audit logs and add allow rules for the Kafka process on the directory.

Example fix

# before
chmod 700 /etc/kafka/secrets   # kafka user cannot list

# after
chown kafka:kafka /etc/kafka/secrets
chmod 750 /etc/kafka/secrets

// Pre-check directory accessibility before provider.get(path):
Path dir = Paths.get(path);
if (!Files.isDirectory(dir)) {
    throw new IllegalArgumentException(path + " is not a directory");
}
if (!Files.isReadable(dir)) {
    throw new IllegalArgumentException(path + " is not readable");
}

try {
    provider.get(path);
} catch (ConfigException e) {
    if (e.getMessage().startsWith("Could not list directory")) {
        // log dir, check perms, degrade to empty ConfigData, or rethrow as app error
    } else { throw e; }
}

When it happens

Trigger: DirectoryConfigProvider.get(path) where path resolves to a real, allowed directory, but listing it fails with IOException (e.g. FileSystemException, AccessDeniedException, ClosedFileSystemException). The listing occurs inside a try-with-resources on the Stream from Files.list.

Common situations: JVM loses read permission on the directory between the isDirectory check and the list call (race with chmod/chown). Network filesystem (NFS/CIFS) hiccup mid-listing. Container filesystem where the directory is a mount that gets unmounted concurrently. SELinux denial on readdir.

Related errors

• Could not read file {path} for property {fileName}
• Path normalisedPath does not exist
• Path normalisedPath could not be resolved
• The provider has not been configured yet.
• Failed to check internal API usage: {errorMessage}

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/46549d36900c1c25.json. Report an issue: GitHub.