apache/seatunnel · critical · IMapStorageException

Failed to get file system

Error message

Failed to get file system

What it means

IMapFileStorage.initialize() wraps any IOException from Hadoop's FileSystem.get(hadoopConf) into an IMapStorageException with this message. It means the storage layer could not obtain a handle to the configured HDFS (or local) file system, so the IMap write-ahead-log storage cannot start. The engine's create() path fails immediately.

Source

Thrown at seatunnel-engine/seatunnel-engine-storage/imap-storage-plugins/imap-storage-file/src/main/java/org/apache/seatunnel/engine/imap/storage/file/IMapFileStorage.java:159

        this.writDataTimeoutMilliseconds =
                (long)
                        configuration.getOrDefault(
                                WRITE_DATA_TIMEOUT_MILLISECONDS_KEY,
                                DEFAULT_WRITE_DATA_TIMEOUT_MILLISECONDS);

        this.region = String.valueOf(System.nanoTime());
        this.businessRootPath =
                namespace
                        + DEFAULT_IMAP_FILE_PATH_SPLIT
                        + clusterName
                        + DEFAULT_IMAP_FILE_PATH_SPLIT
                        + businessName
                        + DEFAULT_IMAP_FILE_PATH_SPLIT;
        try {
            this.fs = FileSystem.get(hadoopConf);
            fs.setWriteChecksum(false);
        } catch (IOException e) {
            throw new IMapStorageException("Failed to get file system", e);
        }
        this.serializer = new ProtoStuffSerializer();
        this.walDisruptor =
                new WALDisruptor(
                        fs,
                        FileConfiguration.valueOf(storageType.toUpperCase()),
                        businessRootPath + region + DEFAULT_IMAP_FILE_PATH_SPLIT,
                        serializer);
    }

    @Override
    public boolean store(Object key, Object value) {
        IMapFileData data;
        try {
            data = parseToIMapFileData(key, value);
        } catch (IOException e) {
            log.error("parse to IMapFileData error, key is {}, value is {}", key, value, e);
            return false;

View on GitHub (pinned to cf67b549a7)

Solutions

1. Verify the HDFS cluster / NameNode is running and reachable (hdfs dfsadmin -report).
2. Check fs.defaultFS / core-site.xml and hdfs-site.xml are on the classpath and HADOOP_CONF_DIR is set.
3. Confirm the storage root path URI in the engine config is correct and accessible by the running user.
4. If Kerberos is used, ensure login/authentication runs before FileSystem.get and the keytab/principal are valid.
5. Inspect the wrapped cause (e.getCause()) in the log for the underlying IOException detail.

Example fix

// before
this.fs = FileSystem.get(hadoopConf);
// after
try {
    this.fs = FileSystem.get(hadoopConf);
} catch (IOException e) {
    throw new IMapStorageException("Failed to get file system for path " + businessRootPath, e);
} // also validate connectivity before init: FileSystem.get(...).getStatus()

org.apache.hadoop.fs.FileSystem local = null;
try {
    local = org.apache.hadoop.fs.FileSystem.get(hadoopConf);
    local.getStatus(); // forces connectivity check
} catch (IOException e) {
    throw new IllegalStateException("HDFS unreachable: " + e.getMessage(), e);
}

boolean fsReachable(org.apache.hadoop.conf.Configuration conf) {
    try { org.apache.hadoop.fs.FileSystem.get(conf).getStatus(); return true; }
    catch (IOException e) { return false; }
}

try { storage.initialize(props); } catch (IMapStorageException e) { log.error("storage init failed", e.getCause()); /* abort start-up or fall back to local FS */ }

When it happens

Trigger: Calling IMapStorageFactory.create()/initialize() with a storage path whose HDFS NameNode is unreachable, an invalid fs.defaultFS URI, missing Hadoop configuration files (core-site.xml/hdfs-site.xml on classpath), or a local filesystem misconfiguration (e.g. file:// URI with bad path).

Common situations: Zeta engine start-up with imap file storage in a cluster where HDFS is down or HA nameservice misconfigured; missing HADOOP_CONF_DIR; user lacks Kerberos ticket before FileSystem.get; wrong hadoop dependency version shading conflicts.

Related errors

• Failed to list files from names${path}
• load all data error
• load all keys error parent path is ${businessRootPath}
• Circular condition chain detected: '%s' already exists in th
• Condition for option '%s' has a null operator

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/b4425fbb485cad70. Report an issue: GitHub.