{"record":{"id":"ea3cf1f501588d23","repo":"apache/seatunnel","slug":"load-all-data-error","errorCode":null,"errorMessage":"load all data error","messagePattern":"load all data error","errorType":"exception","errorClass":"IMapStorageException","httpStatus":null,"severity":"error","filePath":"seatunnel-engine/seatunnel-engine-storage/imap-storage-plugins/imap-storage-file/src/main/java/org/apache/seatunnel/engine/imap/storage/file/IMapFileStorage.java","lineNumber":240,"sourceCode":"                        IMapFileData data = buildDeleteIMapFileData(key);\n                        long requestId = sendToDisruptorQueue(data, WALEventType.APPEND);\n                        walDisruptor.tryAppendPublish(data, requestId);\n                        requestMap.put(requestId, data);\n                    } catch (IOException e) {\n                        log.error(\"parse to IMapFileData error\", e);\n                        failures.add(key);\n                    }\n                });\n        return batchQueryExecuteFailsStatus(requestMap, failures);\n    }\n\n    @Override\n    public Map<Object, Object> loadAll() {\n        try {\n            WALReader reader = new WALReader(fs, fileConfiguration, serializer);\n            return reader.loadAllData(new Path(businessRootPath), new HashSet<>());\n        } catch (IOException e) {\n            throw new IMapStorageException(\"load all data error\", e);\n        }\n    }\n\n    @Override\n    public Set<Object> loadAllKeys() {\n        try {\n            WALReader reader = new WALReader(fs, fileConfiguration, serializer);\n            return reader.loadAllKeys(new Path(businessRootPath));\n        } catch (IOException e) {\n            throw new IMapStorageException(\n                    e, \"load all keys error parent path is {}\", e, businessRootPath);\n        }\n    }\n\n    @Override\n    public void destroy(boolean deleteAllFileFlag) {\n        log.info(\n                \"start destroy IMapFileStorage, businessName is {}, cluster name is {}\",","sourceCodeStart":222,"sourceCodeEnd":258,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-engine/seatunnel-engine-storage/imap-storage-plugins/imap-storage-file/src/main/java/org/apache/seatunnel/engine/imap/storage/file/IMapFileStorage.java#L222-L258","documentation":"loadAll() reads the full WAL data tree under businessRootPath via WALReader.loadAllData; any IOException during reading (missing directory, corrupt file, NameNode/network failure) is rethrown as this IMapStorageException. The in-memory map cannot be fully rebuilt.","triggerScenarios":"Calling IMapFileStorage.loadAll() when businessRootPath does not exist or was deleted, WAL segment files are corrupted, HDFS is intermittently unreachable, or permission to read the WAL directory is missing.","commonSituations":"Manual cleanup of the seatunnel storage directory while a cluster references it; HDFS under replication/decommission causing read failures; disk full on datanodes; restoring storage to a different cluster with a different namespace.","solutions":["Check that businessRootPath exists and is readable (hdfs dfs -ls <path>).","Verify WAL files are not truncated/corrupt; remove or archive damaged segments after stopping the cluster.","Check NameNode/datanode health and network connectivity to HDFS.","Ensure the running user has read permission on the storage directory.","Retry after transient HDFS failures; if data is disposable, call destroy(true) and re-initialize."],"exampleFix":"// before\ntry {\n    return reader.loadAllData(new Path(businessRootPath), new HashSet<>());\n} catch (IOException e) {\n    throw new IMapStorageException(\"load all data error\", e);\n}\n// after\ntry {\n    return reader.loadAllData(new Path(businessRootPath), new HashSet<>());\n} catch (IOException e) {\n    throw new IMapStorageException(e, \"load all data error for path {}\", businessRootPath);\n}","handlingStrategy":"try-catch","validationCode":"org.apache.hadoop.fs.Path root = new org.apache.hadoop.fs.Path(businessRootPath);\nboolean readable = fs.exists(root) && fs.getFileStatus(root).getPermission().getUserAction().implies(org.apache.hadoop.fs.permission.FsAction.READ);","typeGuard":null,"tryCatchPattern":"try { Map<Object,Object> data = storage.loadAll(); } catch (IMapStorageException e) { log.error(\"loadAll failed for {}\", businessRootPath, e); /* retry with backoff or re-init storage */ }","preventionTips":["Do not delete the storage directory while the cluster exists","Monitor HDFS datanode/disk health","Verify permissions on the storage root after user/realm changes","Run hdfs fsck periodically on the storage path"],"tags":["hdfs","io","storage","read-failure"],"backgroundTag":"file-read-failed","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}