{"record":{"id":"4d352a3eaf326ab9","repo":"apache/dolphinscheduler","slug":"close-hadooputils-instance-failed","errorCode":null,"errorMessage":"Close HadoopUtils instance failed","messagePattern":"Close HadoopUtils instance failed","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"warning","filePath":"dolphinscheduler-storage-plugin/dolphinscheduler-storage-hdfs/src/main/java/org/apache/dolphinscheduler/plugin/storage/hdfs/HdfsStorageOperator.java","lineNumber":279,"sourceCode":"        return result;\n    }\n\n    @SneakyThrows\n    @Override\n    public StorageEntity getStorageEntity(String resourceAbsolutePath) {\n        exceptionIfPathEmpty(resourceAbsolutePath);\n        FileStatus fileStatus = fs.getFileStatus(new Path(resourceAbsolutePath));\n        return transformFileStatusToResourceMetadata(fileStatus);\n    }\n\n    @Override\n    public void close() throws IOException {\n        if (fs != null) {\n            try {\n                fs.close();\n            } catch (IOException e) {\n                log.error(\"Close HadoopUtils instance failed\", e);\n                throw new IOException(\"Close HadoopUtils instance failed\", e);\n            }\n        }\n    }\n\n    private StorageEntity transformFileStatusToResourceMetadata(FileStatus fileStatus) {\n        Path fileStatusPath = fileStatus.getPath();\n        String fileAbsolutePath = fileStatusPath.toString();\n        ResourceMetadata resourceMetaData = getResourceMetaData(fileAbsolutePath);\n        return StorageEntity.builder()\n                .fileName(fileStatusPath.getName())\n                .fullName(fileAbsolutePath)\n                .pfullName(resourceMetaData.getResourceParentAbsolutePath())\n                .type(resourceMetaData.getResourceType())\n                .isDirectory(fileStatus.isDirectory())\n                .size(fileStatus.getLen())\n                .relativePath(resourceMetaData.getResourceRelativePath())\n                .createTime(new Date(fileStatus.getModificationTime()))\n                .updateTime(new Date(fileStatus.getModificationTime()))","sourceCodeStart":261,"sourceCodeEnd":297,"githubUrl":"https://github.com/apache/dolphinscheduler/blob/02eac45a1b6676e639fcbfb4be2243de5771b05d/dolphinscheduler-storage-plugin/dolphinscheduler-storage-hdfs/src/main/java/org/apache/dolphinscheduler/plugin/storage/hdfs/HdfsStorageOperator.java#L261-L297","documentation":"HdfsStorageOperator.close closes the underlying Hadoop FileSystem handle. If fs.close() throws IOException, it logs and rethrows an IOException wrapping the original, so callers closing the operator (e.g., in try-with-resources cleanup) are notified the HDFS connection was not cleanly released.","triggerScenarios":"Calling close() when the HDFS client's close fails — typically broken connections to the NameNode, interrupted threads during close, or an already-faulted FileSystem instance.","commonSituations":"NameNode unreachable or network partitioned at shutdown; task thread interrupted during cleanup; Hadoop client/cluster version mismatch causing RPC errors on close.","solutions":["Inspect the wrapped cause (getCause()) to find the underlying HDFS/RPC failure","Verify NameNode connectivity and cluster health; retry the operation","Ensure close() is only called once per operator (use try-with-resources); update Hadoop client version to match the cluster if RPC errors persist"],"exampleFix":"// before\n} catch (IOException e) {\n    log.error(\"close failed\", e);\n}\n// after\ntry (HdfsStorageOperator hdfs = new HdfsStorageOperator(...)) {\n    hdfs.download(src, dst);\n} // IOException from close surfaces with cause intact\n","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n    hdfs.close();\n} catch (IOException e) {\n    log.error(\"HDFS close failed\", e.getCause()); // inspect underlying RPC/network cause\n}","preventionTips":["Use try-with-resources so close failures are surfaced once","Check NameNode reachability before assuming the operator is at fault","Avoid interrupting threads during cleanup; keep Hadoop client aligned with cluster version"],"tags":["hdfs","io","close","hadoop"],"backgroundTag":"file-close-failed","analyzedSha":"02eac45a1b6676e639fcbfb4be2243de5771b05d","analyzedAt":"2026-09-06T17:43:00.555Z","contentChangedAt":"2026-09-06T17:43:00.555Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}