{"record":{"id":"3573313767ee25ec","repo":"apache/flink","slug":"an-io-error-occurred-while-accessing-the-staging-f","errorCode":null,"errorMessage":"An IO error occurred while accessing the staging FileSystem.","messagePattern":"An IO error occurred while accessing the staging FileSystem\\.","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table/FileSystemOutputFormat.java","lineNumber":115,"sourceCode":"        this.formatFactory = formatFactory;\n        this.computer = computer;\n        this.outputFileConfig = outputFileConfig;\n        this.identifier = identifier;\n        this.partitionCommitPolicyFactory = partitionCommitPolicyFactory;\n\n        createStagingDirectory(this.stagingPath);\n    }\n\n    private static void createStagingDirectory(Path stagingPath) {\n        try {\n            final FileSystem stagingFileSystem = stagingPath.getFileSystem();\n            Preconditions.checkState(\n                    !stagingFileSystem.exists(stagingPath),\n                    \"Staging dir %s already exists\",\n                    stagingPath);\n            stagingFileSystem.mkdirs(stagingPath);\n        } catch (IOException e) {\n            throw new RuntimeException(\n                    \"An IO error occurred while accessing the staging FileSystem.\", e);\n        }\n    }\n\n    @Override\n    public void finalizeGlobal(FinalizationContext context) {\n        try {\n            List<PartitionCommitPolicy> policies = Collections.emptyList();\n            if (partitionCommitPolicyFactory != null) {\n                policies =\n                        partitionCommitPolicyFactory.createPolicyChain(\n                                Thread.currentThread().getContextClassLoader(),\n                                () -> {\n                                    try {\n                                        return fsFactory.create(stagingPath.toUri());\n                                    } catch (IOException e) {\n                                        throw new RuntimeException(e);\n                                    }","sourceCodeStart":97,"sourceCodeEnd":133,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table/FileSystemOutputFormat.java#L97-L133","documentation":"FileSystemOutputFormat.createStagingDirectory creates a temporary staging directory for output. It checks the staging path does not already exist, then calls mkdirs. If any IOException occurs during this process (file system access failure, permissions, path already exists race, network error to remote FS), it wraps the exception in a RuntimeException.","triggerScenarios":"The staging directory path already exists (Preconditions.checkState fails but that throws IllegalStateException, not IOException — the RuntimeException catch is for the getFileSystem, exists, or mkdirs calls failing). Remote file system connectivity issues (HDFS down, S3 timeout). Permission denied on the parent directory. Invalid or unreachable staging path URI.","commonSituations":"HDFS or S3 connection failure when the job runs. Staging path on a filesystem where the user lacks write permission. Stale staging directory from a previous failed run (though checkState catches this first with a different error). Network partition to the distributed file system.","solutions":["Verify the staging directory path is reachable and writable: check file system connectivity and permissions.","Ensure no stale staging directories from previous runs conflict; clean up old staging paths.","If using HDFS/S3, verify the file system configuration (fs.defaultFS, credentials) is correct in the job's configuration.","Check that the parent directory of the staging path exists and is writable."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// Verify staging path before creating the output format\nFileSystem fs = stagingPath.getFileSystem();\nif (fs.exists(stagingPath)) {\n    throw new IllegalStateException(\"Staging path already exists: \" + stagingPath);\n}\nif (!fs.exists(stagingPath.getParent())) {\n    throw new IllegalStateException(\"Parent directory does not exist: \" + stagingPath.getParent());\n}","typeGuard":null,"tryCatchPattern":"try {\n    outputFormat.open(...);\n} catch (RuntimeException e) {\n    if (e.getMessage().contains(\"staging FileSystem\")) {\n        // Check file system connectivity and permissions\n        throw new RuntimeException(\"Staging directory creation failed. Check FS connectivity.\", e);\n    }\n    throw e;\n}","preventionTips":["Verify the output file system is reachable before submitting the job.","Ensure the staging path parent directory exists and is writable.","Clean up stale staging directories from previous failed runs.","Test file system connectivity from the JobManager/TaskManager nodes."],"tags":["output-format","staging-directory","io-error","filesystem","flink"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}