{"record":{"id":"b421b65aee5a5616","repo":"apache/flink","slug":"an-error-occurred-while-copying-the-file","errorCode":null,"errorMessage":"An error occurred while copying the file.","messagePattern":"An error occurred while copying the file\\.","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"flink-core/src/main/java/org/apache/flink/api/common/cache/DistributedCache.java","lineNumber":162,"sourceCode":"        if (name == null) {\n            throw new NullPointerException(\"name must not be null\");\n        }\n\n        Future<Path> future = cacheCopyTasks.get(name);\n        if (future == null) {\n            throw new IllegalArgumentException(\n                    \"File with name '\"\n                            + name\n                            + \"' is not available.\"\n                            + \" Did you forget to register the file?\");\n        }\n\n        try {\n            final Path path = future.get();\n            URI tmp = path.makeQualified(path.getFileSystem()).toUri();\n            return new File(tmp);\n        } catch (ExecutionException e) {\n            throw new RuntimeException(\"An error occurred while copying the file.\", e.getCause());\n        } catch (Exception e) {\n            throw new RuntimeException(\n                    \"Error while getting the file registered under '\"\n                            + name\n                            + \"' from the distributed cache\",\n                    e);\n        }\n    }\n\n    // ------------------------------------------------------------------------\n    //  Utilities to read/write cache files from/to the configuration\n    // ------------------------------------------------------------------------\n\n    public static void writeFileInfoToConfig(\n            String name, DistributedCacheEntry e, Configuration conf) {\n        int num = conf.get(getIntConfigOption(CACHE_FILE_NUM), 0) + 1;\n        conf.set(getIntConfigOption(CACHE_FILE_NUM), num);\n        conf.setString(CACHE_FILE_NAME + num, name);","sourceCodeStart":144,"sourceCodeEnd":180,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-core/src/main/java/org/apache/flink/api/common/cache/DistributedCache.java#L144-L180","documentation":"Thrown by DistributedCache when fetching a registered cached file fails during the background copy from the distributed filesystem to local storage. The RuntimeException wraps the cause of an ExecutionException from a Future.get() call, meaning the asynchronous file-copy task threw an exception. Common root causes include network connectivity issues to HDFS/S3, permission denials, the file not existing at the registered path, or disk-full conditions on the local TaskManager.","triggerScenarios":"A user function calls RuntimeContext.getDistributedCache().getFile(name) for a file registered via DistributedCache.registerFileWithClasspathResolve() or ExecutionEnvironment.registerCachedFile(). The underlying future that copies the remote file to local temp storage fails, and ExecutionException is caught and re-thrown as this RuntimeException.","commonSituations":"The cached file path is wrong or was deleted after registration. The TaskManager lacks read permissions on HDFS/S3. Network partition between the TaskManager and the NameNode/object store. Local disk on the TaskManager is full or the temp directory is unwritable.","solutions":["Check the getCause() of this RuntimeException to see the original failure (e.g. FileNotFoundException, AccessControlException, IOException) and address that specific issue.","Verify the file path registered with registerCachedFile() actually exists and is readable by the TaskManager's service account.","Ensure the TaskManager has sufficient free disk space and write permission on its configured temp directory (taskmanager.tmp.dirs / java.io.tmpdir).","If using HDFS/S3, confirm connectivity and credentials from the TaskManager node (not just the client)."],"exampleFix":"// before\nFile cached = getRuntimeContext().getDistributedCache().getFile(\"model\");\n\n// after — inspect the root cause before retrying\ntry {\n    File cached = getRuntimeContext().getDistributedCache().getFile(\"model\");\n} catch (RuntimeException e) {\n    Throwable root = e.getCause() != null ? e.getCause() : e;\n    log.error(\"Distributed cache copy failed for 'model': {}\", root.getMessage());\n    throw e;\n}","handlingStrategy":"try-catch","validationCode":"// Before calling getFile, verify the file is registered\nDistributedCache cache = getRuntimeContext().getDistributedCache();\n// No public API to check registration; wrap in try-catch instead","typeGuard":null,"tryCatchPattern":"try {\n    File cachedFile = getRuntimeContext().getDistributedCache().getFile(\"myFile\");\n} catch (RuntimeException e) {\n    Throwable cause = e.getCause() != null ? e.getCause() : e;\n    log.error(\"Failed to fetch distributed cache file: {}\", cause.getMessage(), cause);\n    throw e;\n}","preventionTips":["Verify cached file paths exist and are accessible from TaskManager nodes before deploying.","Use small files (< 100MB) for distributed cache to minimize copy failure window.","Monitor TaskManager disk space to ensure temp directories have capacity.","Test file accessibility from a TaskManager node using the same service account."],"tags":["distributed-cache","filesystem","io","taskmanager"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}