{"record":{"id":"25b28a8490280886","repo":"apache/dolphinscheduler","slug":"download-resource-file-s-error","errorCode":null,"errorMessage":"Download resource file: %s error","messagePattern":"Download resource file: (.+?) error","errorType":"exception","errorClass":"TaskException","httpStatus":null,"severity":"error","filePath":"dolphinscheduler-worker/src/main/java/org/apache/dolphinscheduler/server/worker/utils/TaskExecutionContextUtils.java","lineNumber":99,"sourceCode":"            ResourceMetadata resourceMetaData = storageOperator.getResourceMetaData(resourceAbsolutePathInStorage);\n            String resourceAbsolutePathInLocal =\n                    Paths.get(taskWorkingDirectory, resourceMetaData.getResourceRelativePath()).toString();\n            File file = new File(resourceAbsolutePathInLocal);\n            if (!file.exists()) {\n                try {\n                    long resourceDownloadStartTime = System.currentTimeMillis();\n                    storageOperator.download(resourceAbsolutePathInStorage, resourceAbsolutePathInLocal, true);\n                    log.info(\"Download resource file {} -> {} successfully\", resourceAbsolutePathInStorage,\n                            resourceAbsolutePathInLocal);\n                    FileUtils.setFileTo755(file);\n                    WorkerServerMetrics\n                            .recordWorkerResourceDownloadTime(System.currentTimeMillis() - resourceDownloadStartTime);\n                    WorkerServerMetrics\n                            .recordWorkerResourceDownloadSize(Files.size(Paths.get(resourceAbsolutePathInLocal)));\n                    WorkerServerMetrics.incWorkerResourceDownloadSuccessCount();\n                } catch (Exception ex) {\n                    WorkerServerMetrics.incWorkerResourceDownloadFailureCount();\n                    throw new TaskException(\n                            String.format(\"Download resource file: %s error\", resourceAbsolutePathInStorage), ex);\n                }\n            }\n            ResourceContext.ResourceItem resourceItem = ResourceContext.ResourceItem.builder()\n                    .resourceAbsolutePathInStorage(resourceAbsolutePathInStorage)\n                    .resourceAbsolutePathInLocal(resourceAbsolutePathInLocal)\n                    .build();\n            resourceContext.addResourceItem(resourceItem);\n        }\n        return resourceContext;\n    }\n\n    public static void clearTaskInstanceWorkingDirectory(TaskExecutionContext taskExecutionContext) {\n        final String execPath = taskExecutionContext.getExecutePath();\n        try {\n            if (StringUtils.isNotEmpty(execPath)) {\n                FileUtils.deleteFile(execPath);\n                log.info(\"Deleted task exec directory: {}\", execPath);","sourceCodeStart":81,"sourceCodeEnd":117,"githubUrl":"https://github.com/apache/dolphinscheduler/blob/02eac45a1b6676e639fcbfb4be2243de5771b05d/dolphinscheduler-worker/src/main/java/org/apache/dolphinscheduler/server/worker/utils/TaskExecutionContextUtils.java#L81-L117","documentation":"downloadResourcesIfNeeded throws TaskException when downloading a task resource file from storage (S3/HDFS/NFS) to the worker fails. The failure counter metric is incremented and the storage path is included in the message; the original exception is chained.","triggerScenarios":"StorageOperator.download of a resource to resourceAbsolutePathInLocal throws: resource missing in storage, storage credentials/network failure, or local disk write error during task resource download.","commonSituations":"Resource deleted from S3/HDFS after workflow definition, wrong storage configuration on worker, expired/misconfigured storage credentials, network interruption to storage service, insufficient local disk space.","solutions":["Check the cause and confirm the resource exists at resourceAbsolutePathInStorage in the configured storage backend.","Verify worker storage configuration (access keys, endpoint, HDFS settings) and network connectivity to storage.","Re-upload the missing/failed resource via the resource center and rerun the task.","Check free disk space and write permissions on the worker's local resource directory."],"exampleFix":"// before: resource removed from storage\nstorage: s3://bucket/resources/old-script.sh  // deleted\n// after: re-upload resource or correct resourceDefinition path\nstorage: s3://bucket/resources/script.sh  // exists and readable","handlingStrategy":"retry","validationCode":"// verify resource exists before task run\nboolean exists = storageOperator.exists(resourceAbsolutePathInStorage);\nif (!exists) throw new IllegalStateException(\"resource missing in storage: \" + resourceAbsolutePathInStorage);\nif (Files.getUsableSpace(Paths.get(localDir)) < expectedSize) throw new IllegalStateException(\"insufficient disk\");","typeGuard":null,"tryCatchPattern":"try { download(); } catch (TaskException e) { if (isTransient(e.getCause())) retryWithBackoff(3); else throw e; }","preventionTips":["Verify resources exist in storage before scheduling workflows","Keep storage credentials current in worker config","Monitor worker->storage network paths","Check worker disk free space regularly"],"tags":["storage","resource-download","s3","hdfs","worker"],"backgroundTag":"file-not-found","analyzedSha":"02eac45a1b6676e639fcbfb4be2243de5771b05d","analyzedAt":"2026-09-06T17:43:00.555Z","contentChangedAt":"2026-09-06T17:43:00.555Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}