apache/dolphinscheduler · error · TaskException

Download resource file: %s error

Error message

Download resource file: %s error

What it means

downloadResourcesIfNeeded throws TaskException when downloading a task resource file from storage (S3/HDFS/NFS) to the worker fails. The failure counter metric is incremented and the storage path is included in the message; the original exception is chained.

Source

Thrown at dolphinscheduler-worker/src/main/java/org/apache/dolphinscheduler/server/worker/utils/TaskExecutionContextUtils.java:99

            ResourceMetadata resourceMetaData = storageOperator.getResourceMetaData(resourceAbsolutePathInStorage);
            String resourceAbsolutePathInLocal =
                    Paths.get(taskWorkingDirectory, resourceMetaData.getResourceRelativePath()).toString();
            File file = new File(resourceAbsolutePathInLocal);
            if (!file.exists()) {
                try {
                    long resourceDownloadStartTime = System.currentTimeMillis();
                    storageOperator.download(resourceAbsolutePathInStorage, resourceAbsolutePathInLocal, true);
                    log.info("Download resource file {} -> {} successfully", resourceAbsolutePathInStorage,
                            resourceAbsolutePathInLocal);
                    FileUtils.setFileTo755(file);
                    WorkerServerMetrics
                            .recordWorkerResourceDownloadTime(System.currentTimeMillis() - resourceDownloadStartTime);
                    WorkerServerMetrics
                            .recordWorkerResourceDownloadSize(Files.size(Paths.get(resourceAbsolutePathInLocal)));
                    WorkerServerMetrics.incWorkerResourceDownloadSuccessCount();
                } catch (Exception ex) {
                    WorkerServerMetrics.incWorkerResourceDownloadFailureCount();
                    throw new TaskException(
                            String.format("Download resource file: %s error", resourceAbsolutePathInStorage), ex);
                }
            }
            ResourceContext.ResourceItem resourceItem = ResourceContext.ResourceItem.builder()
                    .resourceAbsolutePathInStorage(resourceAbsolutePathInStorage)
                    .resourceAbsolutePathInLocal(resourceAbsolutePathInLocal)
                    .build();
            resourceContext.addResourceItem(resourceItem);
        }
        return resourceContext;
    }

    public static void clearTaskInstanceWorkingDirectory(TaskExecutionContext taskExecutionContext) {
        final String execPath = taskExecutionContext.getExecutePath();
        try {
            if (StringUtils.isNotEmpty(execPath)) {
                FileUtils.deleteFile(execPath);
                log.info("Deleted task exec directory: {}", execPath);

View on GitHub (pinned to 02eac45a1b)

Solutions

1. Check the cause and confirm the resource exists at resourceAbsolutePathInStorage in the configured storage backend.
2. Verify worker storage configuration (access keys, endpoint, HDFS settings) and network connectivity to storage.
3. Re-upload the missing/failed resource via the resource center and rerun the task.
4. Check free disk space and write permissions on the worker's local resource directory.

Example fix

// before: resource removed from storage
storage: s3://bucket/resources/old-script.sh  // deleted
// after: re-upload resource or correct resourceDefinition path
storage: s3://bucket/resources/script.sh  // exists and readable

// verify resource exists before task run
boolean exists = storageOperator.exists(resourceAbsolutePathInStorage);
if (!exists) throw new IllegalStateException("resource missing in storage: " + resourceAbsolutePathInStorage);
if (Files.getUsableSpace(Paths.get(localDir)) < expectedSize) throw new IllegalStateException("insufficient disk");

try { download(); } catch (TaskException e) { if (isTransient(e.getCause())) retryWithBackoff(3); else throw e; }

When it happens

Trigger: StorageOperator.download of a resource to resourceAbsolutePathInLocal throws: resource missing in storage, storage credentials/network failure, or local disk write error during task resource download.

Common situations: Resource deleted from S3/HDFS after workflow definition, wrong storage configuration on worker, expired/misconfigured storage credentials, network interruption to storage service, insufficient local disk space.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

• Directory already exists: ${directoryAbsolutePath}
• The directory ${directoryAbsolutePath} already exists in the
• S3 does not support copying directories.
• The file ${dstPath} already exists in the bucket ${bucketNam
• Update the resource file from content: {fileAbsolutePath} fa

AI-assisted analysis of apache/dolphinscheduler@02eac45a1b (2026-09-06). Data as JSON: /api/errors/25b28a8490280886. Report an issue: GitHub.