apache/dolphinscheduler · error · IllegalArgumentException

Can not find valid resource by name %s

Error message

Can not find valid resource by name %s

What it means

getResourcesFileInfo filters loaded resource components by their fullName; when no resource's full name equals the requested name, it throws IllegalArgumentException. Full name in DolphinScheduler is typically the path-like identifier of the resource in the resource center.

Source

Thrown at dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/python/PythonGateway.java:598

    /**
     * Get resource by given program type and full name. It returns map contain resource id, name.
     * Useful in Python API create flink or spark task which need workflow information.
     *
     * @param fullName    full name of the resource
     */
    public Map<String, Object> getResourcesFileInfo(String fullName) {
        Map<String, Object> result = new HashMap<>();

        List<ResourceComponent> resourceComponents =
                resourceService.queryResourceFiles(dummyAdminUser, ResourceType.FILE);
        List<ResourceComponent> namedResources = resourceComponents.stream()
                .filter(s -> fullName.equals(s.getFullName()))
                .collect(Collectors.toList());
        if (CollectionUtils.isEmpty(namedResources)) {
            String msg = String.format("Can not find valid resource by name %s", fullName);
            log.error(msg);
            throw new IllegalArgumentException(msg);
        }

        result.put("name", namedResources.get(0).getName());
        return result;
    }

    /**
     * Get environment info by given environment name. It return environment code.
     * Useful in Python API create task which need environment information.
     *
     * @param environmentName name of the environment
     */
    public Long getEnvironmentInfo(String environmentName) {
        try {
            return environmentService.queryEnvironmentByName(environmentName).getCode();
        } catch (ServiceException e) {
            String msg = String.format("Can not find valid environment by name %s", environmentName);
            log.error(msg);

View on GitHub (pinned to 02eac45a1b)

Solutions

  1. Check the resource's full path in the Resource center and use it exactly (including directory prefix, e.g. /scripts/job.py)
  2. Upload the missing resource before running the Python script
  3. Normalize the fullName (leading slash, no double slashes) to match storage convention
  4. Re-sync resource storage backend if the resource exists in the UI but not the underlying store

Example fix

// before (Python)
task = Shell(..., resource_list=["job.sh"])
// after
task = Shell(..., resource_list=["/scripts/job.sh"])  # full path as in resource center
Defensive patterns

Strategy: validation

Validate before calling

String fullName = "/scripts/job.sh";
boolean exists = resourceCenter.listAllResources().stream()
    .anyMatch(r -> fullName.equals(r.getFullName()));
if (!exists) {
    throw new IllegalStateException("Resource not found: " + fullName);
}

Try / catch

try {
    pythonGateway.getResourcesFileInfo(fullName);
} catch (IllegalArgumentException e) {
    log.error("Resource missing: {}", e.getMessage());
}

Prevention

When it happens

Trigger: Calling getResourcesFileInfo for a resource whose fullName (e.g. directory path + filename) does not match any resource in the resource center.

Common situations: Python task referencing a file/script that was uploaded with a different path; resource deleted; leading/trailing slash or missing directory prefix in the full name; resource center storage migrated (HDFS/S3) and contents out of sync.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

AI-assisted analysis of apache/dolphinscheduler@02eac45a1b (2026-09-06). Data as JSON: /api/errors/23631ccc0978cd88. Report an issue: GitHub.