apache/dolphinscheduler · error · TaskException

DataX job definition is missing, provide inline json or atta

Error message

DataX job definition is missing, provide inline json or attach exactly one .json resource file

What it means

buildDataxJsonFile() requires a DataX job definition from one of two sources: inline json in task params, or exactly one attached .json resource file. When isInlineJsonAbsent() is true (no inline json, or UI empty-object placeholder) and no dedicated .json job resource exists (getJobDefinitionResource() returns null), the task cannot build its json file and throws. Per issue #18389, auxiliary resources like keytab/xml files do not count as the job definition.

Source

Thrown at dolphinscheduler-task-plugin/dolphinscheduler-task-datax/src/main/java/org/apache/dolphinscheduler/plugin/task/datax/DataxTask.java:211

        String fileName = String.format("%s/%s_job.json", taskRequest.getExecutePath(), taskRequest.getTaskAppId());
        String json;

        Path path = new File(fileName).toPath();
        if (Files.exists(path)) {
            return fileName;
        }

        if (dataXParameters.getCustomConfig() == Flag.YES.ordinal()) {
            // An attached resource file is a valid way to supply the job definition. Without
            // this branch the worker downloads the resource but the plugin runs with the empty
            // inline json and the job fails (issue #18389). Existing tasks created through the
            // UI carry an empty object placeholder, treat it the same as no inline json.
            if (dataXParameters.isInlineJsonAbsent()) {
                // the job definition is the single attached .json resource, never the first
                // entry in resourceList, which may be an auxiliary keytab or xml (issue #18389)
                ResourceInfo jobResource = dataXParameters.getJobDefinitionResource();
                if (jobResource == null) {
                    throw new TaskException(
                            "DataX job definition is missing, provide inline json or attach exactly one .json resource file");
                }
                json = readJsonFromResourceFile(jobResource);
            } else {
                json = dataXParameters.getJson();
            }
            json = json.replaceAll("\\r\\n", System.lineSeparator());
        } else {
            ObjectNode job = JSONUtils.createObjectNode();
            job.putArray("content").addAll(buildDataxJobContentJson());
            job.set("setting", buildDataxJobSettingJson());

            ObjectNode root = JSONUtils.createObjectNode();
            root.set("job", job);
            root.set("core", buildDataxCoreJson());
            json = root.toString();
        }

View on GitHub (pinned to 02eac45a1b)

Solutions

1. Paste a complete DataX job json into the task's custom template (inline json) field, or
2. Attach exactly one .json resource file containing the DataX job definition.
3. If auxiliary files (keytab/xml) are needed, attach the .json job file in addition — it must be the sole .json resource.
4. Validate the task params in the UI before saving so the empty-object placeholder is not submitted.

Example fix

// before: params with only an auxiliary resource
{"localParams":[],"resourceList":[{"resourceName":"hdfs.keytab"}],"json":""}
// after: inline json provided
{"localParams":[],"resourceList":[{"resourceName":"hdfs.keytab"}],"json":"{\"job\":{\"content\":[...],\"setting\":{...}}}"}

boolean ok = params.isInlineJsonPresent()
    || (params.getResourceList() != null
        && params.getResourceList().stream().filter(r -> r.getResourceName().endsWith(".json")).count() == 1);

try {
    dataxTaskHandle();
} catch (TaskException e) {
    if (e.getMessage().contains("job definition is missing")) {
        alert("Attach a .json DataX job resource or provide inline json");
    }
}

When it happens

Trigger: Creating a DataX task with neither custom template json filled in nor a .json resource attached; attaching only non-.json resources (keytab, xml); UI leaving the inline json as an empty {} placeholder.

Common situations: Configuring a DataX task that relies on a Kerberos keytab resource but forgetting the actual .json job template; clearing the inline json in the UI; renaming/losing the .json resource so it no longer matches.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

• sagemaker datasource param is not valid
• dinky task params is not valid
• url can not be null
• 10001
• namespace %s does not exist in k8s cluster, please create na

AI-assisted analysis of apache/dolphinscheduler@02eac45a1b (2026-09-06). Data as JSON: /api/errors/49869c1c8d461781. Report an issue: GitHub.