apache/dolphinscheduler · error · IllegalArgumentException

Sqoop hdfs export dir is null

Error message

Sqoop hdfs export dir is null

What it means

HdfsSourceGenerator.generate builds the '--export-dir' argument for an HDFS source. If sourceHdfsParameter.getExportDir() is empty/null, it throws IllegalArgumentException; the outer catch logs 'Sqoop hdfs source params build failed' and the job proceeds without the export dir argument, causing downstream failure.

Source

Thrown at dolphinscheduler-task-plugin/dolphinscheduler-task-sqoop/src/main/java/org/apache/dolphinscheduler/plugin/task/sqoop/generator/sources/HdfsSourceGenerator.java:56

    public String generate(
                           SqoopParameters sqoopParameters, SqoopTaskExecutionContext sqoopTaskExecutionContext) {

        StringBuilder hdfsSourceSb = new StringBuilder();

        try {
            SourceHdfsParameter sourceHdfsParameter =
                    JSONUtils.parseObject(
                            sqoopParameters.getSourceParams(), SourceHdfsParameter.class);

            if (null != sourceHdfsParameter) {
                if (StringUtils.isNotEmpty(sourceHdfsParameter.getExportDir())) {
                    hdfsSourceSb
                            .append(SPACE)
                            .append(HDFS_EXPORT_DIR)
                            .append(SPACE)
                            .append(sourceHdfsParameter.getExportDir());
                } else {
                    throw new IllegalArgumentException("Sqoop hdfs export dir is null");
                }
            }
        } catch (Exception e) {
            log.error(String.format("Sqoop hdfs source params build failed: [%s]", e.getMessage()));
        }

        return hdfsSourceSb.toString();
    }
}

View on GitHub (pinned to 02eac45a1b)

Solutions

1. Set the HDFS exportDir field (e.g. /data/output/events) on the Sqoop task's HDFS source configuration
2. Validate that the directory exists on HDFS and is readable by the sqoop job user
3. Note the message is swallowed into a log line — also check for a later 'sqoop task failed' due to missing --export-dir

Example fix

// before
sourceHdfsParameter.setExportDir("");
// after
sourceHdfsParameter.setExportDir("/data/output/events");

if ("HDFS".equals(params.getSourceType()) &&
    (params.getSourceHdfsParameter() == null || isEmpty(params.getSourceHdfsParameter().getExportDir()))) {
    throw new IllegalStateException("HDFS source requires non-empty exportDir");
}

// This IAE is swallowed into a log line; watch logs instead:
if (logs.stream().anyMatch(l -> l.contains("Sqoop hdfs export dir is null"))) {
    // abort the job and fix exportDir before retry
}

When it happens

Trigger: Source type is HDFS but the HDFS source parameter exportDir was not filled in the task form/JSON (null or empty string).

Common situations: Partially completed Sqoop HDFS-source configuration; exporting a copied task where the dir belonged to another cluster; UI form submitted before entering the path.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

• receivers must not be null
• ENVIRONMENT_NAME_IS_NULL
• ENVIRONMENT_CONFIG_IS_NULL
• 10001
• Directory already exists: ${directoryAbsolutePath}

AI-assisted analysis of apache/dolphinscheduler@02eac45a1b (2026-09-06). Data as JSON: /api/errors/392930dc744a73b9. Report an issue: GitHub.