apache/seatunnel · error · IllegalArgumentException

${key} is required

Error message

${key} is required

What it means

AbstractConfiguration.checkConfiguration validates that required configuration keys are present and non-null in the storage config map, throwing IllegalArgumentException naming the missing key. HDFS/GCS/S3/OSS configuration classes call it during buildConfiguration to fail fast on incomplete storage settings.

Source

Thrown at seatunnel-engine/seatunnel-engine-storage/checkpoint-storage-plugins/checkpoint-storage-hdfs/src/main/java/org/apache/seatunnel/engine/checkpoint/storage/hdfs/common/AbstractConfiguration.java:47

public abstract class AbstractConfiguration {

    protected static final String HDFS_IMPL_KEY = "impl";

    protected static final String COMMON_DISABLE_CACHE = "%s.disable.cache";

    protected static final String DISABLE_CACHE_DEFAULT_VALUE = "TRUE";

    protected static final String DISABLE_CACHE_KEY = "disable.cache";
    /**
     * check the configuration keys
     *
     * @param config configuration
     * @param keys keys
     */
    void checkConfiguration(Map<String, String> config, String... keys) {
        for (String key : keys) {
            if (!config.containsKey(key) || null == config.get(key)) {
                throw new IllegalArgumentException(key + " is required");
            }
        }
    }

    public abstract Configuration buildConfiguration(Map<String, String> config)
            throws CheckpointStorageException;

    /**
     * set extra options for configuration
     *
     * @param hadoopConf hadoop configuration
     * @param config extra options
     * @param prefix prefix of extra options
     */
    void setExtraConfiguration(
            Configuration hadoopConf, Map<String, String> config, String prefix) {
        config.forEach(
                (k, v) -> {

View on GitHub (pinned to cf67b549a7)

Solutions

1. Add the missing key named in the message to your checkpoint storage configuration with a non-null value.
2. Cross-check key names against the plugin's configuration class constants (case-sensitive) — a typo causes the same error.
3. Consult the plugin docs (checkpoint-storage-hdfs/local-file/s3) for the list of required options.
4. Validate the config map in your own bootstrap code before constructing the storage.

Example fix

// before
Map<String, String> config = new HashMap<>();
HdfsStorage storage = new HdfsStorage(config); // throws: fs.defaultFS is required
// after
Map<String, String> config = new HashMap<>();
config.put("fs.defaultFS", "hdfs://namenode:8020");
HdfsStorage storage = new HdfsStorage(config);

java.util.Set<String> required = java.util.Set.of("fs.defaultFS");
for (String key : required) {
    if (config.get(key) == null)
        throw new IllegalArgumentException("checkpoint storage config missing: " + key);
}

boolean hasAllKeys(Map<String,String> config, String... keys) {
    return java.util.Arrays.stream(keys).allMatch(k -> config.get(k) != null);
}

try {
    storage = new HdfsStorage(config);
} catch (IllegalArgumentException e) {
    // message is '<key> is required' — log and abort startup with a clear config hint
    throw new IllegalStateException("Invalid checkpoint storage config: " + e.getMessage(), e);
}

When it happens

Trigger: Building an HdfsStorage (or other plugin storage) from a config map that omits a required key (e.g. HDFS 'fs.defaultFS') or maps it to null.

Common situations: Checkpoint storage plugin config in the job/cluster yaml missing mandatory properties; typo in config key so the expected key is absent; passing an empty or partially-populated config programmatically.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

• ${key} is required
• ${key} is required
• Condition operator must not be null
• AmazonDocumentDB option '' must not be blank
• AmazonDocumentDB option '' must be a valid BSON/JSON documen

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/12d34d4930c9cde0. Report an issue: GitHub.