{"record":{"id":"12d34d4930c9cde0","repo":"apache/seatunnel","slug":"key-is-required-12d34d","errorCode":null,"errorMessage":"${key} is required","messagePattern":"(.+?) is required","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"seatunnel-engine/seatunnel-engine-storage/checkpoint-storage-plugins/checkpoint-storage-hdfs/src/main/java/org/apache/seatunnel/engine/checkpoint/storage/hdfs/common/AbstractConfiguration.java","lineNumber":47,"sourceCode":"public abstract class AbstractConfiguration {\n\n    protected static final String HDFS_IMPL_KEY = \"impl\";\n\n    protected static final String COMMON_DISABLE_CACHE = \"%s.disable.cache\";\n\n    protected static final String DISABLE_CACHE_DEFAULT_VALUE = \"TRUE\";\n\n    protected static final String DISABLE_CACHE_KEY = \"disable.cache\";\n    /**\n     * check the configuration keys\n     *\n     * @param config configuration\n     * @param keys keys\n     */\n    void checkConfiguration(Map<String, String> config, String... keys) {\n        for (String key : keys) {\n            if (!config.containsKey(key) || null == config.get(key)) {\n                throw new IllegalArgumentException(key + \" is required\");\n            }\n        }\n    }\n\n    public abstract Configuration buildConfiguration(Map<String, String> config)\n            throws CheckpointStorageException;\n\n    /**\n     * set extra options for configuration\n     *\n     * @param hadoopConf hadoop configuration\n     * @param config extra options\n     * @param prefix prefix of extra options\n     */\n    void setExtraConfiguration(\n            Configuration hadoopConf, Map<String, String> config, String prefix) {\n        config.forEach(\n                (k, v) -> {","sourceCodeStart":29,"sourceCodeEnd":65,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-engine/seatunnel-engine-storage/checkpoint-storage-plugins/checkpoint-storage-hdfs/src/main/java/org/apache/seatunnel/engine/checkpoint/storage/hdfs/common/AbstractConfiguration.java#L29-L65","documentation":"AbstractConfiguration.checkConfiguration validates that required configuration keys are present and non-null in the storage config map, throwing IllegalArgumentException naming the missing key. HDFS/GCS/S3/OSS configuration classes call it during buildConfiguration to fail fast on incomplete storage settings.","triggerScenarios":"Building an HdfsStorage (or other plugin storage) from a config map that omits a required key (e.g. HDFS 'fs.defaultFS') or maps it to null.","commonSituations":"Checkpoint storage plugin config in the job/cluster yaml missing mandatory properties; typo in config key so the expected key is absent; passing an empty or partially-populated config programmatically.","solutions":["Add the missing key named in the message to your checkpoint storage configuration with a non-null value.","Cross-check key names against the plugin's configuration class constants (case-sensitive) — a typo causes the same error.","Consult the plugin docs (checkpoint-storage-hdfs/local-file/s3) for the list of required options.","Validate the config map in your own bootstrap code before constructing the storage."],"exampleFix":"// before\nMap<String, String> config = new HashMap<>();\nHdfsStorage storage = new HdfsStorage(config); // throws: fs.defaultFS is required\n// after\nMap<String, String> config = new HashMap<>();\nconfig.put(\"fs.defaultFS\", \"hdfs://namenode:8020\");\nHdfsStorage storage = new HdfsStorage(config);","handlingStrategy":"validation","validationCode":"java.util.Set<String> required = java.util.Set.of(\"fs.defaultFS\");\nfor (String key : required) {\n    if (config.get(key) == null)\n        throw new IllegalArgumentException(\"checkpoint storage config missing: \" + key);\n}","typeGuard":"boolean hasAllKeys(Map<String,String> config, String... keys) {\n    return java.util.Arrays.stream(keys).allMatch(k -> config.get(k) != null);\n}","tryCatchPattern":"try {\n    storage = new HdfsStorage(config);\n} catch (IllegalArgumentException e) {\n    // message is '<key> is required' — log and abort startup with a clear config hint\n    throw new IllegalStateException(\"Invalid checkpoint storage config: \" + e.getMessage(), e);\n}","preventionTips":["Copy option names exactly from the plugin's configuration class/docs — they are case-sensitive.","Validate the full storage config map at deployment time before submitting jobs.","Keep checkpoint storage settings versioned with the cluster config to catch regressions."],"tags":["configuration","validation","hdfs","missing-key"],"backgroundTag":"missing-required-config-field","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}