{"id":"3a2d36ffb97136b7","repo":"apache/kafka","slug":"configuration-name-values-must-not-be-duplicated","errorCode":null,"errorMessage":"Configuration 'name' values must not be duplicated.","messagePattern":"Configuration 'name' values must not be duplicated\\.","errorType":"validation","errorClass":"ConfigException","httpStatus":null,"severity":"error","filePath":"clients/src/main/java/org/apache/kafka/common/config/ConfigDef.java","lineNumber":1074,"sourceCode":"\n        @Override\n        public void ensureValid(final String name, final Object value) {\n            if (value == null) {\n                if (isNullAllowed)\n                    return;\n                else\n                    throw new ConfigException(\"Configuration '\" + name + \"' values must not be null.\");\n            }\n\n            @SuppressWarnings(\"unchecked\")\n            List<Object> values = (List<Object>) value;\n            if (!isEmptyAllowed && values.isEmpty()) {\n                String validString = this.validString.validStrings.isEmpty() ? \"any non-empty value\" : this.validString.toString();\n                throw new ConfigException(\"Configuration '\" + name + \"' must not be empty. Valid values include: \" + validString);\n            }\n\n            if (Set.copyOf(values).size() != values.size()) {\n                throw new ConfigException(\"Configuration '\" + name + \"' values must not be duplicated.\");\n            }\n\n            validateIndividualValues(name, values);\n        }\n\n        private void validateIndividualValues(String name, List<Object> values) {\n            boolean hasValidStrings = !validString.validStrings.isEmpty();\n\n            for (Object value : values) {\n                if (value instanceof String) {\n                    String string = (String) value;\n                    if (string.isEmpty()) {\n                        throw new ConfigException(\"Configuration '\" + name + \"' values must not be empty.\");\n                    }\n                    if (hasValidStrings) {\n                        validString.ensureValid(name, value);\n                    }\n                }","sourceCodeStart":1056,"sourceCodeEnd":1092,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/common/config/ConfigDef.java#L1056-L1092","documentation":"Thrown by ValidList.ensureValid after it builds Set.copyOf(values) and finds its size smaller than the list size, i.e. the supplied list contains duplicates. The validator exists specifically to enforce uniqueness (ValidList.anyNonDuplicateValues / ValidList.in), so any repeated element is rejected regardless of whether individual values are themselves legal.","triggerScenarios":"A ConfigKey validated with ValidList receives a list containing the same element twice, e.g. \"a,b,a\" split into [a, b, a]. Triggered during ConfigDef.parse() or any direct call to ValidList.ensureValid(name, values).","commonSituations":"Comma-separated config values where a value is listed twice by accident (copy-paste, merge of two config files, duplicate env-var injection). Also seen when a list is built programmatically by concatenating sources without de-duplication.","solutions":["Inspect the named config's value and remove the duplicate entries so each element appears exactly once.","If the duplicates originate from merging multiple config sources, de-duplicate at build time (new LinkedHashSet<>(list)) before assigning.","Verify shell/env expansion is not appending the same value twice (e.g. VAR=$VAR,extra)."],"exampleFix":"// before\nprops.put(\"my.list.config\", \"read,write,read\");\n\n// after\nprops.put(\"my.list.config\", \"read,write\");","handlingStrategy":"validation","validationCode":"// Deduplicate list-typed config values before passing them in:\nMap<String, Object> safe = new HashMap<>();\nfor (Map.Entry<String, Object> e : configs.entrySet()) {\n    Object v = e.getValue();\n    if (v instanceof List<?>) {\n        List<?> dedup = ((List<?>) v).stream().distinct().collect(Collectors.toList());\n        safe.put(e.getKey(), dedup);\n    } else {\n        safe.put(e.getKey(), v);\n    }\n}","typeGuard":"null","tryCatchPattern":"try {\n    new KafkaAdmin(props);\n} catch (ConfigException ce) {\n    if (ce.getMessage().contains(\"duplicated\")) {\n        log.warn(\"Duplicate entries in '{}'; de-duplicating and retrying\", ce.getName());\n        dedupeAndRetry(props);\n    } else throw ce;\n}","preventionTips":["Build list-valued configs from a Set rather than a List when order is irrelevant.","When joining values from multiple sources (file + env), dedupe the final list, not each source.","Write a config linter that calls Set.copyOf(list).size()==list.size() for every List config key."],"tags":["config","kafka-client","validation","validator","list","duplicates"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}