{"record":{"id":"99574bb0fbe2b238","repo":"t8y2/dbx","slug":"duplicate-partition-in-offsets-partition","errorCode":null,"errorMessage":"duplicate partition in offsets: \" + partition","messagePattern":"duplicate partition in offsets: \" \\+ partition","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"agents/drivers/kafka/src/main/java/com/dbx/agent/kafka/KafkaAgent.java","lineNumber":1532,"sourceCode":"        }\n        JsonElement offsetsElement = params.get(\"offsets\");\n        if (!offsetsElement.isJsonArray()) {\n            throw new IllegalArgumentException(\"offsets must be an array\");\n        }\n        JsonArray offsetArray = offsetsElement.getAsJsonArray();\n        if (offsetArray.isEmpty()) {\n            throw new IllegalArgumentException(\"offsets must contain at least one partition offset\");\n        }\n        for (JsonElement element : offsetArray) {\n            if (!element.isJsonObject()) {\n                throw new IllegalArgumentException(\"each offset must be an object\");\n            }\n            JsonObject value = element.getAsJsonObject();\n            int partition = nonNegativeExactInt(value, \"partition\");\n            long offset = nonNegativeExactLong(value, \"offset\");\n            TopicPartition topicPartition = new TopicPartition(topic, partition);\n            if (offsets.put(topicPartition, new OffsetAndMetadata(offset)) != null) {\n                throw new IllegalArgumentException(\"duplicate partition in offsets: \" + partition);\n            }\n        }\n        return offsets;\n    }\n\n    private static int nonNegativeExactInt(JsonObject object, String name) {\n        long value = nonNegativeExactLong(object, name);\n        if (value > Integer.MAX_VALUE) {\n            throw new IllegalArgumentException(name + \" is outside the supported integer range\");\n        }\n        return (int) value;\n    }\n\n    private static long nonNegativeExactLong(JsonObject object, String name) {\n        JsonElement element = object.get(name);\n        if (element == null || !element.isJsonPrimitive() || !element.getAsJsonPrimitive().isNumber()) {\n            throw new IllegalArgumentException(name + \" must be a non-negative integer\");\n        }","sourceCodeStart":1514,"sourceCodeEnd":1550,"githubUrl":"https://github.com/t8y2/dbx/blob/c0390bff16418b651f4728520d99adf8ce48829a/agents/drivers/kafka/src/main/java/com/dbx/agent/kafka/KafkaAgent.java#L1514-L1550","documentation":"Within one commit request, each partition of the topic may appear only once. The parser accumulates entries into a Map<TopicPartition, OffsetAndMetadata>; if put() returns a previous value for the same TopicPartition it throws IllegalArgumentException \"duplicate partition in offsets: <n>\" (KafkaAgent.java:1532).","triggerScenarios":"The offsets array contains two objects with the same \"partition\" value for the same topic, e.g. [{\"partition\":0,\"offset\":10},{\"partition\":0,\"offset\":20}].","commonSituations":"Merging offset lists from multiple consumers without deduplication; retry logic re-appending an entry; concatenating per-partition results that overlap.","solutions":["Deduplicate by partition before sending, keeping the latest offset entry","Use a Map keyed by partition when building the offsets list so duplicates are impossible","Check upstream code for repeated appends of the same partition"],"exampleFix":"// before\noffsets.add(Map.of(\"partition\",0,\"offset\",10));\noffsets.add(Map.of(\"partition\",0,\"offset\",20));\n// after\nMap<Integer,Long> byPartition = new HashMap<>();\nbyPartition.put(0, 20L); // one entry per partition, latest wins","handlingStrategy":"validation","validationCode":"const parts = offsets.map(o => o.partition); if (new Set(parts).size !== parts.length) throw new Error('duplicate partition');","typeGuard":null,"tryCatchPattern":"try { agent.execute(req); } catch (e) { if (String(e.message).startsWith('duplicate partition in offsets')) { /* dedupe and retry once */ } else throw e; }","preventionTips":["Build offsets in a Map keyed by partition","Deduplicate merged lists before sending","Review merge/concat logic for overlapping partitions"],"tags":["kafka","input-validation","duplicates"],"backgroundTag":"duplicate-key-in-collection","analyzedSha":"c0390bff16418b651f4728520d99adf8ce48829a","analyzedAt":"2026-09-05T23:05:10.900Z","contentChangedAt":"2026-09-05T23:05:10.900Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}