{"record":{"id":"c2ea41e058a09351","repo":"apache/druid","slug":"cleaned-partition-map-s-contains-unexpected-par","errorCode":null,"errorMessage":"Cleaned partition map [%s] contains unexpected partition ID [%s], original partition map: [%s]","messagePattern":"Cleaned partition map \\[(.+?)\\] contains unexpected partition ID \\[(.+?)\\], original partition map: \\[(.+?)\\]","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/supervisor/SeekableStreamSupervisor.java","lineNumber":3619,"sourceCode":"   * @param oldMetadata     metadata containing expired partitions.\n   * @param cleanedMetadata new metadata without expired partitions, generated by the subclass\n   */\n  private void validateMetadataPartitionExpiration(\n      Set<PartitionIdType> newlyExpiredPartitions,\n      SeekableStreamDataSourceMetadata<PartitionIdType, SequenceOffsetType> oldMetadata,\n      SeekableStreamDataSourceMetadata<PartitionIdType, SequenceOffsetType> cleanedMetadata\n  )\n  {\n    Map<PartitionIdType, SequenceOffsetType> oldPartitionSeqNos = oldMetadata.getSeekableStreamSequenceNumbers()\n                                                                             .getPartitionSequenceNumberMap();\n\n    Map<PartitionIdType, SequenceOffsetType> cleanedPartitionSeqNos = cleanedMetadata.getSeekableStreamSequenceNumbers()\n                                                                                     .getPartitionSequenceNumberMap();\n\n    for (Entry<PartitionIdType, SequenceOffsetType> cleanedPartitionSeqNo : cleanedPartitionSeqNos.entrySet()) {\n      if (!oldPartitionSeqNos.containsKey(cleanedPartitionSeqNo.getKey())) {\n        // cleaning the expired partitions added a partition somehow\n        throw new IAE(\n            \"Cleaned partition map [%s] contains unexpected partition ID [%s], original partition map: [%s]\",\n            cleanedPartitionSeqNos,\n            cleanedPartitionSeqNo.getKey(),\n            oldPartitionSeqNos\n        );\n      }\n\n      SequenceOffsetType oldOffset = oldPartitionSeqNos.get(cleanedPartitionSeqNo.getKey());\n      if (newlyExpiredPartitions.contains(cleanedPartitionSeqNo.getKey())) {\n        // this is a newly expired partition, check that we did actually mark it as expired\n        if (!isShardExpirationMarker(cleanedPartitionSeqNo.getValue())) {\n          throw new IAE(\n              \"Newly expired partition [%] was not marked as expired in the cleaned partition map [%s], original partition map: [%s]\",\n              cleanedPartitionSeqNo.getKey(),\n              cleanedPartitionSeqNos,\n              oldPartitionSeqNos\n          );\n        }","sourceCodeStart":3601,"sourceCodeEnd":3637,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/supervisor/SeekableStreamSupervisor.java#L3601-L3637","documentation":"After computing metadata with expired partitions removed, the supervisor sanity-checks that the cleaned partition map is a subset of the original map. If cleaning somehow introduced a partition ID that was not there before, the invariant is broken and IllegalArgumentException is thrown naming the cleaned map, the offending partition, and the original map.","triggerScenarios":"createDataSourceMetadataWithExpiredPartitions returns metadata whose partition map contains a key absent from the pre-cleaning map — i.e. a buggy or non-compliant override of the expiration hook.","commonSituations":"Custom supervisor implementations whose expiration override adds partitions instead of only removing/marking them; concurrent modification of partition maps during expiration processing.","solutions":["Fix the custom createDataSourceMetadataWithExpiredPartitions implementation so it only removes or marks partitions from the input map, never adds new keys.","Log the original and cleaned maps (as the error already prints them) and diff to find the injected partition ID.","Verify the expiredPartitionIds set passed in contains only IDs from the original partition map."],"exampleFix":"// before: override adds rebuilt partition groups into cleaned metadata\ncleanedMap.putAll(recomputedPartitions);\n// after: only retain partitions from the original map\ncleanedMap.keySet().retainAll(oldPartitionSeqNos.keySet());","handlingStrategy":"validation","validationCode":"if (!oldPartitionSeqNos.keySet().containsAll(cleanedPartitionSeqNos.keySet())) {\n    throw new IllegalStateException(\"expiration hook added partitions\");\n}","typeGuard":null,"tryCatchPattern":"try { applyCleanedMetadata(); } catch (IAE e) { log.error(\"Non-subset cleaned map: {}\", e.getMessage()); }","preventionTips":["Ensure expiration overrides only remove/mark partitions","Test custom expiration hooks against the original partition map","Validate expiredPartitionIds ⊆ original partitions"],"tags":["druid","supervisor","invariant","partition-expiration"],"backgroundTag":"internal-invariant-violation","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}