{"record":{"id":"45fb1b5cf48e91ca","repo":"prestodb/presto","slug":"unexpected-partition-s-total-number-of-partitio","errorCode":null,"errorMessage":"Unexpected partition: %s. Total number of partitions: %s.","messagePattern":"Unexpected partition: (.+?)\\. Total number of partitions: (.+?)\\.","errorType":"exception","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"presto-spark-classloader-interface/src/main/java/com/facebook/presto/spark/classloader_interface/PrestoSparkPartitioner.java","lineNumber":44,"sourceCode":"    public PrestoSparkPartitioner(int numPartitions)\n    {\n        this.numPartitions = numPartitions;\n    }\n\n    @Override\n    public int numPartitions()\n    {\n        return numPartitions;\n    }\n\n    @Override\n    public int getPartition(Object key)\n    {\n        requireNonNull(key, \"key is null\");\n        MutablePartitionId mutablePartitionId = (MutablePartitionId) key;\n        int partition = mutablePartitionId.getPartition();\n        if (!(partition >= 0 && partition < numPartitions)) {\n            throw new IllegalArgumentException(format(\"Unexpected partition: %s. Total number of partitions: %s.\", partition, numPartitions));\n        }\n        return partition;\n    }\n}\n","sourceCodeStart":26,"sourceCodeEnd":49,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-spark-classloader-interface/src/main/java/com/facebook/presto/spark/classloader_interface/PrestoSparkPartitioner.java#L26-L49","documentation":"PrestoSparkPartitioner maps a MutablePartitionId key to its partition number for Spark shuffles. getPartition validates the stored partition is within [0, numPartitions) and throws IllegalArgumentException when the key carries an out-of-range partition value, indicating corrupted or miscomputed partition ids.","triggerScenarios":"Passing a MutablePartitionId whose getPartition() value is negative or >= numPartitions into a Spark operation (e.g. partitionBy/saveAsNewAPIHadoopDataset) using this partitioner.","commonSituations":"Bug in upstream partition-id assignment; wrong numPartitions supplied to the partitioner compared with the ids produced; mixed worker versions producing incompatible partition ids during shuffle.","solutions":["Verify all partition ids written into MutablePartitionId are 0 <= id < numPartitions","Ensure the numPartitions passed to PrestoSparkPartitioner matches the number of output partitions used upstream","Check for version skew between nodes producing the shuffle data"],"exampleFix":"// before\nnew MutablePartitionId(-1, position);\n// after\nint pid = Math.max(0, Math.min(computedPartition, numPartitions - 1));\nnew MutablePartitionId(pid, position);","handlingStrategy":"validation","validationCode":"if (partitionId < 0 || partitionId >= numPartitions) { throw new IllegalArgumentException(\"partition out of range: \" + partitionId); }","typeGuard":null,"tryCatchPattern":"try { int p = partitioner.getPartition(key); } catch (IllegalArgumentException e) { /* inspect/repair MutablePartitionId contents */ throw e; }","preventionTips":["Validate partition ids at the point they are written into MutablePartitionId","Keep numPartitions consistent between producer and partitioner","Guard against version skew across nodes in rolling upgrades"],"tags":["spark","shuffle","partitioning","presto"],"backgroundTag":"partition-index-out-of-range","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}