apache/cassandra · error · PartitionSerializationException

Failed to serialize partition key '%s' on table '%s' in keys

Error message

Failed to serialize partition key '%s' on table '%s' in keyspace '%s'.

What it means

When serializing a partition for the wire (messaging or streaming), UnfilledRowIteratorSerializer wraps BufferOverflowException from the underlying buffer in a PartitionSerializationException that names the partition key, table and keyspace. It means the partition (or a single row within it under the legacy pre-4.0 limit) exceeded the maximum message size allowed by the configured transport settings, so the partition cannot be sent as-is.

Source

Thrown at src/java/org/apache/cassandra/db/rows/UnfilteredRowIteratorSerializer.java:112

    public void serialize(UnfilteredRowIterator iterator, ColumnFilter selection, DataOutputPlus out, int version, int rowEstimate) throws IOException
    {
        serialize(iterator, out, version, rowEstimate, MESSAGING, selection);
    }

    public <P> void serialize(UnfilteredRowIterator iterator, DataOutputPlus out, int version, int rowEstimate, ParameterizedSerializer<P> serializer, P param) throws IOException
    {
        SerializationHeader header = new SerializationHeader(false,
                                                             iterator.metadata(),
                                                             iterator.columns(),
                                                             iterator.stats());

        try
        {
            serialize(iterator, header, out, version, rowEstimate, serializer, param);
        }
        catch (BufferOverflowException boe)
        {
            throw new PartitionSerializationException(iterator, boe);
        }
    }

    private void serialize(UnfilteredRowIterator iterator, SerializationHeader header, ColumnFilter selection, DataOutputPlus out, int version, int rowEstimate) throws IOException
    {
        serialize(iterator, header, out, version, rowEstimate, MESSAGING, selection);
    }

    // Should only be used for the on-wire format.
    private <P> void serialize(UnfilteredRowIterator iterator, SerializationHeader header, DataOutputPlus out, int version, int rowEstimate, ParameterizedSerializer<P> serializer, P param) throws IOException
    {
        assert !header.isForSSTable();

        ByteBufferUtil.writeWithVIntLength(iterator.partitionKey().getKey(), out);
        serializeWithoutKey(iterator, header, out, version, rowEstimate, serializer, param);
    }

    public <P> void serializeWithoutKey(UnfilteredRowIterator iterator, SerializationHeader header, DataOutputPlus out, int version, int rowEstimate, ParameterizedSerializer<P> serializer, P param) throws IOException

View on GitHub (pinned to 88fd0f6a0e)

Solutions

1. Reduce partition size: rewrite the oversized partition with a better partition key and re-stream.
2. Increase the relevant limits (in 4.0+: stream_throughput / internode max message size, max_mutation_size_in_mb consistently below commitlog segment size) and retry.
3. Use incremental repair/re-stream or rebuild the target node from scratch instead of moving the giant partition.
4. Split large partitions during migration using a tool like sstablepartitioner before streaming.

Example fix

// before: cassandra.yaml defaults too small for a 1GB partition
# max_mutation_size_in_mb: 16
# commitlog_segment_size_in_mb: 32
// after: raise consistently (4.0+)
max_mutation_size_in_mb: 512
commitlog_segment_size_in_mb: 1024
internode_max_message_size_in_bytes: 536870912 # and re-stream

// estimate partition size before streaming
long size = estimatedPartitionSize(key); if (size > maxMessageSize) rePartitionOrSplit(key);

try { serializePartition(iterator, header, out, version); } catch (PartitionSerializationException e) { handleOversizedPartition(e.partitionKey); }

When it happens

Trigger: Streaming a huge partition larger than the streaming socket/file-transfer buffer limits; sending a partition exceeding internode message size (native_transport / internode max message size in 4.0+); hinted handoff or read-repair of an oversized partition.

Common situations: Very wide partitions from unbounded partition keys; transfer of data written by clients that never bound partition size; tuning max_mutation_size / max message size after the data already exists.

Understand the failure class

Background: payload too large / request exceeds maximum size: why libraries cap bytes and how to fix oversize payloads — this error's family across 50 libraries.

Related errors

• %s something is seriously wrong with the calculated stream c
• Insufficient remaining bytes to deserialize Inner node off-h
• [Stream {}] Error while reading partition {} from stream on
• [Stream {}] Error while reading partition {} from stream on
• Serialized size must not be more than 2GiB

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/37e55c2b88a57cfe. Report an issue: GitHub.