apache/kafka · error · IllegalArgumentException

Invalid partition: %d. Partition number should always be non

Error message

Invalid partition: %d. Partition number should always be non-negative or null.

What it means

Thrown by the ProducerRecord constructor when an explicit partition number is supplied that is negative. Partition numbers in Kafka are 0-indexed; the producer accepts either null (to let the partitioner choose) or a non-negative integer. A negative partition would never map to a real partition, so it is rejected at construction as an IllegalArgumentException.

Source

Thrown at clients/src/main/java/org/apache/kafka/clients/producer/ProducerRecord.java:78

    /**
     * Creates a record with a specified timestamp to be sent to a specified topic and partition
     * 
     * @param topic The topic the record will be appended to
     * @param partition The partition to which the record should be sent
     * @param timestamp The timestamp of the record, in milliseconds since epoch. If null, the producer will assign
     *                  the timestamp using System.currentTimeMillis().
     * @param key The key that will be included in the record
     * @param value The record contents
     * @param headers the headers that will be included in the record
     */
    public ProducerRecord(String topic, Integer partition, Long timestamp, K key, V value, Iterable<Header> headers) {
        if (topic == null)
            throw new IllegalArgumentException("Topic cannot be null.");
        if (timestamp != null && timestamp < 0)
            throw new IllegalArgumentException(
                    String.format("Invalid timestamp: %d. Timestamp should always be non-negative or null.", timestamp));
        if (partition != null && partition < 0)
            throw new IllegalArgumentException(
                    String.format("Invalid partition: %d. Partition number should always be non-negative or null.", partition));
        this.topic = topic;
        this.partition = partition;
        this.key = key;
        this.value = value;
        this.timestamp = timestamp;
        this.headers = new RecordHeaders(headers);
    }

    /**
     * Creates a record with a specified timestamp to be sent to a specified topic and partition
     *
     * @param topic The topic the record will be appended to
     * @param partition The partition to which the record should be sent
     * @param timestamp The timestamp of the record, in milliseconds since epoch. If null, the producer will assign the
     *                  timestamp using System.currentTimeMillis().
     * @param key The key that will be included in the record
     * @param value The record contents

View on GitHub (pinned to c31c9215e1)

Solutions

1. Pass null instead of a negative number when you want the partitioner to choose: new ProducerRecord<>(topic, null, key, value).
2. Validate the partition before construction: Integer p = (partition == null || partition < 0) ? null : partition.
3. If the partition comes from a config that uses -1 as 'unset', translate it to null at the config boundary so -1 never reaches the constructor.

Example fix

// before
int partition = config.getInt("partition", -1);
new ProducerRecord<>(topic, partition, key, value);

// after
int raw = config.getInt("partition", -1);
Integer partition = raw < 0 ? null : raw;
new ProducerRecord<>(topic, partition, key, value);

// Partition must be null (let the partitioner choose) or >= 0.
Integer partition = /* intended partition */;
if (partition != null && partition < 0) {
    throw new IllegalArgumentException("partition must be non-negative or null, got " + partition);
}
ProducerRecord<K, V> record = new ProducerRecord<>(topic, partition, timestamp, key, value);

// Narrows an Integer to a valid partition sentinel or concrete partition.
static Integer requireValidPartition(Integer p, int numPartitions) {
    if (p != null) {
        if (p < 0) throw new IllegalArgumentException("negative partition: " + p);
        if (p >= numPartitions) throw new IllegalArgumentException("partition " + p + " >= numPartitions " + numPartitions);
    }
    return p;
}

try {
    ProducerRecord<K, V> r = new ProducerRecord<>(topic, partition, timestamp, key, value);
    producer.send(r);
} catch (IllegalArgumentException e) {
    // Fall back to null partition so the built-in partitioner picks one
    ProducerRecord<K, V> fallback = new ProducerRecord<>(topic, null, timestamp, key, value);
    producer.send(fallback);
}

When it happens

Trigger: Calling a ProducerRecord constructor with a negative Integer partition, e.g. new ProducerRecord<>(topic, -1, key, value). Commonly seen when callers use -1 as a 'let the partitioner decide' marker instead of passing null.

Common situations: Confusing the producer's API (which uses null for unspecified partition) with libraries or APIs that use -1 as a sentinel; computing a partition from a hash or modulo that returned -1 due to a bug; reading partition from a config that defaults to -1; porting code from a client in another language whose convention is -1 for unset.

Related errors

• Topic cannot be null.
• Invalid timestamp: %d. Timestamp should always be non-negati
• Invalid configuration value for 'acks': {acksString}
• Attempt to allocate {size} bytes, but there is a hard limit
• Producer closed while allocating memory

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/66355ea5ed8ff12d.json. Report an issue: GitHub.