apache/kafka · error · IllegalArgumentException

must provide a positive size and max single allocation size

Error message

must provide a positive size and max single allocation size smaller than size.provided {sizeInBytes} and {maxSingleAllocationBytes} respectively

What it means

Thrown by the SimpleMemoryPool constructor when its invariants are violated: sizeInBytes and maxSingleAllocationBytes must both be positive and the single-allocation cap must not exceed the pool size. This guards the Send/NetworkClient buffer pools (and any other MemoryPool consumer) against a misconfiguration that would make the pool unable to satisfy even a single allocation. It is an IllegalArgumentException raised at construction, before any traffic flows.

Source

Thrown at clients/src/main/java/org/apache/kafka/common/memory/SimpleMemoryPool.java:45


/**
 * a simple pool implementation. this implementation just provides a limit on the total outstanding memory.
 * any buffer allocated must be release()ed always otherwise memory is not marked as reclaimed (and "leak"s)
 */
public class SimpleMemoryPool implements MemoryPool {
    protected final Logger log = LoggerFactory.getLogger(getClass()); //subclass-friendly

    protected final long sizeBytes;
    protected final boolean strict;
    protected final AtomicLong availableMemory;
    protected final int maxSingleAllocationSize;
    protected final AtomicLong startOfNoMemPeriod = new AtomicLong(); //nanoseconds
    protected volatile Sensor oomTimeSensor;

    public SimpleMemoryPool(long sizeInBytes, int maxSingleAllocationBytes, boolean strict, Sensor oomPeriodSensor) {
        if (sizeInBytes <= 0 || maxSingleAllocationBytes <= 0 || maxSingleAllocationBytes > sizeInBytes)
            throw new IllegalArgumentException("must provide a positive size and max single allocation size smaller than size."
                + "provided " + sizeInBytes + " and " + maxSingleAllocationBytes + " respectively");
        this.sizeBytes = sizeInBytes;
        this.strict = strict;
        this.availableMemory = new AtomicLong(sizeInBytes);
        this.maxSingleAllocationSize = maxSingleAllocationBytes;
        this.oomTimeSensor = oomPeriodSensor;
    }

    @Override
    public ByteBuffer tryAllocate(int sizeBytes) {
        if (sizeBytes < 1)
            throw new IllegalArgumentException("requested size " + sizeBytes + "<=0");
        if (sizeBytes > maxSingleAllocationSize)
            throw new IllegalArgumentException("requested size " + sizeBytes + " is larger than maxSingleAllocationSize " + maxSingleAllocationSize);

        long available;
        boolean success = false;
        //in strict mode we will only allocate memory if we have at least the size required.

View on GitHub (pinned to c31c9215e1)

Solutions

Inspect the two numbers in the exception message and confirm both are positive and that maxSingleAllocationBytes <= sizeInBytes.
Check `buffer.memory` (default 32 MiB) and `batch.size` (default 16 KiB) in your producer/broker config; ensure batch.size < buffer.memory and neither is 0 or negative.
If configuring the pool programmatically, add an assert or Objects.checkIndex / requirePositive guard at the call site so bad values are caught at the source.
Validate memory-string parsing in your config layer — make sure values like '32MB' are decoded into the correct byte count, not silently truncated.
For test code, size the pool at least as large as the largest single allocation you intend to make (typically >= batch.size).

Example fix

// before — batch larger than the whole pool throws
props.put(ProducerConfig.BUFFER_MEMORY_CONFIG, 16L * 1024); // 16 KiB
props.put(ProducerConfig.BATCH_SIZE_CONFIG, 64 * 1024);     // 64 KiB
Producer<String,String> p = new KafkaProducer<>(props);    // SimpleMemoryPool ctor throws

// after — keep batch.size well under buffer.memory
props.put(ProducerConfig.BUFFER_MEMORY_CONFIG, 32L * 1024 * 1024); // 32 MiB (default)
props.put(ProducerConfig.BATCH_SIZE_CONFIG,      16 * 1024);      // 16 KiB (default)

Defensive patterns

Strategy: validation

Validate before calling

// Validate constructor args once, before instantiating the pool.
long sizeInBytes = ...;
int maxSingleAllocationBytes = ...;
if (sizeInBytes <= 0 || maxSingleAllocationBytes <= 0 || maxSingleAllocationBytes > sizeInBytes) {
    throw new IllegalArgumentException(
        "sizeInBytes must be > 0 and maxSingleAllocationBytes must be > 0 and <= sizeInBytes");
}
MemoryPool pool = new SimpleMemoryPool(sizeInBytes, maxSingleAllocationBytes, strict, sensor);

Type guard

// Narrow raw long/int inputs to a validated config record.
record PoolConfig(long sizeBytes, int maxSingleAllocationBytes, boolean strict) {
    PoolConfig {
        if (sizeBytes <= 0 || maxSingleAllocationBytes <= 0 || maxSingleAllocationBytes > sizeBytes)
            throw new IllegalArgumentException("Invalid pool config");
    }
}

Try / catch

try {
    pool = new SimpleMemoryPool(sizeBytes, max, strict, sensor);
} catch (IllegalArgumentException e) {
    // misconfiguration — fail fast at startup with the user-facing message.
    failStartup("Bad memory pool configuration: " + e.getMessage());
}

Prevention

Treat pool sizing as startup configuration: validate once at boot, never at request time.
Keep maxSingleAllocationBytes <= sizeInBytes; a useful rule is max = sizeInBytes to start, then tune down.
Source both values from a typed config object with its own constructor guard, so invalid combinations cannot reach SimpleMemoryPool.
Use bytes (not KB/MB) and double-check units — unit confusion is the most common cause of this error.

When it happens

Trigger: Instantiating new SimpleMemoryPool(sizeInBytes, maxSingleAllocationBytes, strict, sensor) where sizeInBytes<=0, maxSingleAllocationBytes<=0, or maxSingleAllocationBytes>sizeInBytes. Also raised when a subclass (e.g. GarbageCollectedMemoryPool) calls super(...) with computed values that go non-positive. In Kafka itself this constructor is reached through ProducerConfig (buffer.memory / batch.size) or broker send-buffer pool wiring on startup.

Common situations: Setting `buffer.memory` or `batch.size` to 0 or negative in producer/broker config; mis-parsing a memory string (e.g. dropping the unit suffix so '64' is read as bytes); a custom MemoryPool subclass passing computed maxSingleAllocationBytes that ends up larger than the total pool; running a test harness that injects a tiny pool size and a batch size larger than it; misconfigured `socket.request.max.bytes` exceeding `buffer.memory`.

Related errors

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/9cef11019208d15d.json. Report an issue: GitHub.