apache/flink · error · IllegalArgumentException

Buffer size must be at least 2.

Error message

Buffer size must be at least 2.

What it means

DelimitedInputFormat's read buffer must hold at least the delimiter plus content to make scanning correct (the buffer is scanned for the delimiter across refill boundaries, and the wrap buffer needs headroom). A buffer of size 0 or 1 cannot reliably contain a multi-byte delimiter, so setBufferSize enforces a minimum of 2.

Source

Thrown at flink-core/src/main/java/org/apache/flink/api/common/io/DelimitedInputFormat.java:267

    public int getLineLengthLimit() {
        return lineLengthLimit;
    }

    public void setLineLengthLimit(int lineLengthLimit) {
        if (lineLengthLimit < 1) {
            throw new IllegalArgumentException("Line length limit must be at least 1.");
        }

        this.lineLengthLimit = lineLengthLimit;
    }

    public int getBufferSize() {
        return bufferSize;
    }

    public void setBufferSize(int bufferSize) {
        if (bufferSize < 2) {
            throw new IllegalArgumentException("Buffer size must be at least 2.");
        }

        this.bufferSize = bufferSize;
    }

    public int getNumLineSamples() {
        return numLineSamples;
    }

    public void setNumLineSamples(int numLineSamples) {
        if (numLineSamples < 0) {
            throw new IllegalArgumentException("Number of line samples must not be negative.");
        }
        this.numLineSamples = numLineSamples;
    }

    // --------------------------------------------------------------------------------------------
    //  User-defined behavior

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Set bufferSize >= 2 (and strictly greater than the delimiter length — see also initBuffers guard, error 310).
2. Use a typical value like 1024 * 1024 (1 MB, the default) or at least a few KB.
3. Leave the default if you do not need to tune memory usage.

Example fix

// before
format.setBufferSize(bufferBytes); // bufferBytes == 1 -> throws

// after
int bs = bufferBytes >= 2 ? bufferBytes : 4 * 1024;
format.setBufferSize(bs);

int bs = configuredBufferSize >= 2 ? configuredBufferSize : 1024 * 1024;
format.setBufferSize(bs);

When it happens

Trigger: Calling format.setBufferSize(0) or format.setBufferSize(1).

Common situations: Tuning the read buffer down too aggressively for memory savings; computing buffer size from a config that defaulted to 0; passing a value smaller than the delimiter length.

Related errors

• Buffer size must be greater than length of delimiter.
• Delimiter must not be null
• Line length limit must be at least 1.
• Number of line samples must not be negative.
• The block size parameter must be set and larger than 0.

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/6aca141003f816a5. Report an issue: GitHub.