apache/flink · error · IOException

Illegal count (must be non-negative): %s

Error message

Illegal count (must be non-negative): %s

What it means

IOException from DataOutputEncoder.writeVarLongCount when asked to encode a negative length/count. The Avro encoder writes collection/array/map counts as zigzag-free non-negative varints; a negative count means the object being serialized reported a negative size, which indicates a bug or corrupted in-memory data, not a user config problem.

Source

Thrown at flink-formats/flink-avro/src/main/java/org/apache/flink/formats/avro/utils/DataOutputEncoder.java:166

        out.write(0);
    }

    // --------------------------------------------------------------------------------------------
    // union
    // --------------------------------------------------------------------------------------------

    @Override
    public void writeIndex(int unionIndex) throws IOException {
        out.writeInt(unionIndex);
    }

    // --------------------------------------------------------------------------------------------
    // utils
    // --------------------------------------------------------------------------------------------

    public static void writeVarLongCount(DataOutput out, long val) throws IOException {
        if (val < 0) {
            throw new IOException("Illegal count (must be non-negative): " + val);
        }

        while ((val & ~0x7FL) != 0) {
            out.write(((int) val) | 0x80);
            val >>>= 7;
        }
        out.write((int) val);
    }
}

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Inspect the record being serialized at failure time — which field and what its collection size reports.
2. Use standard java.util collections and do not mutate records concurrently with serialization.
3. If the record looks valid, capture a minimal reproducer and report as a Flink/Avro bug.

try {
    outEncoder.writeArrayStart();
    outEncoder.setItemCount(items.size());
    ...
} catch (IOException e) {
    if (e.getMessage().startsWith("Illegal count")) {
        // negative collection size: dump the record being serialized for debugging
        log.error("Negative collection size on field; record={}", record, e);
    }
    throw e;
}

When it happens

Trigger: Serializing an Avro record through DataOutputEncoder where an array/map/bytes field's size() returns a negative number; custom collection implementations with broken size(); concurrent mutation of the record during serialization.

Common situations: User-supplied collection classes whose size() overflows or is uninitialized; a record mutated by another thread while being checkpointed.

Related errors

• Failed to serialize row.
• Failed to serialize schema registry.
• Failed to serialize schema registry.
• Unsupported type: {}
• Failed to Java-Deserialize an AvroSerializer instance. Was e

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/7ade04010d55cd13. Report an issue: GitHub.