apache/kafka · error · ConfigException

Must set retries to non-zero when using the idempotent produ

Error message

Must set retries to non-zero when using the idempotent producer.

What it means

Thrown by ProducerConfig.postProcessAndValidateIdempotenceConfigs when enable.idempotence=true is explicitly set by the user while retries is configured to 0. Idempotence relies on the producer retrying on UnknownProducerId / OutOfOrderSequence, so zero retries would break the per-partition sequence-number guarantee. Note: if idempotence was NOT explicitly user-set, the producer silently disables idempotence instead of throwing.

Source

Thrown at clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java:656

            refinedClientId = "producer-" + (transactionalId != null ? transactionalId : PRODUCER_CLIENT_ID_SEQUENCE.getAndIncrement());
        }
        configs.put(CLIENT_ID_CONFIG, refinedClientId);
    }

    private void postProcessAndValidateIdempotenceConfigs(final Map<String, Object> configs) {
        final Map<String, Object> originalConfigs = this.originals();
        final String acksStr = parseAcks(this.getString(ACKS_CONFIG));
        configs.put(ACKS_CONFIG, acksStr);
        final boolean userConfiguredIdempotence = this.originals().containsKey(ENABLE_IDEMPOTENCE_CONFIG);
        boolean idempotenceEnabled = this.getBoolean(ENABLE_IDEMPOTENCE_CONFIG);
        boolean shouldDisableIdempotence = false;

        // For idempotence producers, values for `retries` and `acks` and `max.in.flight.requests.per.connection` need validation
        if (idempotenceEnabled) {
            final int retries = this.getInt(RETRIES_CONFIG);
            if (retries == 0) {
                if (userConfiguredIdempotence) {
                    throw new ConfigException("Must set " + RETRIES_CONFIG + " to non-zero when using the idempotent producer.");
                }
                log.info("Idempotence will be disabled because {} is set to 0.", RETRIES_CONFIG);
                shouldDisableIdempotence = true;
            }

            final short acks = Short.parseShort(acksStr);
            if (acks != (short) -1) {
                if (userConfiguredIdempotence) {
                    throw new ConfigException("Must set " + ACKS_CONFIG + " to all in order to use the idempotent " +
                        "producer. Otherwise we cannot guarantee idempotence.");
                }
                log.info("Idempotence will be disabled because {} is set to {}, not set to 'all'.", ACKS_CONFIG, acks);
                shouldDisableIdempotence = true;
            }

            final int inFlightConnection = this.getInt(MAX_IN_FLIGHT_REQUESTS_PER_CONNECTION);
            if (MAX_IN_FLIGHT_REQUESTS_PER_CONNECTION_FOR_IDEMPOTENCE < inFlightConnection) {
                throw new ConfigException("To use the idempotent producer, " + MAX_IN_FLIGHT_REQUESTS_PER_CONNECTION +

View on GitHub (pinned to c31c9215e1)

Solutions

Remove the `retries=0` override, or set `retries` to a positive value such as Integer.MAX_VALUE (the modern default used together with delivery.timeout.ms).
If fire-and-forget is genuinely required, drop `enable.idempotence=true` (and any transactional.id) so the producer can run with retries=0.
Prefer governing retry behavior via delivery.timeout.ms and let retries default.

Example fix

// before
props.put("enable.idempotence", "true");
props.put("retries", "0");

// after
props.put("enable.idempotence", "true");
props.put("retries", String.valueOf(Integer.MAX_VALUE));
props.put("delivery.timeout.ms", "120000");

Defensive patterns

Strategy: validation

Validate before calling

// Verify retries > 0 whenever idempotence is enabled, BEFORE building the producer.
boolean idempotenceEnabled = Boolean.TRUE.equals(cfg.get(ProducerConfig.ENABLE_IDEMPOTENCE_CONFIG));
int retries = ((Number) cfg.getOrDefault(ProducerConfig.RETRIES_CONFIG, Integer.MAX_VALUE)).intValue();
if (idempotenceEnabled && retries == 0) {
    throw new IllegalStateException("retries must be > 0 when enable.idempotence=true");
}
// Simplest fix: just don't set retries, or set it high.
cfg.putIfAbsent(ProducerConfig.RETRIES_CONFIG, Integer.MAX_VALUE);

Try / catch

// Thrown during KafkaProducer construction as ConfigException.
try {
    producer = new KafkaProducer<>(cfg);
} catch (ConfigException e) {
    if (e.getMessage().contains("retries")) {
        cfg.put(ProducerConfig.RETRIES_CONFIG, Integer.MAX_VALUE);
        producer = new KafkaProducer<>(cfg);
    } else throw e;
}

Prevention

With idempotence on, either leave retries at its default (>0) or set it explicitly to a large value.
Centralize producer config in one builder/helper so the idempotence/retries pairing is enforced once.
Add an assertion/CI check that idempotent producers never have retries=0.

When it happens

Trigger: User provides both `enable.idempotence=true` and `retries=0` in the originals (the originals map is checked via containsKey to distinguish explicit from default). Reached at ProducerConfig.java:655-656 inside postProcessAndValidateIdempotenceConfigs, which is called from postProcessParsedConfig during KafkaProducer construction.

Common situations: Copy-pasting a non-idempotent producer config that hard-coded retries=0 for 'fire-and-forget' semantics and then flipping enable.idempotence=true; upgrading to a Kafka client version where enable.idempotence defaults to true while an older override still pins retries=0.

Related errors

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/92fa713da25736d7.json. Report an issue: GitHub.