apache/kafka · error · ConfigException

Must set acks to all in order to use the idempotent producer

Error message

Must set acks to all in order to use the idempotent producer. Otherwise we cannot guarantee idempotence.

What it means

Thrown by ProducerConfig.postProcessAndValidateIdempotenceConfigs when the user explicitly enables idempotence but sets acks to anything other than 'all' (-1). Idempotent delivery requires full ISR acks so the broker can durably assign and advance the producer's sequence numbers; weaker ack levels would let a successful-but-unreplicated write become indistinguishable from a failed one and break the dedup window. As with retries, if idempotence was defaulted rather than explicitly requested it is silently disabled instead.

Source

Thrown at clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java:665

        final boolean userConfiguredIdempotence = this.originals().containsKey(ENABLE_IDEMPOTENCE_CONFIG);
        boolean idempotenceEnabled = this.getBoolean(ENABLE_IDEMPOTENCE_CONFIG);
        boolean shouldDisableIdempotence = false;

        // For idempotence producers, values for `retries` and `acks` and `max.in.flight.requests.per.connection` need validation
        if (idempotenceEnabled) {
            final int retries = this.getInt(RETRIES_CONFIG);
            if (retries == 0) {
                if (userConfiguredIdempotence) {
                    throw new ConfigException("Must set " + RETRIES_CONFIG + " to non-zero when using the idempotent producer.");
                }
                log.info("Idempotence will be disabled because {} is set to 0.", RETRIES_CONFIG);
                shouldDisableIdempotence = true;
            }

            final short acks = Short.parseShort(acksStr);
            if (acks != (short) -1) {
                if (userConfiguredIdempotence) {
                    throw new ConfigException("Must set " + ACKS_CONFIG + " to all in order to use the idempotent " +
                        "producer. Otherwise we cannot guarantee idempotence.");
                }
                log.info("Idempotence will be disabled because {} is set to {}, not set to 'all'.", ACKS_CONFIG, acks);
                shouldDisableIdempotence = true;
            }

            final int inFlightConnection = this.getInt(MAX_IN_FLIGHT_REQUESTS_PER_CONNECTION);
            if (MAX_IN_FLIGHT_REQUESTS_PER_CONNECTION_FOR_IDEMPOTENCE < inFlightConnection) {
                throw new ConfigException("To use the idempotent producer, " + MAX_IN_FLIGHT_REQUESTS_PER_CONNECTION +
                                          " must be set to at most 5. Current value is " + inFlightConnection + ".");
            }
        }

        if (shouldDisableIdempotence) {
            configs.put(ENABLE_IDEMPOTENCE_CONFIG, false);
            idempotenceEnabled = false;
        }

View on GitHub (pinned to c31c9215e1)

Solutions

1. Set `acks=all` (equivalently `acks=-1`) when enabling idempotence.
2. Remove the explicit `acks` override so the idempotent producer's default of 'all' applies.
3. If the lower acks level is genuinely desired, disable idempotence and do not set a transactional.id.

Example fix

// before
props.put("enable.idempotence", "true");
props.put("acks", "1");

// after
props.put("enable.idempotence", "true");
props.put("acks", "all");

// Ensure acks is 'all' (or -1) when idempotence is on, BEFORE constructing.
boolean idempotenceEnabled = Boolean.TRUE.equals(cfg.get(ProducerConfig.ENABLE_IDEMPOTENCE_CONFIG));
Object acksRaw = cfg.getOrDefault(ProducerConfig.ACKS_CONFIG, "all");
String acks = String.valueOf(acksRaw).trim();
boolean acksAll = acks.equalsIgnoreCase("all") || "-1".equals(acks);
if (idempotenceEnabled && !acksAll) {
    throw new IllegalStateException("acks must be 'all' when enable.idempotence=true");
}
// Or just force it:
if (idempotenceEnabled) cfg.put(ProducerConfig.ACKS_CONFIG, "all");

try {
    producer = new KafkaProducer<>(cfg);
} catch (ConfigException e) {
    if (e.getMessage().contains("acks")) {
        cfg.put(ProducerConfig.ACKS_CONFIG, "all");
        producer = new KafkaProducer<>(cfg);
    } else throw e;
}

When it happens

Trigger: User originals contain `enable.idempotence=true` together with `acks=0`, `acks=1`, or `acks=any-other-than-all-or--1`. The check at ProducerConfig.java:663 fires after parseAcks normalizes 'all' to '-1'.

Common situations: Inheriting a latency-tuning config that pinned `acks=1` for throughput and then turning on idempotence for exactly-once; or setting `acks=0` for fire-and-forget and forgetting it conflicts with the idempotent/transactional producer.

Related errors

• Must set retries to non-zero when using the idempotent produ
• To use the idempotent producer, max.in.flight.requests.per.c
• Cannot set a transactional.id without also enabling idempote
• Invalid producer ID and epoch values: {producerId}:{epoch}.
• Cannot set transaction.timeout.ms when transaction.two.phase

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/8e7ffe81567fa844.json. Report an issue: GitHub.