apache/druid · error · org.apache.druid.java.util.common.IAE

Cannot set kafka property [enable.auto.commit]. Property wil

Error message

Cannot set kafka property [enable.auto.commit]. Property will be forced to [false]. Found [%s]

What it means

Configuration guard in KafkaLookupExtractorFactory.verifyKafkaProperties (called from start()): the user supplied enable.auto.commit in kafkaProperties, but the factory manages offsets itself and forcibly sets this property to false; supplying it is rejected at startup to prevent silently differing behavior.

Source

Thrown at extensions-core/kafka-extraction-namespace/src/main/java/org/apache/druid/query/lookup/KafkaLookupExtractorFactory.java:407

   * Otherwise, lookup data may not be loaded completely.
   */
  private void verifyKafkaProperties()
  {
    if (kafkaProperties.containsKey(ConsumerConfig.GROUP_ID_CONFIG)) {
      throw new IAE(
              "Cannot set kafka property [group.id]. Property is randomly generated for you. Found [%s]",
              kafkaProperties.get(ConsumerConfig.GROUP_ID_CONFIG)
      );
    }
    if (kafkaProperties.containsKey(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG)) {
      throw new IAE(
              "Cannot set kafka property [auto.offset.reset]. Property will be forced to [earliest]. Found [%s]",
              kafkaProperties.get(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG)
      );
    }
    if (kafkaProperties.containsKey(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG) &&
          !kafkaProperties.get(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG).equals("false")) {
      throw new IAE(
          "Cannot set kafka property [enable.auto.commit]. Property will be forced to [false]. Found [%s]",
          kafkaProperties.get(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG)
      );
    }
    Preconditions.checkNotNull(
            kafkaProperties.get(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG),
            "bootstrap.servers required property"
    );
  }

  // Overridden in tests
  Consumer<String, String> getConsumer()
  {
    // Workaround for Kafka String Serializer could not be found
    // Adopted from org.apache.druid.indexing.kafka.KafkaRecordSupplier#getKafkaConsumer
    ClassLoader currCtxCl = Thread.currentThread().getContextClassLoader();
    final Properties properties = getConsumerProperties();
    try {

View on GitHub (pinned to 9b90983fd2)

Solutions

Remove enable.auto.commit from kafkaProperties
Or explicitly set it to "false"

Example fix

// before
"enable.auto.commit": "true"
// after
"enable.auto.commit": "false"  // or remove the property

Defensive patterns

Strategy: validation

Validate before calling

if (kafkaProperties.containsKey("enable.auto.commit") && !"false".equals(kafkaProperties.get("enable.auto.commit"))) {
  kafkaProperties.put("enable.auto.commit", "false");
}

Prevention

Always set enable.auto.commit=false for lookups
Normalize consumer props before start()
Validate specs in CI with a property whitelist

When it happens

Trigger: Calling start() when kafkaProperties contains 'enable.auto.commit' set to anything other than 'false' (e.g. 'true').

Common situations: Copying default Kafka consumer settings (auto-commit defaults to true) into a Druid lookup spec.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/931c4e4d7f17e900. Report an issue: GitHub.