apache/druid · error · org.apache.druid.java.util.common.IAE

Cannot set kafka property [group.id]. Property is randomly g

Error message

Cannot set kafka property [group.id]. Property is randomly generated for you. Found [%s]

What it means

verifyKafkaProperties() rejects a user-supplied group.id because the factory generates a unique random consumer group id for each lookup; allowing a fixed group.id would cause offset tracking conflicts between lookup consumers.

Source

Thrown at extensions-core/kafka-extraction-namespace/src/main/java/org/apache/druid/query/lookup/KafkaLookupExtractorFactory.java:394

    return doubleEventCount;
  }

  ListenableFuture<?> getFuture()
  {
    return future;
  }

  /**
   * Check that the user has not set forbidden Kafka consumer props
   *
   * Some consumer properties must be set in order to guarantee that
   * the consumer will consume the entire topic from the beginning.
   * Otherwise, lookup data may not be loaded completely.
   */
  private void verifyKafkaProperties()
  {
    if (kafkaProperties.containsKey(ConsumerConfig.GROUP_ID_CONFIG)) {
      throw new IAE(
              "Cannot set kafka property [group.id]. Property is randomly generated for you. Found [%s]",
              kafkaProperties.get(ConsumerConfig.GROUP_ID_CONFIG)
      );
    }
    if (kafkaProperties.containsKey(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG)) {
      throw new IAE(
              "Cannot set kafka property [auto.offset.reset]. Property will be forced to [earliest]. Found [%s]",
              kafkaProperties.get(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG)
      );
    }
    if (kafkaProperties.containsKey(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG) &&
          !kafkaProperties.get(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG).equals("false")) {
      throw new IAE(
          "Cannot set kafka property [enable.auto.commit]. Property will be forced to [false]. Found [%s]",
          kafkaProperties.get(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG)
      );
    }
    Preconditions.checkNotNull(

View on GitHub (pinned to 9b90983fd2)

Solutions

Remove the group.id entry from kafkaProperties in the lookup spec
Let Druid generate the group id automatically

Example fix

// before
"group.id": "my-lookup-group"
// after
(removed; Druid generates group.id internally)

Defensive patterns

Strategy: validation

Validate before calling

if (kafkaProperties.containsKey("group.id")) {
  throw new IllegalArgumentException("Remove group.id; Druid generates it");
}

Prevention

Never copy plain Kafka consumer configs into lookup specs
Strip group.id/auto.offset.reset/enable.auto.commit keys programmatically before start
Validate lookup specs in CI

When it happens

Trigger: Calling start() (which invokes verifyKafkaProperties) when the kafkaProperties map contains the key 'group.id'.

Common situations: Copy-pasting a plain Kafka consumer config (which requires group.id) into a Druid kafka lookup spec.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/33fe72011707b8c8. Report an issue: GitHub.