apache/druid · error · org.apache.druid.java.util.common.IAE

Cannot set kafka property [group.id]. Property is randomly…

Error message

Cannot set kafka property [group.id]. Property is randomly generated for you. Found [%s]

What it means

verifyKafkaProperties() rejects a user-supplied group.id because the factory generates a unique random consumer group id for each lookup; allowing a fixed group.id would cause offset tracking conflicts between lookup consumers.

Solutions

1. Remove the group.id entry from kafkaProperties in the lookup spec
2. Let Druid generate the group id automatically

Example fix

// before
"group.id": "my-lookup-group"
// after
(removed; Druid generates group.id internally)

if (kafkaProperties.containsKey("group.id")) {
  throw new IllegalArgumentException("Remove group.id; Druid generates it");
}

When it happens

Trigger: Calling start() (which invokes verifyKafkaProperties) when the kafkaProperties map contains the key 'group.id'.

Common situations: Copy-pasting a plain Kafka consumer config (which requires group.id) into a Druid kafka lookup spec.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

• Cannot set kafka property [auto.offset.reset]. Property…
• Cannot set kafka property [enable.auto.commit]. Property…
• At least one of baseDir or files should be specified
• bucketSize must be a power of two (from 1 up to 128) but…
• Cannot have a null/empty columns

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/33fe72011707b8c8. Report an issue: GitHub.

Appendix: source

Thrown at extensions-core/kafka-extraction-namespace/src/main/java/org/apache/druid/query/lookup/KafkaLookupExtractorFactory.java:394

    return doubleEventCount;
  }

  ListenableFuture<?> getFuture()
  {
    return future;
  }

  /**
   * Check that the user has not set forbidden Kafka consumer props
   *
   * Some consumer properties must be set in order to guarantee that
   * the consumer will consume the entire topic from the beginning.
   * Otherwise, lookup data may not be loaded completely.
   */
  private void verifyKafkaProperties()
  {
    if (kafkaProperties.containsKey(ConsumerConfig.GROUP_ID_CONFIG)) {
      throw new IAE(
              "Cannot set kafka property [group.id]. Property is randomly generated for you. Found [%s]",
              kafkaProperties.get(ConsumerConfig.GROUP_ID_CONFIG)
      );
    }
    if (kafkaProperties.containsKey(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG)) {
      throw new IAE(
              "Cannot set kafka property [auto.offset.reset]. Property will be forced to [earliest]. Found [%s]",
              kafkaProperties.get(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG)
      );
    }
    if (kafkaProperties.containsKey(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG) &&
          !kafkaProperties.get(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG).equals("false")) {
      throw new IAE(
          "Cannot set kafka property [enable.auto.commit]. Property will be forced to [false]. Found [%s]",
          kafkaProperties.get(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG)
      );
    }
    Preconditions.checkNotNull(

View on GitHub (pinned to 9b90983fd2)