{"record":{"id":"9e6a84a348f3c890","repo":"apache/druid","slug":"cannot-set-kafka-property-auto-offset-reset-pro","errorCode":null,"errorMessage":"Cannot set kafka property [auto.offset.reset]. Property will be forced to [earliest]. Found [%s]","messagePattern":"Cannot set kafka property \\[auto\\.offset\\.reset\\]\\. Property will be forced to \\[earliest\\]\\. Found \\[(.+?)\\]","errorType":"validation","errorClass":"org.apache.druid.java.util.common.IAE","httpStatus":null,"severity":"error","filePath":"extensions-core/kafka-extraction-namespace/src/main/java/org/apache/druid/query/lookup/KafkaLookupExtractorFactory.java","lineNumber":400,"sourceCode":"  }\n\n  /**\n   * Check that the user has not set forbidden Kafka consumer props\n   *\n   * Some consumer properties must be set in order to guarantee that\n   * the consumer will consume the entire topic from the beginning.\n   * Otherwise, lookup data may not be loaded completely.\n   */\n  private void verifyKafkaProperties()\n  {\n    if (kafkaProperties.containsKey(ConsumerConfig.GROUP_ID_CONFIG)) {\n      throw new IAE(\n              \"Cannot set kafka property [group.id]. Property is randomly generated for you. Found [%s]\",\n              kafkaProperties.get(ConsumerConfig.GROUP_ID_CONFIG)\n      );\n    }\n    if (kafkaProperties.containsKey(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG)) {\n      throw new IAE(\n              \"Cannot set kafka property [auto.offset.reset]. Property will be forced to [earliest]. Found [%s]\",\n              kafkaProperties.get(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG)\n      );\n    }\n    if (kafkaProperties.containsKey(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG) &&\n          !kafkaProperties.get(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG).equals(\"false\")) {\n      throw new IAE(\n          \"Cannot set kafka property [enable.auto.commit]. Property will be forced to [false]. Found [%s]\",\n          kafkaProperties.get(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG)\n      );\n    }\n    Preconditions.checkNotNull(\n            kafkaProperties.get(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG),\n            \"bootstrap.servers required property\"\n    );\n  }\n\n  // Overridden in tests","sourceCodeStart":382,"sourceCodeEnd":418,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/extensions-core/kafka-extraction-namespace/src/main/java/org/apache/druid/query/lookup/KafkaLookupExtractorFactory.java#L382-L418","documentation":"verifyKafkaProperties() rejects a user-supplied auto.offset.reset because the factory forces 'earliest' so lookup data is fully loaded from the beginning of the topic; any other value would risk an incomplete lookup.","triggerScenarios":"Calling start() when kafkaProperties contains 'auto.offset.reset' with any value (e.g. 'latest').","commonSituations":"Reusing an ingestion-style Kafka consumer config in a lookup spec where offset reset semantics differ.","solutions":["Remove auto.offset.reset from kafkaProperties","Rely on the forced 'earliest' behavior to consume the topic from the beginning"],"exampleFix":"// before\n\"auto.offset.reset\": \"latest\"\n// after\n(removed; Druid forces earliest)","handlingStrategy":"validation","validationCode":"if (kafkaProperties.containsKey(\"auto.offset.reset\")) {\n  kafkaProperties.remove(\"auto.offset.reset\"); // lookups always start at earliest\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Remember lookups force earliest offsets","Sanitize kafkaProperties before constructing the factory","Use a shared config builder that whitelists allowed consumer keys"],"tags":["kafka","config","validation"],"backgroundTag":"invalid-config-value","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}