apache/beam · error · RuntimeException

The version kafka-clients does not support record headers…

Error message

The version kafka-clients does not support record headers, please use version 0.11.0.0 or newer

What it means

KafkaRecord.getHeaders() requires the kafka-clients Consumer to support record headers, an API added in Kafka 0.11.0.0. ConsumerSpEL.hasHeaders() checks this reflectively; when the classpath has an older kafka-clients jar, a RuntimeException is thrown.

Solutions

Upgrade the kafka-clients dependency to 0.11.0.0 or newer
Run `mvn dependency:tree` (or Gradle equivalent) to find and remove transitive pins that force an old kafka-clients version
If headers are not needed, avoid header-related APIs (getHeaders/keys) in the pipeline

Example fix

// before (pom.xml)
<dependency><groupId>org.apache.kafka</groupId><artifactId>kafka-clients</artifactId><version>0.10.2.1</version></dependency>
// after
<dependency><groupId>org.apache.kafka</groupId><artifactId>kafka-clients</artifactId><version>3.6.1</version></dependency>

Defensive patterns

Strategy: validation

Validate before calling

// check kafka-clients version at startup
String version = org.apache.kafka.clients.consumer.ConsumerConfig.class.getPackage().getImplementationVersion();
if (version == null || version.compareTo("0.11.0.0") < 0) throw new IllegalStateException("kafka-clients >= 0.11.0.0 required");

Type guard

boolean headersSupported() { return org.apache.beam.sdk.io.kafka.ConsumerSpEL.hasHeaders(); }

Try / catch

try {
  Headers h = record.getHeaders();
} catch (RuntimeException e) {
  // kafka-clients too old; upgrade dependency
}

Prevention

Pin kafka-clients to a modern version (>= 0.11.0.0)
Run dependency:tree to detect version conflicts/downgrades before release
Avoid header APIs when supporting legacy client deployments

When it happens

Trigger: Calling getHeaders(), recordHeaders(), headers(), keys(), or building a structuralValue of a KafkaRecord while kafka-clients < 0.11.0.0 is on the classpath.

Common situations: Legacy builds pinning kafka-clients 0.9/0.10, transitive dependency conflicts downgrading kafka-clients, or brokers/clients from very old Kafka deployments.

Understand the failure class

Background: "not installed", "pip install", "required for": how missing-dependency errors surface across open-source libraries — this error's family across 34 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/778e3e9860db4f03. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaRecord.java:88

    this.kv = kv;
  }

  public String getTopic() {
    return topic;
  }

  public int getPartition() {
    return partition;
  }

  public long getOffset() {
    return offset;
  }

  @Pure
  public @Nullable Headers getHeaders() {
    if (!ConsumerSpEL.hasHeaders()) {
      throw new RuntimeException(
          "The version kafka-clients does not support record headers, "
              + "please use version 0.11.0.0 or newer");
    }
    return headers;
  }

  public KV<K, V> getKV() {
    return kv;
  }

  public long getTimestamp() {
    return timestamp;
  }

  public KafkaTimestampType getTimestampType() {
    return timestampType;
  }

View on GitHub (pinned to 12126d8942)