apache/beam · error · IllegalArgumentException

The KerberosConsumerFactoryFn requires a location for the…

Error message

The KerberosConsumerFactoryFn requires a location for the krb5.conf file. Please provide either a GCS location or Google Secret Manager location for this file.

What it means

When the external KafkaIO config selects KerberosConsumerFactoryFn as the consumer factory, a 'krb5Location' parameter is mandatory — it must point to the krb5.conf file stored in GCS or Google Secret Manager. setupExternalBuilder throws IllegalArgumentException when that key is missing.

Solutions

  1. Add the 'krb5Location' entry to consumerFactoryFnParams pointing to the krb5.conf in GCS or Secret Manager.
  2. Verify the key is spelled exactly 'krb5Location'.
  3. Upload krb5.conf to GCS/Secret Manager if it isn't stored there yet.
  4. Switch to a non-Kerberos consumer factory if Kerberos is not needed.

Example fix

// before
consumerFactoryFnParams: {}
// after
consumerFactoryFnParams: {"krb5Location": "gs://my-bucket/security/krb5.conf"}
Defensive patterns

Strategy: validation

Validate before calling

if (cls.contains("KerberosConsumerFactoryFn") && !params.containsKey("krb5Location")) throw new IllegalArgumentException("krb5Location required");

Try / catch

try { build(cfg); } catch (IllegalArgumentException e) { addKrb5Param(); }

Prevention

When it happens

Trigger: Setting consumerFactoryFnClass to a class containing 'KerberosConsumerFactoryFn' while consumerFactoryFnParams does not contain the key 'krb5Location'.

Common situations: Kerberized Kafka clusters on GCP Dataflow where the operator forgot the krb5.conf parameter; renaming the parameter key (e.g. 'krb5.conf' instead of 'krb5Location'); switching to the Kerberos factory without updating params.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/25fbbff2eb2cc5b1. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaIO.java:963

              && config.offsetDeduplication != null) {
            builder.setOffsetDeduplication(config.offsetDeduplication);
          }
          if (config.redistribute && config.redistributeByRecordKey != null) {
            builder.setRedistributeByRecordKey(config.redistributeByRecordKey);
          }
        } else {
          builder.setRedistributed(false);
          builder.setRedistributeNumKeys(0);
          builder.setAllowDuplicates(false);
          builder.setOffsetDeduplication(false);
          builder.setRedistributeByRecordKey(false);
        }

        if (config.consumerFactoryFnClass != null) {
          if (config.consumerFactoryFnClass.contains("KerberosConsumerFactoryFn")) {
            try {
              if (!config.consumerFactoryFnParams.containsKey("krb5Location")) {
                throw new IllegalArgumentException(
                    "The KerberosConsumerFactoryFn requires a location for the krb5.conf file. "
                        + "Please provide either a GCS location or Google Secret Manager location for this file.");
              }
              String krb5Location = config.consumerFactoryFnParams.get("krb5Location");
              builder.setConsumerFactoryFn(
                  InstanceBuilder.ofType(
                          new TypeDescriptor<
                              SerializableFunction<
                                  Map<String, Object>, Consumer<byte[], byte[]>>>() {})
                      .fromClassName(config.consumerFactoryFnClass)
                      .withArg(String.class, Objects.requireNonNull(krb5Location))
                      .build());
            } catch (Exception e) {
              throw new RuntimeException(
                  "Unable to construct FactoryFn "
                      + config.consumerFactoryFnClass
                      + ": "
                      + e.getMessage(),

View on GitHub (pinned to 12126d8942)