What causes "key field type should be STRING but was %s" in apache/beam?

Declare the 'key' field as STRING in the schema.

apache/beam · error · IllegalArgumentException

key field type should be STRING but was %s

Error message

key field type should be STRING but was %s

What it means

Beyond existing, the mandatory 'key' field must be of type STRING because Bigtable row keys are byte strings. validateSchema throws IllegalArgumentException when the 'key' field's type name is anything other than STRING.

Source

Thrown at sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/bigtable/BigtableTable.java:162

              List<String> pair = Splitter.on(":").splitToList(colonSeparatedValues);
              columnsMapping.putIfAbsent(pair.get(0), newHashSet());
              columnsMapping.get(pair.get(0)).add(pair.get(1));
            });
    return columnsMapping;
  }

  private static String getMatcherValue(Matcher matcher, String field) {
    String value = matcher.group(field);
    return value == null ? "" : value;
  }

  private static void validateSchema(Schema schema) {
    if (!schema.hasField(KEY)) {
      throw new IllegalStateException(String.format("Schema has to contain '%s' field", KEY));
    } else {
      Schema.Field keyField = schema.getField(KEY);
      if (keyField != null && !(Schema.TypeName.STRING == keyField.getType().getTypeName())) {
        throw new IllegalArgumentException(
            "key field type should be STRING but was " + keyField.getType().getTypeName());
      }
    }
  }

  private static void validateMatcher(Matcher matcher, String location) {
    if (!matcher.matches()) {
      throw new InvalidTableException(
          "Bigtable location must be in the following format:"
              + " 'googleapis.com/bigtable/projects/projectId/instances/instanceId/tables/tableId'"
              + " but was: "
              + location);
    }
  }

  private static void validateColumnsMapping(
      Map<String, Set<String>> columnsMapping, Schema schema) {
    validateColumnsMappingCount(columnsMapping, schema);

View on GitHub (pinned to 12126d8942)

Solutions

Declare the 'key' field as STRING in the schema.
Cast the source key to STRING in the feeding query/pipeline before writing (e.g. CAST(id AS VARCHAR)).
If byte keys are required, encode them as (UTF-8) strings in the schema and use string values in the pipeline.

Example fix

-- before
CREATE EXTERNAL TABLE bt (key INT64, cf_val STRING) TYPE 'bigtable' LOCATION '...';

-- after
CREATE EXTERNAL TABLE bt (key STRING, cf_val STRING) TYPE 'bigtable' LOCATION '...';

Defensive patterns

Strategy: type-guard

Validate before calling

Schema.Field keyField = schema.getField("key");
if (keyField != null && keyField.getType().getTypeName() != Schema.TypeName.STRING) {
  throw new IllegalArgumentException("'key' must be STRING, got " + keyField.getType().getTypeName());
}

Type guard

boolean keyIsString(Schema s) {
  Schema.Field f = s.getField("key");
  return f != null && Schema.TypeName.STRING == f.getType().getTypeName();
}

Try / catch

try {
  BigtableTable t = new BigtableTable(table);
} catch (IllegalArgumentException e) {
  if (e.getMessage().startsWith("key field type should be STRING")) {
    // fix the schema type of 'key' to STRING
  }
}

Prevention

Always declare key as STRING; cast integer/byte keys with CAST(... AS VARCHAR) upstream.
Avoid schema auto-inference for Bigtable sources; declare the schema explicitly.
Add a schema unit test asserting key type == STRING for all bigtable table definitions.

When it happens

Trigger: Declaring a Bigtable external table whose 'key' field is INT64, BYTES, or any non-STRING type; the constructor's validateSchema rejects it immediately.

Common situations: Copying a schema from a warehouse table where the key is an integer; letting type inference derive key as INT64; changing the key type after a schema refactor.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/9adb9e08d551f96e. Report an issue: GitHub.