apache/beam · error · InvalidConfigurationException

'keyField' property cannot be null.

Error message

'keyField' property cannot be null.

What it means

DataStoreV1SchemaIOProvider.determineKeyField validates the 'keyField' configuration property. Because the check tests `configKey != null && configKey.isEmpty()`, an explicitly-set-but-empty keyField throws InvalidConfigurationException "'keyField' property cannot be null."; a non-empty value is returned as-is and null falls back to DEFAULT_KEY_FIELD (typically "__key__").

Solutions

1. Remove the keyField option entirely to use the default key field (Datastore entity __key__).
2. Provide a real property name as keyField, e.g. --keyField=id.
3. If it comes from a parameter/variable, ensure it is either unset or has a non-empty value.

Example fix

// before
--keyField= 
// after
--keyField=id   (or omit the flag to use the default __key__)

if (keyFieldOption != null && keyFieldOption.isEmpty()) {
  throw new IllegalArgumentException("--keyField must be a non-empty property name, or omitted entirely.");
}

static String sanitizeKeyField(String keyField) {
  return (keyField == null || keyField.isEmpty()) ? null /* fall back to default __key__ */ : keyField;
}

try {
  DataStoreV1SchemaIO io = (DataStoreV1SchemaIO) DataStoreV1SchemaIOProvider.schemaIO(location, config);
} catch (InvalidConfigurationException e) {
  // rebuild config without keyField to use the default key field
}

When it happens

Trigger: Specifying location 'project/kind' with an empty keyField option, e.g. --keyField= or withSchemaIO keyField:"" when creating a DataStoreV1 sink/source via SchemaIO.

Common situations: Templated/parameterized pipelines where a parameter resolves to empty string; shell scripts passing an unset variable into the keyField option; copy-pasted config with the value deleted.

Understand the failure class

Background: "must not be empty", "cannot be empty" — required-field validation errors across open-source libraries — this error's family across 41 libraries.

Related errors

• A 'datagen' table requires either 'rows-per-second' (for…
• A schema was provided without a data format (or viceversa)…
• Batch size is too large! It should be smaller or equal than
• BigQuery temp location expected a valid 'gs://' path, but…
• boolean cross product parameter required to explode more…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/f9d3de3d00a21df5. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/datastore/DataStoreV1SchemaIOProvider.java:160

        public POutput expand(PCollection<Row> input) {
          return input
              .apply("Convert Rows to Datastore Entities", RowToEntity.create(keyField, kind))
              .apply("Write Datastore Entities", DatastoreIO.v1().write().withProjectId(projectId));
        }
      };
    }

    public String getProjectId() {
      return projectId;
    }

    public String getKind() {
      return kind;
    }

    private String determineKeyField(String configKey) {
      if (configKey != null && configKey.isEmpty()) {
        throw new InvalidConfigurationException(
            String.format("'%s' property cannot be null.", KEY_FIELD_PROPERTY));
      } else if (configKey != null) {
        return configKey;
      } else {
        return DEFAULT_KEY_FIELD;
      }
    }

    private void validateLocation(String location, Matcher matcher) {
      // TODO: allow users to specify a namespace in a location string.
      if (location == null) {
        throw new InvalidLocationException("DataStoreV1 location must be set. ");
      }
      if (!matcher.matches()) {
        throw new InvalidLocationException(
            "DataStoreV1 location must be in the following format: 'projectId/kind'"
                + " but was:"
                + location);

View on GitHub (pinned to 12126d8942)