apache/beam · error · KeyError

Make sure the values passed in `where_clause_fields` are…

Error message

Make sure the values passed in `where_clause_fields` are the keys in the input `beam.Row`.

What it means

For table-based query configs, _extract_values_from_request indexes request_dict[field] for each field in where_clause_fields; a KeyError is caught and re-raised with a message stating that every name in where_clause_fields must be a key of the input beam.Row (the missing key name is appended).

Solutions

1. Ensure every beam.Row contains all where_clause_fields keys before enrichment.
2. Fix typos/case mismatches between row keys and where_clause_fields.
3. Use the appended missing key name in the message to pinpoint the field.

Example fix

# before (where_clause_fields=["id"])
row = beam.Row(user_id=42)
# after
row = beam.Row(id=42)

When it happens

Trigger: A beam.Row missing one of the fields listed in where_clause_fields: typo, wrong case, field only present on some records, or an optional field not always emitted.

Common situations: Upstream DoFn emits rows with differing schemas; renaming a column in the pipeline without updating where_clause_fields; PCollection rows built with different dict keys per branch.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

• A schema is required to write non-schema'd data.
• All dicts in batch must have the same keys. extra keys
• An explicit schema is required to write non-schema'd…
• Arrow map key field cannot be nullable
• Attempted to encode null for non-nullable field

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/1fc0659bd4f0c4d0. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/transforms/enrichment_handlers/cloudsql.py:586

    Raises:
      KeyError: If required fields are missing from the request
    """
    try:
      if isinstance(self._query_config, TableFunctionQueryConfig):
        return [
            val for val in self._query_config.where_clause_value_fn(request)
        ]
      elif isinstance(self._query_config, TableFieldsQueryConfig):
        request_dict = request._asdict()
        return [
            request_dict[field]
            for field in self._query_config.where_clause_fields
        ]
      else:
        raise ValueError("Unsupported query configuration type")
    except KeyError as e:
      raise KeyError(
          "Make sure the values passed in `where_clause_fields` are "
          "the keys in the input `beam.Row`." + str(e))

  def _extract_parameter_names(self, template: str) -> list[str]:
    """Extract parameter names from a SQL template string.

    Args:
      template: SQL template string with named parameters (e.g., "id = :id")

    Returns:
      List of parameter names found in the template (e.g., ["id"])
    """
    return re.findall(r':(\w+)', template)

  def create_row_key(self, row: beam.Row):
    if isinstance(self._query_config, TableFunctionQueryConfig):
      return tuple(self._query_config.where_clause_value_fn(row))
    if isinstance(self._query_config, TableFieldsQueryConfig):

View on GitHub (pinned to 12126d8942)