apache/beam · error · ValueError

Either fields or condition_value_fn must be specified

Error message

Either fields or condition_value_fn must be specified

What it means

BigQuerySanitized enrichment handler's create_row_key found neither a condition_value_fn nor a fields list on the handler config; with no way to derive the join key from the input row, row-key generation (and hence caching/enrichment) cannot proceed.

Solutions

Provide fields (list of key column names) or condition_value_fn on the handler
If using query_fn exclusively, use the non-batched call path or ensure fields are still supplied for key creation

Example fix

// before
BigQueryEnrichmentHandler(query_fn=fn)
// after
BigQueryEnrichmentHandler(query_fn=fn, fields=['id'])

Defensive patterns

Strategy: validation

Validate before calling

def ensure_key_source(handler):
    if not getattr(handler, 'condition_value_fn', None) and not getattr(handler, 'fields', None):
        raise ValueError('Handler needs fields or condition_value_fn for key creation')

Try / catch

try:
    responses = handler(rows)
except ValueError as e:
    logger.error('Cannot build row key: %s', e)
    raise

Prevention

Always provide fields even when using query_fn if key-based batching is used
Never construct the handler bypassing __init__ validation
Test one enrichment call locally before deploying

When it happens

Trigger: Calling __call__/create_row_key on a BigQueryEnrichmentHandler constructed without condition_value_fn and without fields (possible when configured via query_fn and bypassing validation, or when using BatchingKey = fields-based path with neither set).

Common situations: Handler built with only query_fn but used in a code path (batching key creation) that requires fields/condition_value_fn; validation skipped by direct construction.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/8fef4ed368609ecc. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/transforms/enrichment_handlers/bigquery.py:195

      if self._batching_kwargs:
        return row_list
      else:
        return row_list[0]
    except BadRequest as e:
      raise BadRequest(
          f'Could not execute the query: {query}. Please check if '
          f'the query is properly formatted and the BigQuery '
          f'table exists. {e}')
    except RuntimeError as e:
      raise RuntimeError(f"Could not complete the query request: {query}. {e}")

  def create_row_key(self, row: beam.Row):
    if self.condition_value_fn:
      return tuple(self.condition_value_fn(row))
    if self.fields:
      row_dict = row._asdict()
      return (tuple(row_dict[field] for field in self.fields))
    raise ValueError("Either fields or condition_value_fn must be specified")

  def __call__(self, request: Union[beam.Row, list[beam.Row]], *args, **kwargs):
    if isinstance(request, list):
      values = []
      responses = []
      requests_map: dict[Any, list[beam.Row]] = defaultdict(list)
      batch_size = len(request)
      raw_query = self.query_template
      if batch_size > 1:
        batched_condition_template = ' or '.join(
            [fr'({self.row_restriction_template})'] * batch_size)
        raw_query = self.query_template.replace(
            self.row_restriction_template, batched_condition_template)
      for req in request:
        request_dict = req._asdict()
        try:
          current_values = (
              self.condition_value_fn(req) if self.condition_value_fn else

View on GitHub (pinned to 12126d8942)