apache/beam · error · ValueError

CustomQueryConfig must provide a valid query_fn

Error message

CustomQueryConfig must provide a valid query_fn

What it means

cloudsql.CustomQueryConfig's __post_init__ validates that a query_fn was supplied; if it is falsy (None or empty) a ValueError is raised. CustomQueryConfig exists specifically to let users bring their own SQL query function, so without one the config is unusable.

Solutions

Pass a valid callable as query_fn, e.g. CustomQueryConfig(query_fn=my_query_fn)
If using dynamic config, check the key exists and defaults to a real function
Switch to TableFieldsQueryConfig if you intend to specify table/fields instead of a custom query

Example fix

// before
cfg = CustomQueryConfig(query_fn=None)
// after
cfg = CustomQueryConfig(query_fn=lambda params, **kw: "SELECT * FROM users WHERE id = :id")

Defensive patterns

Strategy: validation

Validate before calling

def make_custom_config(fn):
    if not callable(fn):
        raise ValueError('query_fn must be callable')
    return CustomQueryConfig(query_fn=fn)

Type guard

def is_query_fn(v) -> bool:
    return callable(v)

Try / catch

try:
    cfg = CustomQueryConfig(query_fn=fn)
except ValueError as e:
    raise ValueError('supply a non-empty query_fn when using CustomQueryConfig') from e

Prevention

Never build dataclass configs from dicts without checking required keys
Make query_fn a keyword argument with no default so omission fails at edit time
Prefer CustomQueryConfig only when a real function exists

When it happens

Trigger: Constructing CustomQueryConfig() without query_fn, passing query_fn=None explicitly, or passing an empty/None value instead of a callable.

Common situations: Copy-pasting the dataclass and forgetting the required field; building configs dynamically from YAML/dict where the key is absent (None default).

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/5788f48c5728851a. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/transforms/enrichment_handlers/cloudsql.py:52

from sqlalchemy import create_engine
from sqlalchemy import text
from sqlalchemy.engine import Connection as DBAPIConnection

import apache_beam as beam
from apache_beam.transforms.enrichment import EnrichmentSourceHandler

QueryFn = Callable[[beam.Row], str]
ConditionValueFn = Callable[[beam.Row], list[Any]]


@dataclass
class CustomQueryConfig:
  """Configuration for using a custom query function."""
  query_fn: QueryFn

  def __post_init__(self):
    if not self.query_fn:
      raise ValueError("CustomQueryConfig must provide a valid query_fn")


@dataclass
class TableFieldsQueryConfig:
  """Configuration for using table name, where clause, and field names."""
  table_id: str
  where_clause_template: str
  where_clause_fields: list[str]

  def __post_init__(self):
    if not self.table_id or not self.where_clause_template:
      raise ValueError(
          "TableFieldsQueryConfig must provide table_id and " +
          "where_clause_template")

    if not self.where_clause_fields:
      raise ValueError(
          "TableFieldsQueryConfig must provide non-empty " +

View on GitHub (pinned to 12126d8942)