apache/beam · error · ValueError

Only expressions allowed in SQL at

Error message

Only expressions allowed in SQL at {name}.

What it means

In `extract_expr`, used by the SQL-based MapToFields path (_SqlMapToFieldsTransform), each field config must be either a plain expression string or a dict containing an 'expression' key. Any other shape (e.g. a dict with 'type'/'callable' options meant for the generic path) cannot be turned into a SQL SELECT item, so a ValueError naming the offending field is raised.

Solutions

1. Wrap the expression in an 'expression' key: {field: {expression: "..."}}.
2. Pass the expression as a plain string instead of a dict.
3. Switch the transform to the generic language if you need callable/type-style field configs.

Example fix

// before
fields:
  total:
    type: double
    value: "price * qty"
// after
fields:
  total:
    expression: "(price * qty)"

for name, v in fields.items():
    if not (isinstance(v, str) or (isinstance(v, dict) and 'expression' in v)):
        raise ValueError(f'SQL field {name} must be a string or {{expression: ...}}')

def is_sql_expr(v) -> bool:
    return isinstance(v, str) or (isinstance(v, dict) and 'expression' in v)

try:
    out = sql_map_to_fields(pcoll, fields)
except ValueError as e:
    if 'Only expressions allowed in SQL' in str(e):
        fields = {k: v['expression'] if isinstance(v, dict) and 'expression' in v else v for k, v in fields.items()}
    else:
        raise

When it happens

Trigger: Using language: sql (or a SQL MapToFields transform) with a `fields` entry that is a dict without an 'expression' key — e.g. {'callable': ...}, {'type': 'string'}, or {'value': ...} config copied from the generic/python transform syntax.

Common situations: Copy-pasting a field config from a PyJs (generic) MapToFields into a SQL one; using callable-style or type-only configs that the SQL dialect does not support; missing the nested 'expression' key.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

• Can only drop fields if append is true.
• Error handling output
• Exploding unknown field
• Input schema has multiple fields
• Invalid feature store yaml file provided. Make sure the

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/223a072e6f565f64. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/yaml/yaml_mapping.py:756

@beam.ptransform.ptransform_fn
def _SqlFilterTransform(pcoll, sql_transform_constructor, keep, language):
  return pcoll | sql_transform_constructor(
      f"SELECT * FROM PCOLLECTION WHERE {keep.get('expression')}")


@beam.ptransform.ptransform_fn
def _SqlMapToFieldsTransform(pcoll, sql_transform_constructor, **mapping_args):
  _, fields = normalize_fields(pcoll, **mapping_args)

  def extract_expr(name, v):
    if isinstance(v, str):
      return v
    elif 'expression' in v:
      return v['expression']
    else:
      raise ValueError(f"Only expressions allowed in SQL at {name}.")

  selects = [
      f'({extract_expr(name, expr)}) AS `{name}`'
      for (name, expr) in fields.items()
  ]
  query = "SELECT " + ", ".join(selects) + " FROM PCOLLECTION"
  return pcoll | sql_transform_constructor(query)


@beam.ptransform.ptransform_fn
def _Partition(
    pcoll,
    by: Union[str, dict[str, str]],
    outputs: list[str],
    unknown_output: Optional[str] = None,
    error_handling: Optional[Mapping[str, Any]] = None,
    language: str = 'generic'):
  """Splits an input into several distinct outputs.

View on GitHub (pinned to 12126d8942)