apache/beam · error · ValueError

Unsupported runner .

Error message

Unsupported runner %s.

What it means

The %%beam_sql magic dispatches to a runner-specific flow (DirectRunner local run or Dataflow form); if the detected/configured runner is neither, it raises ValueError('Unsupported runner %s.', runner). Note the comma means the arguments are never formatted into the message by a bare raise, but the intent is that only DirectRunner and DataflowRunner are supported.

Solutions

1. Set the runner to DirectRunner for local execution or DataflowRunner for GCP before running the magic
2. Check the PipelineOptions / %%capture magic configuration for a typo in the runner name
3. Wrap the runner in an interactive environment that maps to a supported backend
4. If you need another runner, execute the SqlChain manually instead of the magic

Example fix

// before
options = PipelineOptions(['--runner=FlinkRunner'])
%%beam_sql output
// after
options = PipelineOptions(['--runner=DirectRunner'])
%%beam_sql output

runner = options.view_as(PipelineOptions).get_all_options().get('runner')
if runner and 'DirectRunner' not in str(runner) and 'DataflowRunner' not in str(runner):
    raise ValueError(f'%%beam_sql supports DirectRunner/DataflowRunner only, got {runner}')

def is_supported_runner(runner):
    name = str(runner)
    return 'DirectRunner' in name or 'DataflowRunner' in name

try:
    get_ipython().run_cell_magic('beam_sql', ..., ...)
except ValueError as e:
    if 'Unsupported runner' in str(e):
        options.pipeline_options(['--runner=DirectRunner'])

When it happens

Trigger: Running %%beam_sql with apache_beam.options.pipeline_options runner set to something other than DirectRunner or DataflowRunner (e.g. FlinkRunner, SparkRunner, or a typo'd runner name).

Common situations: Notebooks with --runner FlinkRunner in PipelineOptions; misspelled runner class; environments where the magic cannot detect a supported runner from the options.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• should be either a single PCollection or a dict of named…
• Aggregator [ ] is not supported
• Asynchronous computation was cancelled.
• Attempting to alter catalog
• Attempting to create a table with unexpected Calcite Schema…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/562eebe566625413. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/runners/interactive/sql/beam_sql_magics.py:242

      collect_data_for_local_run(query, found)
      output_name, output, chain = apply_sql(query, output_name, found)
      chain.current.schemas = schemas
      cache_output(output_name, output)
      return output

    output_name, current_node, chain = apply_sql(
        query, output_name, found, False)
    current_node.schemas = schemas
    # TODO(BEAM-10708): Move the options setup and result handling to a
    # separate module when more runners are supported.
    if runner == 'DataflowRunner':
      _ = chain.to_pipeline()
      _ = DataflowOptionsForm(
          output_name, pcoll_by_name()[output_name],
          verbose).display_for_input()
      return None
    else:
      raise ValueError('Unsupported runner %s.', runner)


@progress_indicated
def collect_data_for_local_run(query: str, found: dict[str, beam.PCollection]):
  from apache_beam.runners.interactive import interactive_beam as ib
  for name, pcoll in found.items():
    try:
      _ = ib.collect(pcoll)
    except (KeyboardInterrupt, SystemExit):
      raise
    except:  # pylint: disable=bare-except
      _LOGGER.error(
          'Cannot collect data for PCollection %s. Please make sure the '
          'PCollections queried in the sql "%s" are all from a single '
          'pipeline using an InteractiveRunner. Make sure there is no '
          'ambiguity, for example, same named PCollections from multiple '
          'pipelines or notebook re-executions.',
          name,

View on GitHub (pinned to 12126d8942)