pathwaycom/pathway · error · RuntimeError

datasink not supported

Error message

datasink not supported

What it means

OutputOperatorHandler lowers each output operator to an engine sink and recognizes a fixed set of datasink types (persisted sinks, exports, etc.). Unknown datasink objects fall through to RuntimeError('datasink not supported') — the mirror of the datasource error, hit when a custom or unrecognized DataSink type is attached to the graph.

Source

Thrown at python/pathway/internals/graph_runner/operator_handler.py:211

        elif isinstance(datasink, CallbackDataSink):
            self.scope.subscribe_table(
                table=engine_table,
                column_paths=column_paths,
                on_change=datasink.on_change,
                on_time_end=datasink.on_time_end,
                on_end=datasink.on_end,
                skip_persisted_batch=datasink.skip_persisted_batch,
                skip_errors=datasink.skip_errors,
                unique_name=datasink.unique_name,
                sort_by_indices=datasink.sort_by_indices(table),
            )
        elif isinstance(datasink, ExportDataSink):
            exported_table = self.scope.export_table(
                table=engine_table, column_paths=column_paths
            )
            datasink.callback(self.scope, exported_table)
        else:
            raise RuntimeError("datasink not supported")


class ContextualizedIntermediateOperatorHandler(
    OperatorHandler[ContextualizedIntermediateOperator],
    operator_type=ContextualizedIntermediateOperator,
):
    def _run(
        self,
        operator: ContextualizedIntermediateOperator,
        output_storages: dict[Table, Storage],
    ):
        for table in operator.intermediate_and_output_tables:
            context = table._id_column.context
            evaluator_cls = ExpressionEvaluator.for_context(context)
            output_storage = output_storages[table]
            evaluator = evaluator_cls(
                context, self.scope, self.state, self.scope_context
            )

View on GitHub (pinned to fa2f74a464)

Solutions

1. Use the supported sink extension point — pw.io.python's write side or pw.io._internal apprehendable callback sinks — rather than subclassing DataSink
2. If subclassing is required, verify your Pathway version's OutputOperatorHandler handles your sink type and update the subclass to a supported base class
3. Align versions: reinstall/pin the same pathway version that introduced your sink class so class and handler match

Example fix

// before
class MySink(pathway.internals.datasinks.DataSink):
    ...
pw.io._internal.write(table, MySink())  # no handler -> error
// after
@pw.io.pythonConnector
class MyWriter:
    def on_change(self, key, row, time, is_add): ...
    def on_end(self): ...

pw.io.python.write(table, MyWriter())

KNOWN_SINKS = {'PersistedDataSink', 'ExportDataSink', 'DataSink'}

def sink_type_name(sink) -> str:
    return type(sink).__name__

assert sink_type_name(sink) in KNOWN_SINKS

def is_supported_datasink(sink) -> bool:
    return type(sink).__name__ in {'DataSink', 'ExportDataSink', 'PersistedDataSink'}

try:
    build_graph()
except RuntimeError as e:
    if "datasink not supported" in str(e):
        # replace custom sink with a pw.io python writer callback and rebuild
        ...

When it happens

Trigger: Attaching a custom DataSink subclass to pw.io without a corresponding handler; version skew where a datasink class exists but this build's handler list lacks its branch; constructing sinks via internal APIs not meant for direct use.

Common situations: Custom output integrations implemented by subclassing internals instead of using the python/callback sink APIs; stale virtualenvs after upgrade; experimental sinks copied between Pathway versions.

Related errors

• datasource not supported
• The column {column} doesn't belong to the target table {tabl
• column out of scope
• table out of scope
• path storage out of scope

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/7e40794a0f2c6293. Report an issue: GitHub.