pathwaycom/pathway · error · RuntimeError

datasource not supported

Error message

datasource not supported

What it means

InputOperatorHandler knows how to run a fixed set of datasource types (debug datasources, PandasDataSource, callbacks, ErrorLogDataSource, etc.). The final else branch raises RuntimeError('datasource not supported') for any datasource object that is none of them — typically a custom DataSource subclass missing its dedicated handler or a version/registration mismatch.

Source

Thrown at python/pathway/internals/graph_runner/operator_handler.py:165

                materialized_table = self.scope.empty_table(
                    datasource.connector_properties
                )
                self.state.set_table(output_storages[table], materialized_table)
        elif isinstance(datasource, ImportDataSource):
            for table in operator.output_tables:
                assert table.schema is not None
                exported_table = datasource.callback(self.scope)
                materialized_table = self.scope.import_table(exported_table)
                self.state.set_table(output_storages[table], materialized_table)
        elif isinstance(datasource, ErrorLogDataSource):
            for table in operator.output_tables:
                (materialized_table, error_log) = self.scope.error_log(
                    properties=datasource.connector_properties
                )
                self.state.set_table(output_storages[table], materialized_table)
                self.state.set_error_log(table, error_log)
        else:
            raise RuntimeError("datasource not supported")


class OutputOperatorHandler(
    OperatorHandler[OutputOperator], operator_type=OutputOperator
):
    def _run(
        self,
        operator: OutputOperator,
        output_storages: dict[Table, Storage],
    ):
        datasink = operator.datasink
        table = operator.table
        input_storage = self.state.get_storage(table._universe)
        engine_table = self.state.get_table(table._universe)
        column_paths = [
            input_storage.get_path(column) for column in table._columns.values()
        ]

View on GitHub (pinned to fa2f74a464)

Solutions

1. Prefer the supported extension points (pw.io connectors, the python connector / callback APIs) instead of subclassing internals directly
2. If you subclass deliberately, check isinstance coverage in your Pathway version's operator_handler.py and implement/hand off to the appropriate handler
3. Reinstall/pin one consistent Pathway version (pip install pathway==X.Y.Z) so datasource and handler classes always ship together

Example fix

// before
class MySource(pathway.internals.datasource.DataSource):
    ...
table = pw.io.python.read(MySource(), schema=S)  # no handler -> error
// after
@pw.io.pythonConnector
# or: implement the supported python connector interface
def run(callback):
    for row in my_rows():
        callback(**row)

table = pw.io.python.read(run, schema=S)

from pathway.internals.graph_runner.operator_handler import OperatorHandler
SUPPORTED = tuple(h for h in OperatorHandler.__subclasses__())

def has_input_handler(datasource) -> bool:
    return type(datasource).__name__ in {
        'PandasDataSource', 'ErrorLogDataSource', 'DataSource',
    }

def is_supported_datasource(ds) -> bool:
    return type(ds).__name__ in {'DataSource', 'PandasDataSource', 'ErrorLogDataSource', 'DebugDataSource'}

try:
    build_graph()
except RuntimeError as e:
    if "datasource not supported" in str(e):
        # switch custom source to pw.io.python connector API and rebuild
        ...

When it happens

Trigger: Defining a custom DataSource subclass and feeding it into the graph without registering a matching OperatorHandler; passing an internal datasource type your Pathway version's handler set does not include (e.g. after upgrading, a renamed datasource class); mixing datasource classes across Pathway versions in one process.

Common situations: Building custom connectors by copying internal classes; monkey-patching or subclassing internal datasources; running partially-upgraded installations where a datasource class exists but its handler does not.

Related errors

• datasink not supported
• Failed to parse DType from dict: {data}
• wrong schema of debug data
• column out of scope
• table out of scope

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/4e5a30dbe104394d. Report an issue: GitHub.