pathwaycom/pathway · error · ValueError

wrong schema of debug data

Error message

wrong schema of debug data

What it means

When running in debug mode, each input operator can be paired with a debug_datasource (e.g. pw.debug.table_from_markdown fixtures) that replaces the live connector during the debug run. This ValueError fires when the debug data's schema dtypes do not equal the real datasource's schema dtypes — the fixture must be type-compatible with what the production connector declares.

Source

Thrown at python/pathway/internals/graph_runner/operator_handler.py:115

    @classmethod
    def for_operator(cls, operator: Operator) -> type[OperatorHandler]:
        return cls._operator_mapping[type(operator)]


class InputOperatorHandler(OperatorHandler[InputOperator], operator_type=InputOperator):
    def _run(
        self,
        operator: InputOperator,
        output_storages: dict[Table, Storage],
    ):
        datasource = operator.datasource
        if self.graph_builder.debug and operator.debug_datasource is not None:
            if (
                datasource.schema._dtypes()
                != operator.debug_datasource.schema._dtypes()
            ):
                raise ValueError("wrong schema of debug data")
            for table in operator.output_tables:
                assert table.schema is not None
                materialized_table = api.static_table_from_pandas(
                    scope=self.scope,
                    df=operator.debug_datasource.data,
                    connector_properties=operator.debug_datasource.connector_properties,
                    schema=operator.debug_datasource.schema,
                )
                self.state.set_table(output_storages[table], materialized_table)
        elif isinstance(datasource, PandasDataSource):
            for table in operator.output_tables:
                assert table.schema is not None
                materialized_table = api.static_table_from_pandas(
                    scope=self.scope,
                    df=datasource.data,
                    connector_properties=datasource.connector_properties,
                    schema=datasource.schema,
                )

View on GitHub (pinned to fa2f74a464)

Solutions

1. Align the fixture's column names and types with the connector schema — for datetimes include offsets in the fixture (or match naive) so both sides produce the same dtype
2. Regenerate the debug data from a sample of the real source (e.g. convert a real row to markdown) to guarantee dtype parity
3. If schema is in flux, update both files in the same commit and run the debug pipeline in CI to catch drift early

Example fix

// before
class Input(pw.Schema):
    ts: pw.DateTimeUtc
    val: str

table = pw.debug.table_from_markdown('''
 | ts | val
 | 1700000000 | 1   # ints, but schema says DateTimeUtc/str
''')
// after
table = pw.debug.table_from_markdown('''
 | ts                  | val
 | 2023-11-14T22:13:20Z| a
 | 2023-11-15T10:00:00Z| b
''')

def debug_schema_matches(connector_schema, debug_datasource) -> bool:
    return connector_schema._dtypes() == debug_datasource.schema._dtypes()

try:
    pw.debug.compute_and_print(pipeline(debug=True))
except ValueError as e:
    if "wrong schema of debug data" in str(e):
        # regenerate the markdown fixture from a real sample row
        ...

When it happens

Trigger: pw.debug runs where the markdown/pandas fixture column types differ from the connector schema: fixture supplies ints where the schema declares str, missing datetime timezone semantics (naive vs UTC), column sets differing so _dtypes() dicts are unequal.

Common situations: Iterating on a connector schema but not updating the debug markdown fixture; timezone-naive debug strings vs DateTimeUtc schema; Optional vs plain dtype differences between fixture and schema; CI running debug builds against stale fixtures.

Related errors

• not supported type of debug data
• Value {val} is not of type {dtype}.
• datasource not supported
• output: {result_schema} of the iterated function does not c
• output schema validation error, received {output.as_typehint

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/618164faaba68501. Report an issue: GitHub.