{"record":{"id":"618164faaba68501","repo":"pathwaycom/pathway","slug":"wrong-schema-of-debug-data","errorCode":null,"errorMessage":"wrong schema of debug data","messagePattern":"wrong schema of debug data","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/pathway/internals/graph_runner/operator_handler.py","lineNumber":115,"sourceCode":"\n    @classmethod\n    def for_operator(cls, operator: Operator) -> type[OperatorHandler]:\n        return cls._operator_mapping[type(operator)]\n\n\nclass InputOperatorHandler(OperatorHandler[InputOperator], operator_type=InputOperator):\n    def _run(\n        self,\n        operator: InputOperator,\n        output_storages: dict[Table, Storage],\n    ):\n        datasource = operator.datasource\n        if self.graph_builder.debug and operator.debug_datasource is not None:\n            if (\n                datasource.schema._dtypes()\n                != operator.debug_datasource.schema._dtypes()\n            ):\n                raise ValueError(\"wrong schema of debug data\")\n            for table in operator.output_tables:\n                assert table.schema is not None\n                materialized_table = api.static_table_from_pandas(\n                    scope=self.scope,\n                    df=operator.debug_datasource.data,\n                    connector_properties=operator.debug_datasource.connector_properties,\n                    schema=operator.debug_datasource.schema,\n                )\n                self.state.set_table(output_storages[table], materialized_table)\n        elif isinstance(datasource, PandasDataSource):\n            for table in operator.output_tables:\n                assert table.schema is not None\n                materialized_table = api.static_table_from_pandas(\n                    scope=self.scope,\n                    df=datasource.data,\n                    connector_properties=datasource.connector_properties,\n                    schema=datasource.schema,\n                )","sourceCodeStart":97,"sourceCodeEnd":133,"githubUrl":"https://github.com/pathwaycom/pathway/blob/fa2f74a4649b7c5908690cf60137263d8d80de5f/python/pathway/internals/graph_runner/operator_handler.py#L97-L133","documentation":"When running in debug mode, each input operator can be paired with a debug_datasource (e.g. pw.debug.table_from_markdown fixtures) that replaces the live connector during the debug run. This ValueError fires when the debug data's schema dtypes do not equal the real datasource's schema dtypes — the fixture must be type-compatible with what the production connector declares.","triggerScenarios":"pw.debug runs where the markdown/pandas fixture column types differ from the connector schema: fixture supplies ints where the schema declares str, missing datetime timezone semantics (naive vs UTC), column sets differing so _dtypes() dicts are unequal.","commonSituations":"Iterating on a connector schema but not updating the debug markdown fixture; timezone-naive debug strings vs DateTimeUtc schema; Optional vs plain dtype differences between fixture and schema; CI running debug builds against stale fixtures.","solutions":["Align the fixture's column names and types with the connector schema — for datetimes include offsets in the fixture (or match naive) so both sides produce the same dtype","Regenerate the debug data from a sample of the real source (e.g. convert a real row to markdown) to guarantee dtype parity","If schema is in flux, update both files in the same commit and run the debug pipeline in CI to catch drift early"],"exampleFix":"// before\nclass Input(pw.Schema):\n    ts: pw.DateTimeUtc\n    val: str\n\ntable = pw.debug.table_from_markdown('''\n | ts | val\n | 1700000000 | 1   # ints, but schema says DateTimeUtc/str\n''')\n// after\ntable = pw.debug.table_from_markdown('''\n | ts                  | val\n | 2023-11-14T22:13:20Z| a\n | 2023-11-15T10:00:00Z| b\n''')","handlingStrategy":"validation","validationCode":"def debug_schema_matches(connector_schema, debug_datasource) -> bool:\n    return connector_schema._dtypes() == debug_datasource.schema._dtypes()","typeGuard":null,"tryCatchPattern":"try:\n    pw.debug.compute_and_print(pipeline(debug=True))\nexcept ValueError as e:\n    if \"wrong schema of debug data\" in str(e):\n        # regenerate the markdown fixture from a real sample row\n        ...","preventionTips":["Keep debug fixtures and connector schemas in the same commit so they drift together","Generate fixture data from real source samples","Run the debug pipeline in CI to catch fixture/schema divergence immediately"],"tags":["pathway","debug","schema-mismatch","test-fixture","datasource"],"backgroundTag":null,"analyzedSha":"fa2f74a4649b7c5908690cf60137263d8d80de5f","analyzedAt":"2026-08-15T01:48:17.006Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}