{"record":{"id":"666ee3542e9ef281","repo":"pathwaycom/pathway","slug":"schema-does-not-have-columns-missing-columns","errorCode":null,"errorMessage":"schema does not have columns {missing_columns}","messagePattern":"schema does not have columns (.+?)","errorType":"exception","errorClass":"AssertionError","httpStatus":null,"severity":"error","filePath":"python/pathway/internals/schema.py","lineNumber":614,"sourceCode":"        with open(path, mode=\"w\") as f:\n            f.write(class_definition)\n\n    def assert_matches_schema(\n        self,\n        other: type[Schema],\n        *,\n        allow_superset: bool = True,\n        ignore_primary_keys: bool = True,\n        allow_subtype: bool = True,\n        ignore_properties: bool = True,\n    ) -> None:\n        self_dict = self._dtypes()\n        other_dict = other._dtypes()\n\n        # Check if self has all columns of other\n        if self_dict.keys() < other_dict.keys():\n            missing_columns = other_dict.keys() - self_dict.keys()\n            raise AssertionError(f\"schema does not have columns {missing_columns}\")\n\n        # Check if types of columns are the same\n        for col in other_dict:\n            assert other_dict[col] == self_dict[col] or (\n                allow_subtype and dt.dtype_issubclass(self_dict[col], other_dict[col])\n            ), (\n                f\"type of column {col} does not match - its type is {self_dict[col]} in {self.__name__}\",\n                f\" and {other_dict[col]} in {other.__name__}\",\n            )\n\n        # When allow_superset=False, check that self does not have extra columns\n        if not allow_superset and self_dict.keys() > other_dict.keys():\n            extra_columns = self_dict.keys() - other_dict.keys()\n            raise AssertionError(\n                f\"there are extra columns: {extra_columns} which are not present in the provided schema\"\n            )\n\n        # Check whether primary keys are the same","sourceCodeStart":596,"sourceCodeEnd":632,"githubUrl":"https://github.com/pathwaycom/pathway/blob/fa2f74a4649b7c5908690cf60137263d8d80de5f/python/pathway/internals/schema.py#L596-L632","documentation":"Schema.assert_same_schema_as(other) (and schema assertions used in tests / connector validation) first checks that self contains all columns of other. If self's column set is missing some of other's columns, the missing names are collected and reported with this AssertionError — this is the subset check, raised before any type comparison.","triggerScenarios":"Calling table_a.schema.assert_same_schema_as(table_b.schema) where table_a lacks columns present in table_b; also assert_schema_correct style validation in io connectors with a user-declared schema missing declared fields.","commonSituations":"Test assertions after select/reduce that dropped columns; input data not containing all columns declared in a schema passed to pw.io.*.read; refactors that removed columns without updating the expected schema.","solutions":["Add the missing columns (e.g. with_columns(missing_col=pw.const(None)) or restore them in select) so self covers other.","If the smaller schema is intended, compare against it instead: swap arguments so the superset schema is 'self', or trim other with .without(...).","For connector reads, ensure the declared schema only lists columns actually present in the data."],"exampleFix":"# before\na.schema.assert_same_schema_as(expected.schema)  # a lacks 'extra'\n\n# after\na_ext = a.with_columns(extra=pw.const(None).astype(int))\na_ext.schema.assert_same_schema_as(expected.schema)","handlingStrategy":"validation","validationCode":"import pathway as pw\n\ndef covers_all_columns(self_schema, other_schema) -> bool:\n    return set(other_schema.keys()) <= set(self_schema.keys())","typeGuard":null,"tryCatchPattern":"try:\n    actual.schema.assert_same_schema_as(expected.schema)\nexcept AssertionError as e:\n    if 'does not have columns' in str(e):\n        missing = set(expected.column_names()) - set(actual.column_names())\n        actual = actual.with_columns(**{c: pw.const(None) for c in missing})\n    else:\n        raise","preventionTips":["In tests, compare column sets explicitly before calling assert_same_schema_as.","Keep connector-declared schemas minimal: only columns the data actually contains."],"tags":["pathway","schema","assertion","validation","testing"],"backgroundTag":null,"analyzedSha":"fa2f74a4649b7c5908690cf60137263d8d80de5f","analyzedAt":"2026-08-15T01:48:17.006Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}