{"record":{"id":"0e61fb55eef52bcf","repo":"pathwaycom/pathway","slug":"there-are-extra-columns-extra-columns-which-are","errorCode":null,"errorMessage":"there are extra columns: {extra_columns} which are not present in the provided schema","messagePattern":"there are extra columns: (.+?) which are not present in the provided schema","errorType":"exception","errorClass":"AssertionError","httpStatus":null,"severity":"error","filePath":"python/pathway/internals/schema.py","lineNumber":628,"sourceCode":"\n        # Check if self has all columns of other\n        if self_dict.keys() < other_dict.keys():\n            missing_columns = other_dict.keys() - self_dict.keys()\n            raise AssertionError(f\"schema does not have columns {missing_columns}\")\n\n        # Check if types of columns are the same\n        for col in other_dict:\n            assert other_dict[col] == self_dict[col] or (\n                allow_subtype and dt.dtype_issubclass(self_dict[col], other_dict[col])\n            ), (\n                f\"type of column {col} does not match - its type is {self_dict[col]} in {self.__name__}\",\n                f\" and {other_dict[col]} in {other.__name__}\",\n            )\n\n        # When allow_superset=False, check that self does not have extra columns\n        if not allow_superset and self_dict.keys() > other_dict.keys():\n            extra_columns = self_dict.keys() - other_dict.keys()\n            raise AssertionError(\n                f\"there are extra columns: {extra_columns} which are not present in the provided schema\"\n            )\n\n        # Check whether primary keys are the same\n        if not ignore_primary_keys:\n            assert self.primary_key_columns() == other.primary_key_columns(), (\n                f\"primary keys in the schemas do not match - they are {self.primary_key_columns()} in {self.__name__}\",\n                f\" and {other.primary_key_columns()} in {other.__name__}\",\n            )\n        if not ignore_properties:\n            self_columns = self.columns()\n            other_columns = other.columns()\n            for column_name, column_schema in self_columns.items():\n                other_column_schema = other_columns[column_name]\n                assert column_schema == other_column_schema\n\n\ndef _schema_builder(","sourceCodeStart":610,"sourceCodeEnd":646,"githubUrl":"https://github.com/pathwaycom/pathway/blob/fa2f74a4649b7c5908690cf60137263d8d80de5f/python/pathway/internals/schema.py#L610-L646","documentation":"The second phase of assert_same_schema_as: with allow_superset=False, self must not have columns beyond other's. Extra columns in self (present in the actual schema but not in the expected one) are collected and reported by this AssertionError.","triggerScenarios":"table.schema.assert_same_schema_as(expected.schema, allow_superset=False) while table has additional columns (e.g. an upstream connector added metadata columns, or select kept too many columns).","commonSituations":"Strict schema checks in tests after adding a derived column; connectors appending internal columns (like _timestamped or partition metadata); comparing a widened schema against a strict expected schema.","solutions":["Drop the extra columns before comparing: table.select(*expected_columns) or schema.without(*extra).","If extras are acceptable, call with allow_superset=True (the default).","Update the expected schema to include the new columns if they are intentional."],"exampleFix":"# before\nactual.schema.assert_same_schema_as(expected.schema, allow_superset=False)\n# AssertionError: extra columns: {'meta'}\n\n# after\nactual.select(*expected.column_names()).schema.assert_same_schema_as(\n    expected.schema, allow_superset=False\n)","handlingStrategy":"validation","validationCode":"import pathway as pw\n\ndef no_extra_columns(self_schema, other_schema) -> bool:\n    return set(self_schema.keys()) <= set(other_schema.keys())","typeGuard":null,"tryCatchPattern":"try:\n    actual.schema.assert_same_schema_as(expected.schema, allow_superset=False)\nexcept AssertionError as e:\n    if 'extra columns' in str(e):\n        actual = actual.select(*expected.column_names())\n        actual.schema.assert_same_schema_as(expected.schema, allow_superset=False)\n    else:\n        raise","preventionTips":["Project to the expected columns (select(*expected.column_names())) before strict comparisons.","Use allow_superset=True unless a strict contract genuinely requires no extras."],"tags":["pathway","schema","assertion","validation","testing"],"backgroundTag":null,"analyzedSha":"fa2f74a4649b7c5908690cf60137263d8d80de5f","analyzedAt":"2026-08-15T01:48:17.006Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}