pathwaycom/pathway · error · ValueError

columns do not match in the argument of Table.concat(). Miss

Error message

columns do not match in the argument of Table.concat(). Missing columns: {missing_keys}. Superfluous columns: {superfluous_keys}.

What it means

Raised by Table.concat() (via the public concat path) when an argument table's column set differs from self's. concat stacks tables vertically, so every table must expose exactly the same column names before ids are unioned. The error lists which columns are missing and which are superfluous.

Source

Thrown at python/pathway/internals/table.py:1635

        ... 12 | 12  | Tom   | 40
        ... ''')
        >>> pw.universes.promise_are_pairwise_disjoint(t1, t2)
        >>> t3 = t1.concat(t2)
        >>> pw.debug.compute_and_print(t3, include_id=False)
        age | owner | pet
        8   | Alice | 2
        9   | Bob   | 1
        10  | Alice | 1
        11  | Alice | 30
        12  | Tom   | 40
        """
        for other in others:
            if other.keys() != self.keys():
                self_keys = set(self.keys())
                other_keys = set(other.keys())
                missing_keys = self_keys - other_keys
                superfluous_keys = other_keys - self_keys
                raise ValueError(
                    "columns do not match in the argument of Table.concat()."
                    + (
                        f" Missing columns: {missing_keys}."
                        if missing_keys is not None
                        else ""
                    )
                    + (
                        f" Superfluous columns: {superfluous_keys}."
                        if superfluous_keys is not None
                        else ""
                    )
                )
        schema = {}
        all_args: list[Table] = [self, *others]

        for key in self.keys():
            schema[key] = _types_lca_with_error(
                *[arg.schema._dtypes()[key] for arg in all_args],

View on GitHub (pinned to fa2f74a464)

Solutions

1. Project both tables to a common column set: t2 = t2.select(*t1.keys()) or t2.select(t1.colA, t1.colB)
2. Rename mismatched columns first with t2 = t2.rename(correct='wrong') so key sets match
3. Add missing constant columns: t2 = t2.with_columns(dummy=pw.const(None)) to fill gaps
4. Check schemas before concat: assert set(t1.keys()) == set(t2.keys())

Example fix

# before
t3 = pw.Table.concat(t1, t2)  # t2 has extra column 'ts'

# after
t3 = pw.Table.concat(t1, t2.select(*t1.keys()))

def concat_safe(t1, t2):
    assert set(t1.keys()) == set(t2.keys()), (
        f'missing={set(t1.keys())-set(t2.keys())} '
        f'superfluous={set(t2.keys())-set(t1.keys())}'
    )
    return pw.Table.concat(t1, t2)

try:
    t3 = pw.Table.concat(t1, t2)
except ValueError as e:
    if 'columns do not match' in str(e):
        t3 = pw.Table.concat(t1, t2.select(*t1.keys()))

When it happens

Trigger: pw.Table.concat(t1, t2) (or t1.concat(t2)) where t2.keys() != t1.keys(); e.g. t2 has an extra 'timestamp' column or lacks 'age'.

Common situations: Concatenating outputs of different connectors or API responses whose schemas drifted; one branch of a pipeline added a derived column; renamed columns on one table only.

Related errors

• Columns of the argument in Table.update_cells() not present
• Columns do not match between argument of Table.update_rows()
• Column {old_name} does not exist in a given table.
• Table.with_schema() argument has to have the same column nam
• Failed to find the column '{column._name}' in table {column.

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/e8309e81fff336db. Report an issue: GitHub.