pathwaycom/pathway · error · TypeError

Cannot flatten column of type {dtype}.

Error message

Cannot flatten column of type {dtype}.

What it means

Table.flatten() computes the output dtype by inspecting the input column's type: LIST/ARRAY (minus one dimension), STR (chars), JSON, and ANY are supported. For any other dtype (INT, FLOAT, BOOL, DATE, etc.) there is no element type to flatten to, so a TypeError is raised at graph-construction time.

Source

Thrown at python/pathway/internals/column.py:1085

            return dtype.wrapped
        if isinstance(dtype, dt.Tuple):
            if dtype in (dt.ANY_TUPLE, dt.Tuple()):
                return dt.ANY
            assert not isinstance(dtype.args, EllipsisType)
            return_dtype = dtype.args[0]
            for single_dtype in dtype.args[1:]:
                return_dtype = dt.types_lca(return_dtype, single_dtype, raising=False)
            return return_dtype
        elif dtype == dt.STR:
            return dt.STR
        elif dtype == dt.ANY:
            return dt.ANY
        elif isinstance(dtype, dt.Array):
            return dtype.strip_dimension()
        elif dtype == dt.JSON:
            return dt.JSON
        else:
            raise TypeError(f"Cannot flatten column of type {dtype}.")

    @cached_property
    def universe(self) -> Universe:
        ret = Universe()
        if self.orig_universe.is_empty():
            ret.register_as_empty(no_warn=False)
        return ret

    @cached_property
    def flatten_result_column(self) -> Column:
        return MaterializedColumn(
            self.universe,
            cp.ColumnProperties(
                dtype=self._get_flatten_column_dtype(),
                append_only=self.flatten_column.properties.append_only,
            ),
        )

View on GitHub (pinned to fa2f74a464)

Solutions

1. Verify the column dtype with t.schema or t.flatten_column.dtype before calling flatten.
2. Flatten only LIST/ARRAY columns; for strings use t.column.dt.flatten() semantics (string flattening is supported for STR) — do not flatten numeric columns.
3. If the column should be a list, fix the input schema/connector type mapping so the column is parsed as list[T].

Example fix

# before
t = t.select(t.numbers)  # numbers: int due to wrong schema
result = t.flatten()

# after
# fix the schema so the column is a list
class InputSchema(pw.Schema):
    values: list[int]
t = pw.io.csv.read(path, schema=InputSchema)
result = t.flatten()

import pathway as dt_types
from pathway.internals import dtype as dt
col_dtype = table.flatten_column.dtype
flattenable = col_dtype in (dt.STR, dt.ANY, dt.JSON) or isinstance(col_dtype, (dt.List, dt.Array))
assert flattenable, f'cannot flatten dtype {col_dtype}'

from pathway.internals import dtype as dt

def dtype_flattenable(d) -> bool:
    return d in (dt.STR, dt.ANY, dt.JSON) or isinstance(d, (dt.List, dt.Array))

When it happens

Trigger: Calling t.flatten() (Table.flatten, which flattens t.this) or column.flatten() on a column typed int, float, bool, Optional[int], Pointer, DATE_TIME, or a tuple type not handled by the LCA branch.

Common situations: Applying flatten to a column whose schema declares a scalar; assuming flatten works like pandas explode on any dtype; dtype ANY_json vs plain scalars confusion after schema changes.

Related errors

• direction argument of join should be of type asof_join.Direc
• The interval argument of a join should be of a type pathway.
• Expected a ColumnReference, found a string. Did you mean thi
• not supported type of debug data
• DateTimeNaive cannot contain timezone information. Use pw.Da

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/81c94275d03a35b1. Report an issue: GitHub.