pathwaycom/pathway · error · ValueError

resulting universe does not match the universe of the indica

Error message

resulting universe does not match the universe of the indicated argument

What it means

In a pandas transformer with output_universe set, process_pandas_output verifies that the DataFrame returned by the UDF has exactly the same index (same order, same values) as the pandas input of the designated argument, because that index becomes the output table's universe. If result.index does not .equals() the chosen input's index, this ValueError is raised at runtime on every batch.

Source

Thrown at python/pathway/stdlib/utils/pandas_transformer.py:73

    func_spec: FunctionSpec,
    output_schema: type[schema.Schema],
    output_universe: str | int | None,
) -> pw.Table:
    output_universe_arg_index = _argument_index(func_spec, output_universe)
    func = func_spec.func

    def process_pandas_output(
        result: pd.DataFrame | pd.Series, pandas_input: list[pd.DataFrame] = []
    ):
        if isinstance(result, pd.Series):
            result = pd.DataFrame(result)

        result.columns = output_schema.column_names()  # type: ignore

        if output_universe_arg_index is not None and not result.index.equals(
            pandas_input[output_universe_arg_index].index
        ):
            raise ValueError(
                "resulting universe does not match the universe of the indicated argument"
            )
        else:
            if not result.index.is_unique:
                raise ValueError("index of resulting DataFrame must be unique")
            index_as_series = result.index.to_series()
            if not index_as_series.map(lambda x: isinstance(x, Pointer)).all():
                new_index = index_as_series.map(lambda x: ref_scalar(x))
                result.reindex(new_index)
            assert result.index.is_unique

        return result

    if len(func_spec.arg_names) == 0:
        result = func()
        result = process_pandas_output(result)
        output = table_from_pandas(result)
    else:

View on GitHub (pinned to fa2f74a464)

Solutions

1. Preserve the designated argument's index: operate column-wise and return a frame indexed like that input (e.g. build the result with pd.DataFrame({...}, index=input.index))
2. If you must transform, restore the index before returning: result = result.set_index(designated_input.index) or reindex to the original order
3. Avoid merge/reset_index/sort_values on the universe-defining frame; apply them only to value columns

Example fix

# before
@pw.pandas_transformer(output_universe='df')
def f(df: pd.DataFrame) -> pd.DataFrame:
    return df.assign(y=df.x * 2).sort_values('y')  # index reordered

# after
@pw.pandas_transformer(output_universe='df')
def f(df: pd.DataFrame) -> pd.DataFrame:
    return pd.DataFrame({'y': df.x * 2, 'z': df.x + 1}, index=df.index)

# inside the UDF, before returning:
assert result.index.equals(designated_input.index), 'output index must equal the output_universe argument index'

try:
    out = pw.Table.from_pandas(udf_output)  # or the transformer call
except ValueError as e:
    if 'resulting universe does not match' in str(e):
        raise ValueError('UDF reordered the output index; rebuild result with input.index') from e
    raise

When it happens

Trigger: The UDF calls reset_index(), sort_values(), drop_duplicates(), or join/merge that reorders rows; returning a DataFrame built from scratch instead of preserving the input index; filtering rows out of the designated input's frame.

Common situations: Wrapping existing pandas code (sorting, groupby-then-merge) as a Pathway UDF; using merge which produces a new RangeIndex; chaining .head() or column-based filtering inside the UDF.

Related errors

• index of resulting DataFrame must be unique
• wrong output universe. No argument of name: {arg}
• wrong output universe. Index out of range
• string {freq} cannot be parsed as a duration
• You cannot use {dep.to_column_expression()} in this context.

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/80676639fb14f148. Report an issue: GitHub.