{"record":{"id":"80676639fb14f148","repo":"pathwaycom/pathway","slug":"resulting-universe-does-not-match-the-universe-of","errorCode":null,"errorMessage":"resulting universe does not match the universe of the indicated argument","messagePattern":"resulting universe does not match the universe of the indicated argument","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/pathway/stdlib/utils/pandas_transformer.py","lineNumber":73,"sourceCode":"    func_spec: FunctionSpec,\n    output_schema: type[schema.Schema],\n    output_universe: str | int | None,\n) -> pw.Table:\n    output_universe_arg_index = _argument_index(func_spec, output_universe)\n    func = func_spec.func\n\n    def process_pandas_output(\n        result: pd.DataFrame | pd.Series, pandas_input: list[pd.DataFrame] = []\n    ):\n        if isinstance(result, pd.Series):\n            result = pd.DataFrame(result)\n\n        result.columns = output_schema.column_names()  # type: ignore\n\n        if output_universe_arg_index is not None and not result.index.equals(\n            pandas_input[output_universe_arg_index].index\n        ):\n            raise ValueError(\n                \"resulting universe does not match the universe of the indicated argument\"\n            )\n        else:\n            if not result.index.is_unique:\n                raise ValueError(\"index of resulting DataFrame must be unique\")\n            index_as_series = result.index.to_series()\n            if not index_as_series.map(lambda x: isinstance(x, Pointer)).all():\n                new_index = index_as_series.map(lambda x: ref_scalar(x))\n                result.reindex(new_index)\n            assert result.index.is_unique\n\n        return result\n\n    if len(func_spec.arg_names) == 0:\n        result = func()\n        result = process_pandas_output(result)\n        output = table_from_pandas(result)\n    else:","sourceCodeStart":55,"sourceCodeEnd":91,"githubUrl":"https://github.com/pathwaycom/pathway/blob/fa2f74a4649b7c5908690cf60137263d8d80de5f/python/pathway/stdlib/utils/pandas_transformer.py#L55-L91","documentation":"In a pandas transformer with output_universe set, process_pandas_output verifies that the DataFrame returned by the UDF has exactly the same index (same order, same values) as the pandas input of the designated argument, because that index becomes the output table's universe. If result.index does not .equals() the chosen input's index, this ValueError is raised at runtime on every batch.","triggerScenarios":"The UDF calls reset_index(), sort_values(), drop_duplicates(), or join/merge that reorders rows; returning a DataFrame built from scratch instead of preserving the input index; filtering rows out of the designated input's frame.","commonSituations":"Wrapping existing pandas code (sorting, groupby-then-merge) as a Pathway UDF; using merge which produces a new RangeIndex; chaining .head() or column-based filtering inside the UDF.","solutions":["Preserve the designated argument's index: operate column-wise and return a frame indexed like that input (e.g. build the result with pd.DataFrame({...}, index=input.index))","If you must transform, restore the index before returning: result = result.set_index(designated_input.index) or reindex to the original order","Avoid merge/reset_index/sort_values on the universe-defining frame; apply them only to value columns"],"exampleFix":"# before\n@pw.pandas_transformer(output_universe='df')\ndef f(df: pd.DataFrame) -> pd.DataFrame:\n    return df.assign(y=df.x * 2).sort_values('y')  # index reordered\n\n# after\n@pw.pandas_transformer(output_universe='df')\ndef f(df: pd.DataFrame) -> pd.DataFrame:\n    return pd.DataFrame({'y': df.x * 2, 'z': df.x + 1}, index=df.index)","handlingStrategy":"validation","validationCode":"# inside the UDF, before returning:\nassert result.index.equals(designated_input.index), 'output index must equal the output_universe argument index'","typeGuard":null,"tryCatchPattern":"try:\n    out = pw.Table.from_pandas(udf_output)  # or the transformer call\nexcept ValueError as e:\n    if 'resulting universe does not match' in str(e):\n        raise ValueError('UDF reordered the output index; rebuild result with input.index') from e\n    raise","preventionTips":["Construct result frames with pd.DataFrame({...}, index=designated_input.index)","Never call reset_index/sort_values/merge on the universe-defining frame inside the UDF","Add a UDF unit test asserting result.index.equals(input.index)"],"tags":["pathway","pandas","udf","universe","index-mismatch"],"backgroundTag":null,"analyzedSha":"fa2f74a4649b7c5908690cf60137263d8d80de5f","analyzedAt":"2026-08-15T01:48:17.006Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}