{"record":{"id":"7aa5b6a161f038e9","repo":"sgl-project/sglang","slug":"unknown-data-df-columns-you-may-need-to-set","errorCode":null,"errorMessage":"Unknown data: {df.columns}. You may need to set `--data-type` if using e.g. simple_evals.","messagePattern":"Unknown data: (.+?)\\. You may need to set `--data-type` if using e\\.g\\. simple_evals\\.","errorType":"exception","errorClass":"Exception","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/debug_utils/text_comparator.py","lineNumber":168,"sourceCode":"            filter_name = filter_names[0]\n            print(f\"Choose {filter_name=} among {filter_names}\")\n            df = df.filter(pl.col(\"filter\") == filter_name)\n\n        df = df.select(\n            pl.col(\"category\"),\n            pl.col(\"trial_index\"),\n            prompt_id=pl.col(\"doc_id\"),\n            prompt=pl.col(\"arguments\").struct.field(\"gen_args_0\").struct.field(\"arg_0\"),\n            output=pl.col(\"resps\").list.get(0).list.get(0),\n            correct=pl.col(\"exact_match\").cast(bool),\n        )\n\n        return df\n    elif \"prompt_id\" in df.columns:\n        print(\"Transform mode: SGLang bench\")\n        return df\n    else:\n        raise Exception(\n            f\"Unknown data: {df.columns}. You may need to set `--data-type` if using e.g. simple_evals.\"\n        )\n\n\ndef _compute_df_meta(df_input: pl.DataFrame):\n    df_input = df_input.sort(\"prompt_id\", \"category\", \"trial_index\")\n    df_meta = pl.DataFrame(\n        [\n            _handle_one_prompt(df_one_prompt)\n            for df_one_prompt in df_input.partition_by(\"prompt_id\", maintain_order=True)\n        ]\n    )\n    df_meta = df_meta.with_columns(\n        correctness_delta=pl.col(\"correctness_target\") - pl.col(\"correctness_baseline\"),\n    )\n    df_meta = df_meta.sort(\"correctness_delta\", \"output_same_prefix_len\")\n    return df_meta\n","sourceCodeStart":150,"sourceCodeEnd":186,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/debug_utils/text_comparator.py#L150-L186","documentation":"The text comparator CLI transforms an input DataFrame (polars) before comparing model outputs, recognizing schemas by their columns. If the DataFrame has neither the expected simple_evals-style columns nor a `prompt_id` column, the transform mode cannot be inferred and it raises a generic Exception telling you to pass --data-type.","triggerScenarios":"Running the text_comparator main() with a parquet/csv input whose column set matches no known schema (no `prompt_id` column and not the simple_evals layout).","commonSituations":"Feeding a benchmark output file from a different harness (e.g. simple_evals exports) without setting --data-type; a column renamed by a newer version of the eval script; hand-built parquet with custom headers.","solutions":["Inspect df.columns (printed in the message) and pass --data-type matching the actual producer (e.g. simple_evals)","Rename your DataFrame columns to one of the supported schemas (e.g. add prompt_id/category/trial_index for SGLang bench mode)","Re-export the data from the tool that produced it so the schema is preserved"],"exampleFix":"# before\npython -m sglang.srt.debug_utils.text_comparator results.parquet ...\n# after\npython -m sglang.srt.debug_utils.text_comparator results.parquet --data-type simple_evals ...","handlingStrategy":"type-guard","validationCode":"cols = set(df.columns)\\nif 'prompt_id' not in cols and not cols >= {'question_id','model_response'}:  # simple_evals schema\\n    raise ValueError(f'unsupported columns {cols}; pass --data-type')","typeGuard":"def is_sgl_bench_df(df) -> bool:\\n    return 'prompt_id' in df.columns","tryCatchPattern":"try:\\n    df = _transform_df_input(df)\\nexcept Exception as e:\\n    print(df.columns)\\n    raise SystemExit('pass --data-type matching the producer tool') from e","preventionTips":["Always pass --data-type when inputs come from external eval harnesses","Pin the export script version that produced parquet files","Validate the column schema right after loading, before transform"],"tags":["data-schema","polars","cli","text-comparison"],"backgroundTag":"unrecognized-data-schema","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}