sgl-project/sglang · error · Exception

Unknown data: {df.columns}. You may need to set `--data-type

Error message

Unknown data: {df.columns}. You may need to set `--data-type` if using e.g. simple_evals.

What it means

The text comparator CLI transforms an input DataFrame (polars) before comparing model outputs, recognizing schemas by their columns. If the DataFrame has neither the expected simple_evals-style columns nor a `prompt_id` column, the transform mode cannot be inferred and it raises a generic Exception telling you to pass --data-type.

Source

Thrown at python/sglang/srt/debug_utils/text_comparator.py:168

            filter_name = filter_names[0]
            print(f"Choose {filter_name=} among {filter_names}")
            df = df.filter(pl.col("filter") == filter_name)

        df = df.select(
            pl.col("category"),
            pl.col("trial_index"),
            prompt_id=pl.col("doc_id"),
            prompt=pl.col("arguments").struct.field("gen_args_0").struct.field("arg_0"),
            output=pl.col("resps").list.get(0).list.get(0),
            correct=pl.col("exact_match").cast(bool),
        )

        return df
    elif "prompt_id" in df.columns:
        print("Transform mode: SGLang bench")
        return df
    else:
        raise Exception(
            f"Unknown data: {df.columns}. You may need to set `--data-type` if using e.g. simple_evals."
        )


def _compute_df_meta(df_input: pl.DataFrame):
    df_input = df_input.sort("prompt_id", "category", "trial_index")
    df_meta = pl.DataFrame(
        [
            _handle_one_prompt(df_one_prompt)
            for df_one_prompt in df_input.partition_by("prompt_id", maintain_order=True)
        ]
    )
    df_meta = df_meta.with_columns(
        correctness_delta=pl.col("correctness_target") - pl.col("correctness_baseline"),
    )
    df_meta = df_meta.sort("correctness_delta", "output_same_prefix_len")
    return df_meta

View on GitHub (pinned to 0132848349)

Solutions

Inspect df.columns (printed in the message) and pass --data-type matching the actual producer (e.g. simple_evals)
Rename your DataFrame columns to one of the supported schemas (e.g. add prompt_id/category/trial_index for SGLang bench mode)
Re-export the data from the tool that produced it so the schema is preserved

Example fix

# before
python -m sglang.srt.debug_utils.text_comparator results.parquet ...
# after
python -m sglang.srt.debug_utils.text_comparator results.parquet --data-type simple_evals ...

Defensive patterns

Strategy: type-guard

Validate before calling

cols = set(df.columns)\nif 'prompt_id' not in cols and not cols >= {'question_id','model_response'}:  # simple_evals schema\n    raise ValueError(f'unsupported columns {cols}; pass --data-type')

Type guard

def is_sgl_bench_df(df) -> bool:\n    return 'prompt_id' in df.columns

Try / catch

try:\n    df = _transform_df_input(df)\nexcept Exception as e:\n    print(df.columns)\n    raise SystemExit('pass --data-type matching the producer tool') from e

Prevention

Always pass --data-type when inputs come from external eval harnesses
Pin the export script version that produced parquet files
Validate the column schema right after loading, before transform

When it happens

Trigger: Running the text_comparator main() with a parquet/csv input whose column set matches no known schema (no `prompt_id` column and not the simple_evals layout).

Common situations: Feeding a benchmark output file from a different harness (e.g. simple_evals exports) without setting --data-type; a column renamed by a newer version of the eval script; hand-built parquet with custom headers.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/7aa5b6a161f038e9. Report an issue: GitHub.