sgl-project/sglang · critical · ValueError

Error: --model-type requires a non-empty value.

Error message

Error: --model-type requires a non-empty value.

What it means

The bfloat16 branch of deterministic_all_reduce supports only world sizes 2,4,6,8; anything else throws at the dispatch switch.

Source

Thrown at python/sglang/cli/serve.py:51

    while i < len(extra_argv):
        arg = extra_argv[i]
        if arg == "--model-type":
            if i + 1 >= len(extra_argv):
                raise ValueError("Error: --model-type requires a value.")
            backend_name = extra_argv[i + 1]
            i += 2
            continue

        if arg.startswith("--model-type="):
            backend_name = arg.split("=", 1)[1]
            i += 1
            continue

        filtered_argv.append(arg)
        i += 1

    if not backend_name:
        raise ValueError("Error: --model-type requires a non-empty value.")
    return backend_name, filtered_argv


def _normalize_positional_model_path(extra_argv):
    """Allow `sglang serve <model>` while preserving existing flag parsing."""
    if extra_argv and not extra_argv[0].startswith("-"):
        return ["--model-path", extra_argv[0], *extra_argv[1:]], True
    return extra_argv, False


def _print_llm_help(_request: ServeRequest) -> None:
    from sglang.srt.server_args import prepare_server_args

    try:
        prepare_server_args(["--help"])
    except SystemExit:
        pass  # argparse --help calls sys.exit

View on GitHub (pinned to 0132848349)

Solutions

1. Run TP in {2,4,6,8}
2. Disable the deterministic custom kernel on unsupported topologies
3. Verify visible GPU count matches intended TP

assert fa.world_size_ in (2,4,6,8), 'deterministic bf16 allreduce needs TP in (2,4,6,8)'

When it happens

Trigger: Deterministic allreduce with bf16 tensors on a CustomAllreduce with world_size_ outside {2,4,6,8}.

Common situations: Deterministic inference enabled on odd or >8-GPU ROCm topologies with bf16 models (e.g. TP=3, TP=16).

Related errors

• This browser does not support worker image decoding
• Generate subcommand is not yet supported for model: {model_p
• Error: --model-type requires a value.
• delta payload size mismatch: expected ${expectedSize}, got $
• Missing previous frame for delta payload

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/01b1004c4625c867. Report an issue: GitHub.