sgl-project/sglang · error · ValueError

Error: --model-type requires a value.

Error message

Error: --model-type requires a value.

What it means

In the BFloat16 branch (gated by __HIP_ARCH__ >= 800), deterministic_all_reduce requires bf16 element counts divisible by the pack width P::size.

Source

Thrown at python/sglang/cli/serve.py:37

logger = logging.getLogger(__name__)


def _extract_model_type_override(extra_argv):
    """Extract and remove the backend selector from argv.

    Validation is deferred to :class:`ServeBackendRegistry` because installed
    out-of-tree entry points dynamically extend the accepted values.
    """

    backend_name = "auto"
    filtered_argv = []
    i = 0
    while i < len(extra_argv):
        arg = extra_argv[i]
        if arg == "--model-type":
            if i + 1 >= len(extra_argv):
                raise ValueError("Error: --model-type requires a value.")
            backend_name = extra_argv[i + 1]
            i += 2
            continue

        if arg.startswith("--model-type="):
            backend_name = arg.split("=", 1)[1]
            i += 1
            continue

        filtered_argv.append(arg)
        i += 1

    if not backend_name:
        raise ValueError("Error: --model-type requires a non-empty value.")
    return backend_name, filtered_argv


def _normalize_positional_model_path(extra_argv):

View on GitHub (pinned to 0132848349)

Solutions

1. Pad to a multiple of the pack width and slice after
2. Choose hidden sizes divisible by 8
3. Use RCCL for non-conforming tensors

d = 8  # bf16 pack width
assert out.numel() % d == 0, f'size must be multiple of {d}'

When it happens

Trigger: Calling deterministic allreduce with a bfloat16 tensor whose numel is not a multiple of the pack width.

Common situations: bf16 models with hidden dims not divisible by the pack width; sliced/dynamic-shape tensors in deterministic mode on ROCm.

Related errors

• Previous frame size does not match current delta payload
• Unsupported content type ${header.content_type}
• Error: --model-type requires a non-empty value.
• Missing previous frame for delta payload
• This browser does not support worker image decoding

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/5c2cbfa767e4e716. Report an issue: GitHub.