sgl-project/sglang · error · ValueError

Invalid modality '{modality}' in --limit-mm-data-per-request

Error message

Invalid modality '{modality}' in --limit-mm-data-per-request.Allowed modalities are: {list(allowed_modalities)}

What it means

Raised while validating --limit-mm-data-per-request when a key in the per-modality limit dict is not one of image, video, audio. The dict maps modality name to a per-request item cap; unknown modality names are rejected.

Source

Thrown at python/sglang/srt/server_args.py:9659

                cuda_graph_config=with_phase(
                    cfg.cuda_graph_config, Phase.PREFILL, backend=Backend.DISABLED
                ),
            )
            self._declare("_handle_other_validations", skip_server_warmup=True)

        # Validate limit_mm_per_prompt modalities
        if cfg.limit_mm_data_per_request:
            if isinstance(cfg.limit_mm_data_per_request, str):
                self._declare(
                    "_handle_other_validations",
                    limit_mm_data_per_request=json.loads(cfg.limit_mm_data_per_request),
                )

            if isinstance(cfg.limit_mm_data_per_request, dict):
                allowed_modalities = {"image", "video", "audio"}
                for modality in cfg.limit_mm_data_per_request.keys():
                    if modality not in allowed_modalities:
                        raise ValueError(
                            f"Invalid modality '{modality}' in --limit-mm-data-per-request."
                            f"Allowed modalities are: {list(allowed_modalities)}"
                        )

        # Validate preferred_sampling_params
        if cfg.preferred_sampling_params:
            if isinstance(cfg.preferred_sampling_params, str):
                self._declare(
                    "_handle_other_validations",
                    preferred_sampling_params=json.loads(cfg.preferred_sampling_params),
                )

            # Validate preferred_sampling_params doesn't use tokenizer-dependent features
            if cfg.skip_tokenizer_init:
                from sglang.srt.sampling.sampling_params import SamplingParams

                test_params = SamplingParams(**cfg.preferred_sampling_params)
                # raises if tokenizer-dependent features used

View on GitHub (pinned to 0132848349)

Solutions

1. Use exact keys: image, video, audio
2. Remove limits for modalities you don't need rather than inventing keys
3. Check the model's supported modalities before limiting

Example fix

# before
--limit-mm-data-per-request '{"images": 1}'
# after
--limit-mm-data-per-request '{"image": 1}'

allowed = {'image', 'video', 'audio'}
if isinstance(args.limit_mm_data_per_request, dict):
    bad = set(args.limit_mm_data_per_request) - allowed
    assert not bad, f'invalid modalities: {bad}; allowed: {sorted(allowed)}'

def valid_mm_limits(d: dict) -> bool:
    return set(d) <= {'image', 'video', 'audio'}

When it happens

Trigger: Passing e.g. --limit-mm-data-per-request '{"images": 1}' (plural), '{"text": 1}', or any key outside {image, video, audio}.

Common situations: Pluralized keys ('images' instead of 'image'); assuming a newer modality (e.g. 'video' variants) is supported in an older SGLang; typos in JSON keys.

Related errors

• Decode context parallel size (--dcp-size / --decode-context-
• mm_preprocess_cache_size_mb must be non-negative
• --mamba-max-states-per-path must be -1 (unlimited) or a posi
• --swa-full-tokens-ratio should be in range (0, 1.0].
• --asr-max-buffer-seconds must be positive (got {cfg.asr_max_

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/d26495fd68cc91e8. Report an issue: GitHub.