vllm-project/vllm · error · ValueError

'mm_encoder_fp8_scale_path' and…

Error message

'mm_encoder_fp8_scale_path' and 'mm_encoder_fp8_scale_save_path' require 'mm_encoder_attn_dtype' to be 'fp8'.

What it means

MultiModalConfig's post-init validator requires mm_encoder_attn_dtype to be 'fp8' when either mm_encoder_fp8_scale_path or mm_encoder_fp8_scale_save_path is set. Static or saved FP8 scale files only apply to an encoder running FP8 attention, so any other dtype makes them meaningless and is rejected.

Solutions

Add --mm-encoder-attn-dtype fp8 alongside the scale path flags.
If you do not want FP8 encoder attention, remove --mm-encoder-fp8-scale-path and --mm-encoder-fp8-scale-save-path.

Example fix

# before
vllm serve Qwen/Qwen2.5-VL-7B --mm-encoder-fp8-scale-path scales.pt

# after
vllm serve Qwen/Qwen2.5-VL-7B --mm-encoder-attn-dtype fp8 --mm-encoder-fp8-scale-path scales.pt

Defensive patterns

Strategy: validation

Validate before calling

def check_fp8_opts(attn_dtype, scale_path, save_path):
    if (scale_path or save_path) and attn_dtype != "fp8":
        raise SystemExit("--mm-encoder-attn-dtype fp8 is required when scale paths are set")

Prevention

Always emit the FP8 trio together: attn dtype, then exactly one scale-path flag.
In config templates, make the scale-path fields conditional on dtype == 'fp8'.

When it happens

Trigger: Setting --mm-encoder-fp8-scale-path /path/scales.pt or --mm-encoder-fp8-scale-save-path while mm_encoder_attn_dtype is omitted (None) — note the field is typed Literal["fp8"] | None, so it must be explicitly set to "fp8".

Common situations: Calibrating FP8 for a multimodal encoder but forgetting the dtype flag; reordering CLI flags during a refactor and dropping --mm-encoder-attn-dtype fp8.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/5685a43b3a68fbc9. Report an issue: GitHub.

Appendix: source

Thrown at vllm/config/multimodal.py:323

        )
        return AttentionBackendEnum[value.upper()]

    @model_validator(mode="after")
    def _validate_multimodal_config(self):
        if self.mm_processor_cache_type != "shm" and (
            self.mm_shm_cache_max_object_size_mb
            != MultiModalConfig.mm_shm_cache_max_object_size_mb
        ):
            raise ValueError(
                "'mm_shm_cache_max_object_size_mb' should only be set when "
                "'mm_processor_cache_type' is 'shm'."
            )
        # Validate FP8 scale path combinations.
        if self.mm_encoder_attn_dtype != "fp8" and (
            self.mm_encoder_fp8_scale_path is not None
            or self.mm_encoder_fp8_scale_save_path is not None
        ):
            raise ValueError(
                "'mm_encoder_fp8_scale_path' and "
                "'mm_encoder_fp8_scale_save_path' require "
                "'mm_encoder_attn_dtype' to be 'fp8'."
            )
        if (
            self.mm_encoder_fp8_scale_path is not None
            and self.mm_encoder_fp8_scale_save_path is not None
        ):
            raise ValueError(
                "'mm_encoder_fp8_scale_save_path' cannot be used with "
                "'mm_encoder_fp8_scale_path' (saving requires dynamic scaling)."
            )

        # Validate file paths exist.
        if self.mm_encoder_fp8_scale_path is not None:
            scale_path = Path(self.mm_encoder_fp8_scale_path)
            if not scale_path.is_file():
                raise FileNotFoundError(f"FP8 scale file not found: {scale_path}")

View on GitHub (pinned to c794754062)