vllm-project/vllm · error · ValueError

This model does not support `--runner generate`.

Error message

This model does not support `--runner generate`.

What it means

Error "This model does not support `--runner generate`." thrown in vllm-project/vllm.

Source

Thrown at vllm/config/model.py:656

        )
        self.convert_type = self._get_convert_type(
            architectures, self.runner_type, self.convert
        )

        if (
            is_pooling_model
            and not is_generative_model
            and self.runner_type in ("draft", "generate")
        ):
            raise ValueError(
                f"Embedding models do not support `--runner {self.runner_type}`. "
                "Use `--runner pooling` or `--runner auto` for embedding models."
            )
        if self.runner_type == "generate" and not is_generative_model:
            generate_converts = _RUNNER_CONVERTS["generate"]
            if self.convert_type not in generate_converts:
                # Currently we don't have any converters for generative models
                raise ValueError("This model does not support `--runner generate`.")
        if self.runner_type == "pooling" and not is_pooling_model:
            pooling_converts = _RUNNER_CONVERTS["pooling"]
            if self.convert_type not in pooling_converts:
                convert_option = "<" + "|".join(pooling_converts) + ">"
                raise ValueError(
                    "This model does not support `--runner pooling`. "
                    f"You can pass `--convert {convert_option} to adapt "
                    "it into a pooling model."
                )

        # Note: Initialize these attributes early because transformers fallback
        # may fail to load dynamic modules in child processes
        model_info, arch = registry.inspect_model_cls(architectures, self)
        self._model_info = model_info
        self._architecture = arch
        logger.info("Resolved architecture: %s", arch)

        # Set default tokenizer modes based on model architecture

View on GitHub (pinned to c794754062)

Solutions

1. Use --runner pooling or --runner auto instead of --runner generate for this model.

When it happens

Trigger: Raised at vllm/config/model.py:656 when validation fails: This model does not support `--runner generate`.. Typically triggered by an incompatible or incomplete vLLM configuration, an unsupported platform/backend combination, or a runtime resource/dependency that is missing.

Common situations: Commonly encountered at vllm/config/model.py:656 during vLLM startup/config validation or runtime setup when: (1) conflicting CLI flags or config fields are combined, (2) the current platform (CUDA/ROCm/CPU/XPU) or installed optional packages do not support the requested feature, or (3) a required value is absent or out of range. Resolve by correcting the configuration as described in the message, or by selecting a supported alternative.

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/7ba1ccf6b1be8793. Report an issue: GitHub.