vllm-project/vllm · error · ValueError

To load a model from object storage (S3/GCS/Azure), 'load_fo

Error message

To load a model from object storage (S3/GCS/Azure), 'load_format' must be 'modelexpress', 'runai_streamer' or 'runai_streamer_sharded', but got '{self.load_config.load_format}'. Model: {self.model_config.model}

What it means

When the model path points at object storage (S3/GCS/Azure), vLLM can only stream the weights via one of the dedicated loaders: 'modelexpress', 'runai_streamer', or 'runai_streamer_sharded'. If the detected object-storage model is paired with any other explicit load_format, validation raises. Note: with load_format='auto' vLLM auto-selects 'runai_streamer'.

Source

Thrown at vllm/config/vllm.py:2218

            from vllm.model_executor.models.adapters import SequenceClassificationConfig

            SequenceClassificationConfig.verify_and_update_config(self)

        if hasattr(self.model_config, "model_weights") and is_runai_obj_uri(
            self.model_config.model_weights
        ):
            if self.load_config.load_format == "auto":
                logger.info(
                    "Detected Run:ai model config. "
                    "Overriding `load_format` to 'runai_streamer'"
                )
                self.load_config.load_format = "runai_streamer"
            elif self.load_config.load_format not in (
                "modelexpress",
                "runai_streamer",
                "runai_streamer_sharded",
            ):
                raise ValueError(
                    f"To load a model from object storage (S3/GCS/Azure), "
                    f"'load_format' must be 'modelexpress', 'runai_streamer' or "
                    f"'runai_streamer_sharded', "
                    f"but got '{self.load_config.load_format}'. "
                    f"Model: {self.model_config.model}"
                )

    def compile_debug_dump_path(self) -> Path | None:
        """Returns a rank-aware path for dumping
        torch.compile debug information.
        """
        if self.compilation_config.debug_dump_path is None:
            return None
        tp_rank = self.parallel_config.rank
        dp_rank = self.parallel_config.data_parallel_index
        append_path = f"rank_{tp_rank}_dp_{dp_rank}"
        path = self.compilation_config.debug_dump_path / append_path
        return path

View on GitHub (pinned to c794754062)

Solutions

1. Remove the explicit `--load-format` so 'auto' detection picks runai_streamer for object storage.
2. Or set `--load-format runai_streamer` (or 'runai_streamer_sharded' / 'modelexpress') explicitly.
3. Or download the model locally and point --model at the local path.

Example fix

# before
vllm serve s3://my-bucket/my-model --load-format safetensors

# after
vllm serve s3://my-bucket/my-model --load-format runai_streamer

OBJECT_PREFIXES = ("s3://", "gs://", "az://")
STREAMING_FORMATS = {"auto", "modelexpress", "runai_streamer", "runai_streamer_sharded"}
if model.startswith(OBJECT_PREFIXES) and load_format not in STREAMING_FORMATS:
    load_format = "runai_streamer"

try:
    LLM(model=s3_path, load_format=lf, ...)
except ValueError as e:
    if "load_format" in str(e) and "object storage" in str(e):
        LLM(model=s3_path, load_format="runai_streamer", ...)
    else:
        raise

When it happens

Trigger: Setting `--model s3://bucket/...` (or gs://, az://) together with `--load-format` set to a non-streaming format like 'safetensors', 'pt', or 'dummy'

Common situations: Moving from local checkpoints to an S3-hosted model while keeping an existing `--load-format safetensors` in the launch config; using a custom loader that cannot read from cloud storage.

Related errors

• tokenizer error: {0}

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/b2fd9c0d7eb5572e. Report an issue: GitHub.