{"record":{"id":"b2fd9c0d7eb5572e","repo":"vllm-project/vllm","slug":"to-load-a-model-from-object-storage-s3-gcs-azure","errorCode":null,"errorMessage":"To load a model from object storage (S3/GCS/Azure), 'load_format' must be 'modelexpress', 'runai_streamer' or 'runai_streamer_sharded', but got '{self.load_config.load_format}'. Model: {self.model_config.model}","messagePattern":"To load a model from object storage \\(S3/GCS/Azure\\), 'load_format' must be 'modelexpress', 'runai_streamer' or 'runai_streamer_sharded', but got '(.+?)'\\. Model: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/vllm.py","lineNumber":2218,"sourceCode":"            from vllm.model_executor.models.adapters import SequenceClassificationConfig\n\n            SequenceClassificationConfig.verify_and_update_config(self)\n\n        if hasattr(self.model_config, \"model_weights\") and is_runai_obj_uri(\n            self.model_config.model_weights\n        ):\n            if self.load_config.load_format == \"auto\":\n                logger.info(\n                    \"Detected Run:ai model config. \"\n                    \"Overriding `load_format` to 'runai_streamer'\"\n                )\n                self.load_config.load_format = \"runai_streamer\"\n            elif self.load_config.load_format not in (\n                \"modelexpress\",\n                \"runai_streamer\",\n                \"runai_streamer_sharded\",\n            ):\n                raise ValueError(\n                    f\"To load a model from object storage (S3/GCS/Azure), \"\n                    f\"'load_format' must be 'modelexpress', 'runai_streamer' or \"\n                    f\"'runai_streamer_sharded', \"\n                    f\"but got '{self.load_config.load_format}'. \"\n                    f\"Model: {self.model_config.model}\"\n                )\n\n    def compile_debug_dump_path(self) -> Path | None:\n        \"\"\"Returns a rank-aware path for dumping\n        torch.compile debug information.\n        \"\"\"\n        if self.compilation_config.debug_dump_path is None:\n            return None\n        tp_rank = self.parallel_config.rank\n        dp_rank = self.parallel_config.data_parallel_index\n        append_path = f\"rank_{tp_rank}_dp_{dp_rank}\"\n        path = self.compilation_config.debug_dump_path / append_path\n        return path","sourceCodeStart":2200,"sourceCodeEnd":2236,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/vllm.py#L2200-L2236","documentation":"When the model path points at object storage (S3/GCS/Azure), vLLM can only stream the weights via one of the dedicated loaders: 'modelexpress', 'runai_streamer', or 'runai_streamer_sharded'. If the detected object-storage model is paired with any other explicit load_format, validation raises. Note: with load_format='auto' vLLM auto-selects 'runai_streamer'.","triggerScenarios":"Setting `--model s3://bucket/...` (or gs://, az://) together with `--load-format` set to a non-streaming format like 'safetensors', 'pt', or 'dummy'","commonSituations":"Moving from local checkpoints to an S3-hosted model while keeping an existing `--load-format safetensors` in the launch config; using a custom loader that cannot read from cloud storage.","solutions":["Remove the explicit `--load-format` so 'auto' detection picks runai_streamer for object storage.","Or set `--load-format runai_streamer` (or 'runai_streamer_sharded' / 'modelexpress') explicitly.","Or download the model locally and point --model at the local path."],"exampleFix":"# before\nvllm serve s3://my-bucket/my-model --load-format safetensors\n\n# after\nvllm serve s3://my-bucket/my-model --load-format runai_streamer","handlingStrategy":"validation","validationCode":"OBJECT_PREFIXES = (\"s3://\", \"gs://\", \"az://\")\nSTREAMING_FORMATS = {\"auto\", \"modelexpress\", \"runai_streamer\", \"runai_streamer_sharded\"}\nif model.startswith(OBJECT_PREFIXES) and load_format not in STREAMING_FORMATS:\n    load_format = \"runai_streamer\"","typeGuard":null,"tryCatchPattern":"try:\n    LLM(model=s3_path, load_format=lf, ...)\nexcept ValueError as e:\n    if \"load_format\" in str(e) and \"object storage\" in str(e):\n        LLM(model=s3_path, load_format=\"runai_streamer\", ...)\n    else:\n        raise","preventionTips":["Use load_format='auto' and let vLLM detect Run:ai config","Gate --load-format on the model path scheme"],"tags":["model-loading","object-storage","s3","load-format"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}