{"record":{"id":"47a1db0070b07590","repo":"hiyouga/LlamaFactory","slug":"vllm-engine-does-not-support-rope-scaling","errorCode":null,"errorMessage":"vLLM engine does not support RoPE scaling.","messagePattern":"vLLM engine does not support RoPE scaling\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/hparams/parser.py","lineNumber":684,"sourceCode":"    return model_args, data_args, training_args, finetuning_args, generating_args\n\n\ndef get_infer_args(args: dict[str, Any] | list[str] | None = None) -> _INFER_CLS:\n    model_args, data_args, finetuning_args, generating_args = _parse_infer_args(args)\n\n    # Setup logging\n    _set_transformers_logging()\n\n    # Check arguments\n    if model_args.infer_backend == \"vllm\":\n        if finetuning_args.stage != \"sft\":\n            raise ValueError(\"vLLM engine only supports auto-regressive models.\")\n\n        if model_args.quantization_bit is not None:\n            raise ValueError(\"vLLM engine does not support bnb quantization (GPTQ and AWQ are supported).\")\n\n        if model_args.rope_scaling is not None:\n            raise ValueError(\"vLLM engine does not support RoPE scaling.\")\n\n        if model_args.adapter_name_or_path is not None and len(model_args.adapter_name_or_path) != 1:\n            raise ValueError(\"vLLM only accepts a single adapter. Merge them first.\")\n\n    _set_env_vars()\n    _verify_model_args(model_args, data_args, finetuning_args)\n    _check_extra_dependencies(model_args, finetuning_args)\n\n    # Post-process model arguments\n    if model_args.export_dir is not None and model_args.export_device == \"cpu\":\n        model_args.device_map = {\"\": torch.device(\"cpu\")}\n        if data_args.cutoff_len != DataArguments().cutoff_len:  # override cutoff_len if it is not default\n            model_args.model_max_length = data_args.cutoff_len\n    else:\n        model_args.device_map = \"auto\"\n\n    model_args.configure_kt_loading(finetuning_args, data_args.cutoff_len)\n","sourceCodeStart":666,"sourceCodeEnd":702,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/hparams/parser.py#L666-L702","documentation":"Raised in get_infer_args when infer_backend is vllm and model_args.rope_scaling is not None. The vLLM engine in this codebase does not thread transformers-style rope_scaling dictionaries into its engine options, so the setting is rejected up front rather than silently ignored.","triggerScenarios":"A chat config with infer_backend: vllm plus a rope_scaling: {...} block in the model section (common for long-context setups copied from HF configs).","commonSituations":"Copying the rope_scaling block from a training config or from a model card's long-context recipe into a vLLM chat config.","solutions":["Remove rope_scaling from the model args when using infer_backend: vllm.","Configure RoPE on the vLLM side instead via its own engine kwargs (e.g. rope_theta / long-context settings passed through vllm_extra_config if supported).","Use a model checkpoint whose rope scaling is already baked into its config, or switch to infer_backend: hf."],"exampleFix":"# before\ninfer_backend: vllm\nrope_scaling:\n  rope_type: linear\n  factor: 4.0\n\n# after\ninfer_backend: vllm\n# (omit rope_scaling; rely on model's native config)","handlingStrategy":"validation","validationCode":"if cfg[\"model_args\"].get(\"infer_backend\") == \"vllm\":\n    assert not cfg[\"model_args\"].get(\"rope_scaling\"), \\\n        \"rope_scaling is unsupported on the vLLM backend; remove it or use infer_backend: hf\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Handle long context under vLLM through vLLM-native options or an already-scaled checkpoint.","Grep your serving YAMLs for rope_scaling in CI config checks."],"tags":["vllm","rope-scaling","long-context","infer-config"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}