hiyouga/LlamaFactory · error · NotImplementedError

Unknown backend: {model_args.infer_backend}

Error message

Unknown backend: {model_args.infer_backend}

What it means

NotImplementedError raised by ChatModel.__init__ when model_args.infer_backend matches none of the supported engines (huggingface, vllm, sglang). Typically the result of a typo or a config value from a different version's enum.

Source

Thrown at src/llamafactory/chat/chat_model.py:75

                self.engine: BaseEngine = VllmEngine(model_args, data_args, finetuning_args, generating_args)
            except ImportError as e:
                raise ImportError(
                    "vLLM not install, you may need to run `pip install vllm`\n"
                    "or try to use HuggingFace backend: --infer_backend huggingface"
                ) from e
        elif model_args.infer_backend == EngineName.SGLANG:
            try:
                from .sglang_engine import SGLangEngine

                self.engine: BaseEngine = SGLangEngine(model_args, data_args, finetuning_args, generating_args)
            except ImportError as e:
                raise ImportError(
                    "SGLang not install, you may need to run `pip install sglang[all]`\n"
                    "or try to use HuggingFace backend: --infer_backend huggingface"
                ) from e
        else:
            raise NotImplementedError(f"Unknown backend: {model_args.infer_backend}")

        self._loop = asyncio.new_event_loop()
        self._thread = Thread(target=_start_background_loop, args=(self._loop,), daemon=True)
        self._thread.start()

    def chat(
        self,
        messages: list[dict[str, str]],
        system: Optional[str] = None,
        tools: Optional[str] = None,
        images: Optional[list["ImageInput"]] = None,
        videos: Optional[list["VideoInput"]] = None,
        audios: Optional[list["AudioInput"]] = None,
        **input_kwargs,
    ) -> list["Response"]:
        r"""Get a list of responses of the chat model."""
        task = asyncio.run_coroutine_threadsafe(
            self.achat(messages, system, tools, images, videos, audios, **input_kwargs), self._loop

View on GitHub (pinned to f28afaf635)

Solutions

1. Set infer_backend to one of: 'huggingface', 'vllm', 'sglang' (exact lowercase as defined in EngineName).
2. Check src/llamafactory/extras/constants.py EngineName for the accepted values in your installed version.
3. Trim/normalize config strings (strip whitespace, lowercase) before building ChatModel.

Example fix

# before
ChatModel({..., 'infer_backend': 'VLLM'})
# after
ChatModel({..., 'infer_backend': 'vllm'})

VALID_BACKENDS = {"huggingface", "vllm", "sglang"}
assert args["infer_backend"] in VALID_BACKENDS, f"pick from {VALID_BACKENDS}"

const VALID = new Set(['huggingface', 'vllm', 'sglang']);
const isValidBackend = (b) => VALID.has(b);

try { ChatModel(cfg) } catch (e) { if (e instanceof NotImplementedError && e.message.startsWith('Unknown backend')) { cfg.infer_backend = 'huggingface'; return ChatModel(cfg); } throw e; }

When it happens

Trigger: infer_backend set to 'vllm ' (trailing space), 'VLLM' (wrong case — note the enum comparison is exact), 'tensorrt', or any string not in EngineName; YAML configs copied from newer/older docs with backend names this version does not know.

Common situations: Hand-edited YAML typos; enum value drift across LlamaFactory versions; programmatic configs injecting unvalidated strings.

Related errors

• Plugin configuration must have a 'name' field.
• Unknown sample backend: {args.sample_backend}
• kernel_config.include_kernels must select at least one FLA k
• Unsupported Flash Linear Attention kernels: {sorted(unsuppor
• chunk_size must be one of {SUPPORTED_CHUNK_SIZES}, got {chun

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/67a2b6a9cbe30113. Report an issue: GitHub.