hiyouga/LlamaFactory · error · ValueError

Unknown sample backend: {args.sample_backend}

Error message

Unknown sample backend: {args.sample_backend}

What it means

`BaseSampler.__init__` in the v1 core only wires up `SampleBackend.HF` (the HuggingFace generation engine); any other value falls into the `else` branch and raises `Unknown sample backend`. It is an exhaustive-match guard: until more engines (vllm/sglang-style) are registered for v1 sampling, `hf` is the only accepted backend.

Source

Thrown at src/llamafactory/v1/core/base_sampler.py:43

    Args:
        args: Sample arguments.
        model_args: Model arguments.
        model: Model.
        renderer: Renderer.
    """

    def __init__(
        self,
        args: SampleArguments,
        model_args: ModelArguments,
        model: HFModel,
        renderer: Renderer,
    ) -> None:
        if args.sample_backend == SampleBackend.HF:
            self.engine = HuggingFaceEngine(args, model_args, model, renderer)
        else:
            raise ValueError(f"Unknown sample backend: {args.sample_backend}")

    async def generate(self, messages: list[Message], tools: str | None = None) -> AsyncGenerator[str, None]:
        """Generate tokens asynchronously.

        Args:
            messages: List of messages.
            tools: Tools string.

        Yields:
            Generated tokens.
        """
        async for token in self.engine.generate(messages, tools):
            yield token

    async def batch_infer(self, dataset: TorchDataset) -> list[Sample]:
        """Batch infer samples.

        Args:

View on GitHub (pinned to f28afaf635)

Solutions

1. Set `sample_backend: hf` in the v1 sample args
2. If you need vllm/sglang inference, use the v0 chat path (`USE_V1` unset) until v1 registers those engines
3. Check the `SampleBackend` enum for currently available members before choosing

Example fix

# before (yaml)
sample:
  sample_backend: vllm

# after (yaml)
sample:
  sample_backend: hf

from llamafactory.v1.core.base_sampler import SampleBackend  # adjust import path

if args.sample_backend not in {SampleBackend.HF}:
    raise SystemExit(f"v1 sampling currently supports only 'hf', got {args.sample_backend}")

def is_supported_sample_backend(value: object) -> bool:
    return value == SampleBackend.HF

When it happens

Trigger: Constructing a v1 `BaseSampler` (e.g. in RLHF/online sampling flows or `sample` entrypoint) with `sample_backend` set to anything other than `hf`, such as `vllm` or a typo.

Common situations: Porting a v0 chat/sampling config that used `vllm` or `sglang` engines into v1; assuming v1 supports the same backend list as v0's chat engines.

Related errors

• Unknown backend: {model_args.infer_backend}
• world_size ({helper.get_world_size()}) must be divisible by
• mp_replicate_size * mp_shard_size must equal to world_size,
• world_size ({helper.get_world_size()}) must be divisible by
• dp_size * cp_size must equal to world_size, got {self.dp_siz

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/cebe145688540005. Report an issue: GitHub.