hiyouga/LlamaFactory · error · ValueError

chunk_size must be one of {SUPPORTED_CHUNK_SIZES}, got {chun

Error message

chunk_size must be one of {SUPPORTED_CHUNK_SIZES}, got {chunk_size!r}.

What it means

The FLA plugin validates `chunk_size` (used when the chunk_gated_delta_rule kernel is configured) against SUPPORTED_CHUNK_SIZES. It must be a real int (bools are explicitly rejected because bool is a subclass of int) and a member of the supported set; anything else raises a ValueError echoing the offending value.

Source

Thrown at src/llamafactory/v1/plugins/model_plugins/kernels/ops/linear_attention/fla.py:81

        config = kwargs.get("config") or {}
        include_kernels = config.get("include_kernels", "auto")
        chunk_size = config.get("chunk_size", 64)

        if include_kernels == "auto" or include_kernels is True:
            selected = list(FLASH_LINEAR_ATTENTION_KERNELS)
        elif isinstance(include_kernels, str):
            selected = [name.strip() for name in include_kernels.split(",") if name.strip()]
        else:
            raise TypeError("kernel_config.include_kernels must be 'auto' or a comma-separated string.")

        if not selected:
            raise ValueError("kernel_config.include_kernels must select at least one FLA kernel.")

        unsupported = set(selected).difference(FLASH_LINEAR_ATTENTION_KERNELS)
        if unsupported:
            raise ValueError(f"Unsupported Flash Linear Attention kernels: {sorted(unsupported)}")
        if isinstance(chunk_size, bool) or not isinstance(chunk_size, int) or chunk_size not in SUPPORTED_CHUNK_SIZES:
            raise ValueError(f"chunk_size must be one of {SUPPORTED_CHUNK_SIZES}, got {chunk_size!r}.")

        from fsdp_turbo.ops.registry import get_op
        from fsdp_turbo.utils.patch import patch_model_members

        patched = 0
        named_modules = tuple(model.named_modules())
        for op_name in selected:
            module_attribute = FLA_MODULE_ATTRIBUTES[op_name]
            op = get_op(op_name)
            configured_op = partial(op, chunk_size=chunk_size) if op_name == CHUNK_GATED_DELTA_RULE else op
            targets = {
                f"{type(module).__module__}.{type(module).__name__}.{module_attribute}"
                for _, module in named_modules
                if callable(getattr(module, module_attribute, None))
            }
            matched = patch_model_members(model, sorted(targets), configured_op) if targets else 0
            if matched == 0:
                raise RuntimeError(f"FLA operator `{op_name}` did not match any model module attributes.")

View on GitHub (pinned to f28afaf635)

Solutions

1. Read the allowed values from SUPPORTED_CHUNK_SIZES in your installed fla.py and use one of them (64 is the default)
2. Ensure the YAML value is a plain unquoted integer
3. If the value comes from templating, cast to int before building kernel_config

Example fix

# before
kernel_config:
  chunk_size: "64"

# after
kernel_config:
  chunk_size: 64

from llamafactory.v1.plugins.model_plugins.kernels.ops.linear_attention.fla import SUPPORTED_CHUNK_SIZES
assert isinstance(chunk_size, int) and not isinstance(chunk_size, bool) and chunk_size in SUPPORTED_CHUNK_SIZES, \
    f"chunk_size must be in {SUPPORTED_CHUNK_SIZES}"

def is_valid_chunk_size(v) -> bool:
    return isinstance(v, int) and not isinstance(v, bool) and v in SUPPORTED_CHUNK_SIZES

When it happens

Trigger: Setting chunk_size to a value outside SUPPORTED_CHUNK_SIZES, a float like 64.0, a string like "64", or a boolean. YAML `chunk_size: true` parses to bool True which is explicitly caught by the isinstance(chunk_size, bool) guard.

Common situations: YAML configs where the value is quoted ("64") or written as a float; users guessing chunk sizes not supported by the Triton kernels; config templating emitting strings.

Related errors

• kernel_config.include_kernels must select at least one FLA k
• Unsupported Flash Linear Attention kernels: {sorted(unsuppor
• Unknown backend: {model_args.infer_backend}
• Plugin configuration must have a 'name' field.
• kernel_config.include_kernels must be 'auto' or a comma-sepa

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/61d0bc52bcb01272. Report an issue: GitHub.