sgl-project/sglang · error · ValueError

Unable to find matching target for {layer_name} in the compr

Error message

Unable to find matching target for {layer_name} in the compressed-tensors config.

What it means

find_matched_target failed to match the layer name (or its class name, or fused-projection mapping) against any target in the compressed-tensors config. Every layer of a quantized model must match a config target; a miss means the config doesn't describe this layer.

Source

Thrown at python/sglang/srt/layers/quantization/compressed_tensors/utils.py:144

    :param layer_name: layer name
    :param module: torch.nn.Module
    :param targets: list of targets to match the layer against
    :param fused_mapping: map from fused layer names to its components
    :param fused_strategy: either "all" or "any". If using "all", fused
        layers match if "all" of its components match
    """

    if layer_name is None:
        layer_name = ""

    matched_target = (
        _find_first_match(layer_name, targets)
        or _find_first_match(module.__class__.__name__, targets, True)
        or _match_fused_layer(layer_name, targets, fused_mapping)
    )

    if matched_target is None:
        raise ValueError(
            f"Unable to find matching target for {layer_name} in the "
            "compressed-tensors config."
        )

    return matched_target


def _find_first_match(
    value: str, targets: Iterable[str], check_contains: bool = False
) -> Optional[str]:
    """
    Returns first element of target that matches value either
    exactly or as a regex after 're:'. If check_contains is set to True,
    additionally checks if the target string is contained within the value.

    :param value: string to compare the list of targets against
    :param targets: list of targets to match the layer against
    :param check_contains: whether or not to do a substring match

View on GitHub (pinned to 0132848349)

Solutions

1. Ensure quantization_config targets cover all linear layers, e.g. add the missing module name pattern or use a wildcard
2. Re-quantize from the exact model revision being served
3. Check for renamed/fused modules and match the naming the quantizer used

Example fix

// before
"targets": ["model.layers.0.self_attn.q_proj", "model.layers.1.self_attn.q_proj"]
// after
"targets": ["model.*.self_attn.*_proj", "model.*.mlp.*_proj"]

import re
targets = cfg["quantization_config"]["targets"]
layer = "model.layers.0.mlp.down_proj"
assert any(re.fullmatch(t.replace("*", ".*"), layer) for t in targets), "layer not covered"

def layer_covered(layer, targets):
    import fnmatch
    return any(fnmatch.fnmatch(layer, t) for t in targets)

When it happens

Trigger: A layer name in the model (e.g. 'model.layers.5.mlp.gate_up_proj' or a renamed module) not covered by quantization_config.targets, and not resolvable via module class name or the fused-layer mapping, when get_scheme_dict/get_linear_scheme builds the scheme.

Common situations: Architectural changes (new module names) not reflected in the quant config; checkpoints quantized against a different model revision; typos in target patterns; custom models with non-standard layer names.

Related errors

• Found different quantization schemes for {shard_proj_names}
• Quantization method specified in the model config ({quant_me
• KV cache dtype mismatch: prefill server has kv_cache_dtype={
• The Triton WNA16 MoE backend only supports symmetric INT4 gr
• Unsupported FusedMoe scheme: {weight_quant}, {input_quant}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/e85c575aa63a7ddf. Report an issue: GitHub.