sgl-project/sglang · critical · ValueError

num nextn_predict_layers is not in the config

Error message

num nextn_predict_layers is not in the config

What it means

Shared helper resolve_nextn_layer_id requires config.num_nextn_predict_layers to exist (and equal 1) to locate the MTP/nextn layer index in HF checkpoint naming. It is called from load_weights of bailing_moe_v3.

Source

Thrown at python/sglang/srt/models/bailing_moe_v3.py:249

        return layer_group_size[layer_idx] == 1
    if layer_group_size > 0:
        return (layer_idx + 1) % layer_group_size != 0
    else:
        return False


_NEXTN_SPEC_WEIGHT_NAMES = (
    "final_layernorm",
    "eh_proj",
    "enorm",
    "hnorm",
)


def resolve_nextn_layer_id(config: PretrainedConfig) -> int:
    """Locate the nextn predict layer index in the HF checkpoint name space."""
    if not hasattr(config, "num_nextn_predict_layers"):
        raise ValueError("num nextn_predict_layers is not in the config")
    assert config.num_nextn_predict_layers == 1, "Only 1 nextn layer is supported"
    return 0 if config.num_hidden_layers == 1 else config.num_hidden_layers


def rewrite_nextn_weight_name(name: str, nextn_layer_prefix: str) -> Optional[str]:
    """Map a HF nextn-layer weight name onto the local NextN module namespace.

    The caller must already have ensured ``name.startswith(nextn_layer_prefix)``.
    Returns the rewritten name, or None to signal the weight should be skipped
    (e.g. shared head / embed tokens which are reused from the target model).
    """
    if "shared_head.head" in name or "embed_tokens" in name:
        return None
    for spec in _NEXTN_SPEC_WEIGHT_NAMES:
        if spec in name:
            return name.replace(nextn_layer_prefix, "model")
    return name.replace(nextn_layer_prefix, "model.decoder")

View on GitHub (pinned to 0132848349)

Solutions

1. Add "num_nextn_predict_layers": 1 to config.json
2. Disable MTP/speculative decoding so the nextn path is never hit

Example fix

// before
resolve_nextn_layer_id(cfg)  # cfg has no field
// after
cfg.num_nextn_predict_layers = 1
resolve_nextn_layer_id(cfg)

assert getattr(config, "num_nextn_predict_layers", 0) == 1

When it happens

Trigger: Loading a Bailing v3 MoE checkpoint that contains nextn weights when the config lacks num_nextn_predict_layers.

Common situations: Speculative decoding setup with an older or edited config.json.

Related errors

• num nextn_predict_layers is not in the config
• num_nextn_predict_layers is not in the config
• online c128 does not support MTP
• Eagle3 MLA draft post_load_weights only supports float dtype
• Qwen3-Next MTP shared expert fusion currently supports exact

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/9355109b2784978c. Report an issue: GitHub.