{"record":{"id":"3a2209c213c17f11","repo":"sgl-project/sglang","slug":"num-nextn-predict-layers-is-not-in-the-config","errorCode":null,"errorMessage":"num_nextn_predict_layers is not in the config","messagePattern":"num_nextn_predict_layers is not in the config","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"python/sglang/srt/models/bailing_moe.py","lineNumber":887,"sourceCode":"            return self.logits_processor(\n                input_ids, hidden_states, self.lm_head, forward_batch, aux_hidden_states\n            )\n        else:\n            return hidden_states\n\n    def load_weights(self, weights: Iterable[Tuple[str, torch.Tensor]], is_nextn=False):\n        if is_nextn:\n            if hasattr(self.config, \"num_nextn_predict_layers\"):\n                num_nextn_layers = self.config.num_nextn_predict_layers\n                assert num_nextn_layers == 1, \"Only 1 nextn layer is supported\"\n                # compatible with old design\n                nextn_layer_id = (\n                    0\n                    if self.config.num_hidden_layers == 1\n                    else self.config.num_hidden_layers\n                )\n            else:\n                raise ValueError(\"num_nextn_predict_layers is not in the config\")\n\n        stacked_params_mapping = [\n            # (param_name, shard_name, shard_id)\n            (\"gate_up_proj\", \"gate_proj\", 0),\n            (\"gate_up_proj\", \"up_proj\", 1),\n        ]\n\n        if is_nextn:\n            nextn_layer_prefix = f\"model.layers.{nextn_layer_id}\"\n            nextn_spec_weight_names = [\n                \"final_layernorm\",\n                \"eh_proj\",\n                \"enorm\",\n                \"hnorm\",\n            ]\n        # Params for weights, fp8 weight scales, fp8 activation scales\n        # (param_name, weight_name, expert_id, shard_id)\n        expert_params_mapping = FusedMoE.make_expert_params_mapping(","sourceCodeStart":869,"sourceCodeEnd":905,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/bailing_moe.py#L869-L905","documentation":"When loading weights, bailing_moe expects speculative-MTP (nextn) layer info: either config already contains the layer indices or it must define num_nextn_predict_layers so the nextn layer id can be derived. If neither is present, load_weights raises.","triggerScenarios":"Loading a Bailing MoE checkpoint in MTP/speculative mode (or a checkpoint containing nextn weights) when config.json lacks num_nextn_predict_layers.","commonSituations":"Speculative decoding configs assembled by hand; newer checkpoints that store the field under a different name.","solutions":["Add \"num_nextn_predict_layers\": 1 to config.json (only 1 nextn layer is supported)","Or ensure the config carries the explicit nextn layer index fields the loader checks before this branch","Disable speculative/MTP decoding if the checkpoint has no nextn weights"],"exampleFix":"// before\n{ \"num_hidden_layers\": 60 }\n// after\n{ \"num_hidden_layers\": 60, \"num_nextn_predict_layers\": 1 }","handlingStrategy":"validation","validationCode":"assert hasattr(config, \"num_nextn_predict_layers\"), \"add num_nextn_predict_layers=1 for MTP\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["When enabling MTP, always add num_nextn_predict_layers to config.json"],"tags":["mtp","speculative-decoding","bailing-moe","weight-loading"],"backgroundTag":"missing-config-field","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}