{"record":{"id":"3cf8bdcbc3a41c33","repo":"sgl-project/sglang","slug":"unsupported-mlp-layer-types-lid-mlp-type-exp","errorCode":null,"errorMessage":"Unsupported mlp_layer_types[{lid}]={mlp_type}; expected 'sparse' or 'dense'","messagePattern":"Unsupported mlp_layer_types\\[(.+?)\\]=(.+?); expected 'sparse' or 'dense'","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"python/sglang/srt/models/mellum.py","lineNumber":415,"sourceCode":"\n        self.attn_tp_size = get_parallel().attn_tp_size\n        self.attn_tp_rank = get_parallel().attn_tp_rank\n\n        mlp_layer_types = cfg.mlp_layer_types\n        num_experts = cfg.num_experts\n\n        if len(mlp_layer_types) != cfg.num_hidden_layers:\n            raise ValueError(\n                \"Expected len(mlp_layer_types) == num_hidden_layers, got \"\n                f\"{len(mlp_layer_types)} and {cfg.num_hidden_layers}\"\n            )\n\n        def _is_sparse(lid: int) -> bool:\n            if lid < 0 or lid >= cfg.num_hidden_layers:\n                return False\n            mlp_type = mlp_layer_types[lid]\n            if mlp_type not in (\"sparse\", \"dense\"):\n                raise ValueError(\n                    f\"Unsupported mlp_layer_types[{lid}]={mlp_type}; \"\n                    \"expected 'sparse' or 'dense'\"\n                )\n            return mlp_type == \"sparse\"\n\n        self.is_layer_sparse = _is_sparse(layer_id)\n\n        if self.is_layer_sparse:\n            if num_experts <= 0:\n                raise ValueError(\n                    \"Sparse MLP requested but num_experts <= 0 in Mellum config\"\n                )\n            self.mlp = Qwen3MoeSparseMoeBlock(\n                layer_id=layer_id,\n                config=cfg,\n                quant_config=quant_config,\n                prefix=add_prefix(\"mlp\", prefix),\n            )","sourceCodeStart":397,"sourceCodeEnd":433,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/mellum.py#L397-L433","documentation":"Each entry of Mellum's config.mlp_layer_types must be exactly 'sparse' or 'dense'; anything else (typos, nulls, other strings) is rejected when the layer's sparsity is resolved during __init__. The error names the offending layer index and value to make the config bug easy to locate.","triggerScenarios":"mlp_layer_types contains a value like 'moe', 'Sparse', '', or None at index lid; case-mismatched or whitespace-padded strings from manual config edits.","commonSituations":"Hand-edited config.json during conversion; configs authored from memory instead of copying the original; locale/case differences in generated configs.","solutions":["Inspect config.json mlp_layer_types[lid] and normalize every entry to exactly 'sparse' or 'dense'","Check for typos, casing, or trailing whitespace in the list","Validate the list programmatically before loading the model (see validation snippet)"],"exampleFix":"// before\n\"mlp_layer_types\": [\"Dense\", \"moe\"]\n// after\n\"mlp_layer_types\": [\"dense\", \"sparse\"]","handlingStrategy":"validation","validationCode":"cfg = AutoConfig.from_pretrained(path)\nassert all(v in (\"sparse\", \"dense\") for v in cfg.mlp_layer_types), [\n    i for i, v in enumerate(cfg.mlp_layer_types) if v not in (\"sparse\", \"dense\")]","typeGuard":"def mlp_types_valid(cfg) -> bool:\n    return all(v in (\"sparse\", \"dense\") for v in getattr(cfg, \"mlp_layer_types\", []))","tryCatchPattern":null,"preventionTips":["Generate mlp_layer_types from a template, never type entries by hand","Run a config schema check in your serving startup script"],"tags":["mellum","config-validation","moe"],"backgroundTag":"invalid-enum-value","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}