{"record":{"id":"36677aad2583f008","repo":"sgl-project/sglang","slug":"cannot-found-moe-block-size-for-shape-valid-shape","errorCode":null,"errorMessage":"cannot found moe_block_size for shape {valid_shape_m}","messagePattern":"cannot found moe_block_size for shape (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/moe/moe_runner/humming.py","lineNumber":511,"sourceCode":"            apply_routed_scaling_factor=runner_input.apply_routed_scaling_factor,\n        )\n\n        return HummingRunnerOutput(hidden_states=output)\n\n    def _prepare_indexed_gemm_kwargs(\n        self, topk_ids: torch.Tensor\n    ) -> tuple[dict[str, Any], dict[str, Any]]:\n        from sglang.srt.layers.moe.fused_moe_triton import moe_align_block_size\n\n        configs = self.get_humming_gemm_configs(HummingGemmType.INDEXED)\n        valid_shape_m = self.estimate_local_valid_shape_m(topk_ids)\n\n        for min_shape_m, max_shape_m, config in configs[\"w13_tuning_config\"]:\n            if valid_shape_m > min_shape_m and valid_shape_m <= max_shape_m:\n                moe_block_size = config[\"block_shape\"][0]\n                break\n        else:\n            raise ValueError(f\"cannot found moe_block_size for shape {valid_shape_m}\")\n\n        sorted_ids, expert_ids, num_tokens_padded = moe_align_block_size(\n            topk_ids=topk_ids,\n            block_size=moe_block_size,\n            num_experts=self.num_experts,\n            ignore_invalid_expert=True,\n        )\n\n        moe_common_kwargs = {\n            \"sorted_ids\": sorted_ids,\n            \"expert_ids\": expert_ids,\n            \"num_tokens_padded\": num_tokens_padded,\n            \"compute_config\": configs[\"compute_config_str\"],\n            \"valid_shape_m\": valid_shape_m,\n        }\n\n        top_k = topk_ids.size(1)\n        moe_kwargs1 = {","sourceCodeStart":493,"sourceCodeEnd":529,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/moe/moe_runner/humming.py#L493-L529","documentation":"_prepare_indexed_gemm_kwargs picks the moe_block_size by finding the first w13_tuning_config entry whose (min_shape_m, max_shape_m] range contains valid_shape_m. If valid_shape_m exceeds all configured ranges, the for/else raises because no block size applies.","triggerScenarios":"Running the indexed-GEMM humming path with an M (token count) larger than the largest max_shape_m in the tuned w13_tuning_config table — e.g. very large batch/prefill or many tokens per expert after alignment.","commonSituations":"Huge prefill batches or high concurrency exceeding the tuning table shipped with the humming backend; a stale tuning config after a batch-size increase or --max-running-requests raise.","solutions":["Reduce batch size / chunked prefill size so shape_m falls inside the tuned range","Update/extend the w13_tuning_config table with a range covering larger M","Add a default block-size fallback entry with max_shape_m = inf in the tuning config"],"exampleFix":"# tuning config addition (conceptual)\n# before\n{\"w13_tuning_config\": [{\"shape_range\": [1, 4096], \"block_shape\": [128, 128]}]}\n# after\n{\"w13_tuning_config\": [{\"shape_range\": [1, 4096], \"block_shape\": [128, 128]},\n                         {\"shape_range\": [4096, float(\"inf\")], \"block_shape\": [64, 128]}]}","handlingStrategy":"validation","validationCode":"max_m = max(cfg[1] for cfg in runner.tuning['w13_tuning_config'])\nassert estimated_shape_m <= max_m, f'shape_m {estimated_shape_m} exceeds tuned max {max_m}'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Cap --chunked-prefill-size and max running requests below the tuning table's max M","Extend tuning configs when raising batch limits"],"tags":["sglang","moe","humming","tuning-config","batch-size","index-out-of-range"],"backgroundTag":"value-outside-tuned-range","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}