{"record":{"id":"49b59457995ca050","repo":"sgl-project/sglang","slug":"layer-sharded-hicache-backup-does-not-support-layo","errorCode":null,"errorMessage":"Layer-sharded HiCache backup does not support layout: {self.layout}","messagePattern":"Layer-sharded HiCache backup does not support layout: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/mem_cache/pool_host/mla.py","lineNumber":389,"sourceCode":"                        dst_indices=host_indices,\n                        item_size=self.token_stride_size,\n                    )\n            elif self.layout == \"page_first\":\n                if self.can_use_jit:\n                    jit_transfer_hicache_one_layer_mla(\n                        cache_dst=self.data_refs[host_layer_id],\n                        cache_src=device_pool.kv_buffer[device_layer_id],\n                        indices_dst=host_indices,\n                        indices_src=device_indices,\n                        element_dim=self.kv_cache_dim,\n                    )\n                else:\n                    raise ValueError(\n                        \"Layer-sharded MLA HiCache backup with page_first layout \"\n                        \"requires the JIT one-layer kernel.\"\n                    )\n            else:\n                raise ValueError(\n                    f\"Layer-sharded HiCache backup does not support layout: {self.layout}\"\n                )\n        elif io_backend == \"direct\":\n            if self.layout == \"layer_first\":\n                transfer_kv_direct(\n                    src_layers=[device_pool.kv_buffer[device_layer_id]],\n                    dst_layers=[self.kv_buffer[host_layer_id]],\n                    src_indices=device_indices,\n                    dst_indices=host_indices,\n                    page_size=self.page_size,\n                )\n            else:\n                raise ValueError(\n                    \"Layer-sharded direct HiCache backup only supports \"\n                    f\"layer_first layout, got {self.layout}\"\n                )\n        else:\n            raise ValueError(","sourceCodeStart":371,"sourceCodeEnd":407,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/mem_cache/pool_host/mla.py#L371-L407","documentation":"MLAHiCachePool._backup_from_device_per_layer only knows how to write layer_first and page_first host layouts when the device KV pool is layer-sharded (e.g. with context parallelism). This error fires when the pool's layout is anything else (such as page_first_kv_split) and a per-layer D2H backup is attempted, because no code path exists for that layout combination.","triggerScenarios":"Running a model with a layer-sharded device pool (CP) plus a host HiCache layout other than layer_first/page_first, causing backup_from_device_all_layer to loop over owned layers and call _backup_from_device_per_layer, which falls through the layout if/elif chain.","commonSituations":"Using --hicache with page_first_kv_split layout (Ascend-style KV-split buffers) together with context parallelism, or a new layout added to the class without extending this branch.","solutions":["Switch the host HiCache layout to layer_first (or page_first with the JIT kernel enabled) so the per-layer path is supported","Disable layer sharding of the device pool (e.g. drop context parallelism / CP>1) so the all-layer path handles the layout","If you control the code, extend _backup_from_device_per_layer with a branch for the missing layout"],"exampleFix":"# before\nserver_args.hicache_layout = \"page_first_kv_split\"  # with CP layer sharding\n# after\nserver_args.hicache_layout = \"layer_first\"","handlingStrategy":"validation","validationCode":"SUPPORTED_PER_LAYER = {\"layer_first\", \"page_first\"}\nif pool._is_device_layer_sharded(device_pool) and pool.layout not in SUPPORTED_PER_LAYER:\n    raise ConfigError(f\"layout {pool.layout} unsupported for layer-sharded backup\")","typeGuard":"def supports_layer_sharded_backup(pool) -> bool:\n    return pool.layout in (\"layer_first\", \"page_first\")","tryCatchPattern":"try:\n    pool.backup_from_device_all_layer(...)\nexcept ValueError as e:\n    if \"Layer-sharded\" in str(e):\n        logger.error(\"layout/io-backend combo unsupported; falling back to disabled HiCache\")\n        disable_hicache()\n    else:\n        raise","preventionTips":["Validate (layout, io_backend, layer-sharding) combos at server start","Document the supported matrix next to backup_from_device_all_layer","Add startup-time rejection instead of runtime ValueError in the hot path"],"tags":["hicache","mla","layout","context-parallelism","sglang"],"backgroundTag":"unsupported-configuration-combination","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}