{"record":{"id":"6a159203fce906a9","repo":"sgl-project/sglang","slug":"gguf-bf16-payload-does-not-have-a-byte-pair-layout","errorCode":null,"errorMessage":"GGUF BF16 payload does not have a byte-pair layout","messagePattern":"GGUF BF16 payload does not have a byte-pair layout","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/model_loader/expert_pack_loader.py","lineNumber":52,"sourceCode":"    KIMI_K3_MODEL_TYPE,\n    validate_expert_pack_model_config,\n)\nfrom sglang.srt.model_loader.kimi_k3_gguf import kimi_k3_nonexpert_weights_iterator\nfrom sglang.srt.model_loader.loader import (\n    BaseModelLoader,\n    _initialize_model,\n    device_loading_context,\n)\nfrom sglang.srt.model_loader.utils import set_default_torch_dtype\nfrom sglang.srt.runtime_context import get_exec, get_parallel\n\nlogger = logging.getLogger(__name__)\n\n\ndef _bf16_tensor(data: np.ndarray) -> torch.Tensor:\n    raw = np.asarray(data)\n    if raw.dtype != np.uint8 or raw.shape[-1] % 2:\n        raise ValueError(\"GGUF BF16 payload does not have a byte-pair layout\")\n    values = raw.view(np.uint16).reshape(*raw.shape[:-1], raw.shape[-1] // 2)\n    return torch.from_numpy(values.copy()).view(torch.bfloat16)\n\n\ndef _compressor_component(source_name: str) -> str | None:\n    if \"_compressor_kv.weight\" in source_name:\n        return \"kv\"\n    if \"_compressor_gate.weight\" in source_name:\n        return \"gate\"\n    return None\n\n\ndef _fused_compressor_name(checkpoint_name: str) -> str:\n    result = checkpoint_name.replace(\".wkv.weight\", \".wkv_gate.weight\")\n    result = result.replace(\".wgate.weight\", \".wkv_gate.weight\")\n    if result == checkpoint_name:\n        raise ValueError(f\"invalid compressor checkpoint name: {checkpoint_name}\")\n    return result","sourceCodeStart":34,"sourceCodeEnd":70,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/model_loader/expert_pack_loader.py#L34-L70","documentation":"_bf16_tensor reinterprets a GGUF tensor's raw bytes as BF16 pairs; it requires the numpy array to be uint8 with an even last dimension (bytes). If the payload dtype is not uint8 or its last axis has odd length, bytes cannot form 2-byte BF16 values and ValueError is raised — the buffer isn't a valid BF16 byte-pair layout.","triggerScenarios":"deepseek4_nonexpert_weights_iterator hitting a BF16 GGUF tensor whose numpy payload arrives as a non-uint8 dtype (e.g. pre-decoded float32/uint16) or with an odd trailing byte count (truncated/misaligned data).","commonSituations":"Corrupted or truncated GGUF file, a gguf reader version that returns typed arrays instead of raw bytes, or an endianness/packing mismatch in a custom GGUF conversion.","solutions":["Verify the GGUF file integrity (llama.cpp gguf-dump / re-download the file)","Ensure tensor data is read as raw bytes (uint8) — update gguf/sglang to compatible versions","Re-convert the model to GGUF with an official, up-to-date convert script","Check that tensor byte sizes are even and match expected element counts for BF16"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"raw = np.asarray(data)\nassert raw.dtype == np.uint8 and raw.shape[-1] % 2 == 0, 'BF16 payload must be even-length uint8 bytes'","typeGuard":"def is_bf16_byte_payload(raw: np.ndarray) -> bool:\n    return raw.dtype == np.uint8 and raw.shape[-1] % 2 == 0","tryCatchPattern":"try:\n    t = _bf16_tensor(raw)\nexcept ValueError:\n    raise RuntimeError('GGUF file appears corrupted or incompatible; re-download/re-convert') from None","preventionTips":["Verify GGUF integrity (size/sha256) before loading","Keep gguf reader and converter versions matched","Fail fast on odd byte counts or non-uint8 arrays with a clear message"],"tags":["gguf","bf16","data-layout","weights"],"backgroundTag":"invalid-binary-payload","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}