{"record":{"id":"5d516b217064aa32","repo":"hiyouga/LlamaFactory","slug":"ktransformers-is-incompatible-with-deepspeed-zero","errorCode":null,"errorMessage":"KTransformers is incompatible with DeepSpeed ZeRO-3.","messagePattern":"KTransformers is incompatible with DeepSpeed ZeRO-3\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/hparams/parser.py","lineNumber":536,"sourceCode":"    if training_args.deepspeed is not None and (finetuning_args.use_galore or finetuning_args.use_apollo):\n        raise ValueError(\"GaLore and APOLLO are incompatible with DeepSpeed yet.\")\n\n    if (\n        not finetuning_args.use_mca\n        and not finetuning_args.use_megatron_bridge\n        and training_args.fp8\n        and model_args.quantization_bit is not None\n    ):\n        raise ValueError(\"FP8 training is not compatible with quantization. Please disable one of them.\")\n\n    if model_args.infer_backend != EngineName.HF:\n        raise ValueError(\"vLLM/SGLang backend is only available for API, CLI and Web.\")\n\n    if model_args.use_unsloth and is_deepspeed_zero3_enabled():\n        raise ValueError(\"Unsloth is incompatible with DeepSpeed ZeRO-3.\")\n\n    if model_args.use_kt and is_deepspeed_zero3_enabled():\n        raise ValueError(\"KTransformers is incompatible with DeepSpeed ZeRO-3.\")\n\n    _set_env_vars()\n    _verify_model_args(model_args, data_args, finetuning_args)\n    _check_extra_dependencies(model_args, finetuning_args, training_args)\n    _verify_trackio_args(training_args)\n\n    if (\n        not finetuning_args.use_mca\n        and not finetuning_args.use_megatron_bridge\n        and training_args.fp8_enable_fsdp_float8_all_gather\n        and not training_args.fp8\n    ):\n        logger.warning_rank0(\"fp8_enable_fsdp_float8_all_gather requires fp8=True. Setting fp8=True.\")\n        model_args.fp8 = True\n\n    if (\n        training_args.do_train\n        and finetuning_args.finetuning_type == \"lora\"","sourceCodeStart":518,"sourceCodeEnd":554,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/hparams/parser.py#L518-L554","documentation":"Raised in get_train_args when model_args.use_kt is true and DeepSpeed ZeRO-3 is enabled in the training config. KTransformers offloads layers to custom CPU/GPU kernels, which is fundamentally incompatible with ZeRO-3's sharding of parameters across processes. The check runs before any training starts, so it fails fast at argument-parsing time.","triggerScenarios":"Set enable_kt: true (use_kt) in the model section of a YAML training config while the DeepSpeed config file (e.g. ds_z3_config.json with zero_optimization.stage=3) is passed via deepspeed. Any combination of use_kt + stage-3 ZeRO triggers it.","commonSituations":"Copying a ZeRO-3 multi-GPU example config and adding KTransformers flags to speed up local training; upgrading a previously working KT setup after switching to a ZeRO-3 deepspeed stage to fit a larger model.","solutions":["Switch the DeepSpeed config to ZeRO stage 0/1/2 (or remove the deepspeed argument entirely) when using KTransformers.","Remove enable_kt: true from the model arguments and let the standard HF/DeepSpeed path load the model.","Use ZeRO-2 with offload if memory was the reason for ZeRO-3, since KT already handles its own layer placement."],"exampleFix":"# before\nmodel_args:\n  enable_kt: true\ndeepspeed: examples/deepspeed/ds_z3_config.json\n\n# after\nmodel_args:\n  enable_kt: true\ndeepspeed: examples/deepspeed/ds_z2_config.json","handlingStrategy":"validation","validationCode":"# before launching training\nimport json, os\nif cfg.get(\"model_args\", {}).get(\"enable_kt\"):\n    ds = cfg.get(\"deepspeed\")\n    if ds and os.path.isfile(ds):\n        stage = json.load(open(ds))[\"zero_optimization\"][\"stage\"]\n        assert stage != 3, \"KTransformers cannot be combined with ZeRO-3; use stage <= 2\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Validate the deepspeed stage against enable_kt in a config lint step of your training script.","Keep separate YAML presets for KT (no ZeRO-3) and multi-node ZeRO-3 runs."],"tags":["ktransformers","deepspeed","zero3","training-config"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}