{"record":{"id":"a5bca08713ba6b67","repo":"hiyouga/LlamaFactory","slug":"please-use-scripts-pissa-init-py-to-initialize-pis-a5bca0","errorCode":null,"errorMessage":"Please use scripts/pissa_init.py to initialize PiSSA in DeepSpeed ZeRO-3.","messagePattern":"Please use scripts/pissa_init\\.py to initialize PiSSA in DeepSpeed ZeRO-3\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/hparams/parser.py","lineNumber":496,"sourceCode":"        raise ValueError(\"Please specify dataset for training.\")\n\n    if (training_args.do_eval or training_args.do_predict or training_args.predict_with_generate) and (\n        data_args.eval_dataset is None and data_args.val_size < 1e-6\n    ):\n        raise ValueError(\"Please make sure eval_dataset be provided or val_size >1e-6\")\n\n    if training_args.predict_with_generate:\n        if is_deepspeed_zero3_enabled():\n            raise ValueError(\"`predict_with_generate` is incompatible with DeepSpeed ZeRO-3.\")\n\n        if finetuning_args.compute_accuracy:\n            raise ValueError(\"Cannot use `predict_with_generate` and `compute_accuracy` together.\")\n\n    if training_args.do_train and model_args.quantization_device_map == \"auto\":\n        raise ValueError(\"Cannot use device map for quantized models in training.\")\n\n    if finetuning_args.pissa_init and is_deepspeed_zero3_enabled():\n        raise ValueError(\"Please use scripts/pissa_init.py to initialize PiSSA in DeepSpeed ZeRO-3.\")\n\n    if finetuning_args.pure_bf16:\n        if not (is_torch_bf16_gpu_available() or (is_torch_npu_available() and torch.npu.is_bf16_supported())):\n            raise ValueError(\"This device does not support `pure_bf16`.\")\n\n        if is_deepspeed_zero3_enabled():\n            raise ValueError(\"`pure_bf16` is incompatible with DeepSpeed ZeRO-3.\")\n\n    if training_args.parallel_mode == ParallelMode.DISTRIBUTED:\n        if finetuning_args.use_galore and finetuning_args.galore_layerwise:\n            raise ValueError(\"Distributed training does not support layer-wise GaLore.\")\n\n        if finetuning_args.use_apollo and finetuning_args.apollo_layerwise:\n            raise ValueError(\"Distributed training does not support layer-wise APOLLO.\")\n\n        if finetuning_args.use_badam:\n            if finetuning_args.badam_mode == \"ratio\":\n                raise ValueError(\"Radio-based BAdam does not yet support distributed training, use layer-wise BAdam.\")","sourceCodeStart":478,"sourceCodeEnd":514,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/hparams/parser.py#L478-L514","documentation":"Raised in parser.py:496 when finetuning_args.pissa_init is true and ZeRO-3 is enabled. PiSSA initialization factorizes the LoRA target matrices in-place, which requires full (unsharded) weight access; under ZeRO-3 the weights are partitioned so the init script must run beforehand.","triggerScenarios":"Config combining `pissa_init: true` with a DeepSpeed ZeRO-3 config (stage 3).","commonSituations":"Enabling PiSSA (a fast-convergence LoRA variant) on a multi-GPU setup that already uses ZeRO-3 to fit the base model.","solutions":["First run scripts/pissa_init.py to pre-compute and save the PiSSA-initialized base model, then point `model_name_or_path` at that output and disable pissa_init for the ZeRO-3 training run","Or train without ZeRO-3 (stage <= 2) so pissa_init can run inline"],"exampleFix":"# before (YAML)\ndeepspeed: examples/deepspeed/ds_z3_config.json\npissa_init: true\n\n# after\n# step 1: python scripts/pissa_init.py -m <model> -a <adapter> ...\n# step 2 (YAML):\ndeepspeed: examples/deepspeed/ds_z3_config.json\nmodel_name_or_path: <pissa_output_dir>\n# pissa_init removed","handlingStrategy":"validation","validationCode":"if config.get(\"pissa_init\") and is_zero3(config.get(\"deepspeed\")):\n    raise SystemExit(\"Run scripts/pissa_init.py first, then train the converted model without pissa_init\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Two-step PiSSA workflow: convert with the script, then train from its output","Never combine pissa_init with ZeRO-3 configs"],"tags":["pissa","lora","deepspeed","zero-3","incompatible-flags"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}