{"record":{"id":"12ac2b91e1a05509","repo":"hiyouga/LlamaFactory","slug":"stage-does-not-supported-stage","errorCode":null,"errorMessage":"Stage does not supported: {stage}.","messagePattern":"Stage does not supported: (.+?)\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"scripts/stat_utils/cal_lr.py","lineNumber":79,"sourceCode":"            packing=packing,\n            preprocessing_num_workers=16,\n            output_dir=\"dummy_dir\",\n            overwrite_cache=True,\n            do_train=True,\n        )\n    )\n    tokenizer_module = load_tokenizer(model_args)\n    tokenizer = tokenizer_module[\"tokenizer\"]\n    template = get_template_and_fix_tokenizer(tokenizer, data_args)\n    trainset = get_dataset(template, model_args, data_args, training_args, stage, **tokenizer_module)[\"train_dataset\"]\n    if stage == \"pt\":\n        data_collator = DataCollatorForLanguageModeling(tokenizer=tokenizer, mlm=False)\n    elif stage == \"sft\":\n        data_collator = MultiModalDataCollatorForSeq2Seq(\n            template=template, tokenizer=tokenizer, label_pad_token_id=IGNORE_INDEX\n        )\n    else:\n        raise NotImplementedError(f\"Stage does not supported: {stage}.\")\n\n    dataloader = DataLoader(trainset, batch_size, shuffle=False, collate_fn=data_collator, pin_memory=True)\n    valid_tokens, total_tokens = 0, 0\n    for batch in tqdm(dataloader, desc=\"Collecting valid tokens\"):\n        valid_tokens += torch.sum(batch[\"labels\"] != IGNORE_INDEX).item()\n        total_tokens += torch.numel(batch[\"labels\"])\n\n    valid_ratio = valid_tokens / total_tokens\n    token_batch_size = cutoff_len * batch_size * valid_ratio\n    lr = BASE_LR * math.sqrt(token_batch_size / BASE_BS)  # lr ~ sqrt(batch_size)\n    lr = lr / 6.0 if is_mistral_or_gemma else lr\n    print(\n        f\"Optimal learning rate is {lr:.2e} for valid ratio% {valid_ratio * 100:.2f} \"\n        f\"and effective token batch size {token_batch_size:.2f}\"\n    )\n\n\nif __name__ == \"__main__\":","sourceCodeStart":61,"sourceCodeEnd":97,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/scripts/stat_utils/cal_lr.py#L61-L97","documentation":"The HyperParallel workflow builds HyperParallelArguments from finetuning args (workflow.py:48); the integration lives in the separate `hyper_parallel` package, so the path first checks is_hyper_parallel_available() and raises ImportError with install instructions when the package is absent. This is the entry gate before any HP feature (CP, activation optimization) can be used.","triggerScenarios":"Setting HyperParallel-related finetuning args (e.g. hyper_parallel_cp_size > 1 or an activation_mode) in an environment where `import hyper_parallel` fails; _prepare_hp_args raises immediately during workflow setup.","commonSituations":"Enabling experimental HP features in a base LlamaFactory install; fresh clones where extras/requirements-hyper-parallel were not installed; version drift making the hyper_parallel import fail even though it is listed.","solutions":["pip install hyper_parallel","If the import still fails after install, verify with `python -c \"import hyper_parallel\"` and check for conflicting packages (e.g. mismatched torch) before retrying training","If you don't need HP features, remove the HyperParallel args from the YAML"],"exampleFix":"# before\n# YAML has hyper_parallel_cp_size: 4, package missing -> ImportError\n\n# after\npip install hyper_parallel","handlingStrategy":"validation","validationCode":"from llamafactory.extras.packages import is_hyper_parallel_available\nif uses_hyper_parallel(finetuning_args):\n    assert is_hyper_parallel_available(), 'pip install hyper_parallel'","typeGuard":"def hyper_parallel_ready() -> bool:\n    try:\n        import hyper_parallel  # noqa: F401\n        return True\n    except ImportError:\n        return False","tryCatchPattern":"try:\n    run_exp()\nexcept ImportError as e:\n    if 'hyper_parallel' in str(e):\n        raise SystemExit('pip install hyper_parallel') from e\n    raise","preventionTips":["Install HP extras together with enabling HP flags; keep one env per backend","Gate experimental backends behind availability checks in your launcher"],"tags":["dependency","hyper-parallel","installation"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}