hiyouga/LlamaFactory · error · ValueError

`extra_config` file not found

Error message

`extra_config` file not found: {self.extra_config}

What it means

Raised by MegatronBridgeArguments.load_extra_config (megatron_bridge_args.py:191) when extra_config is a string that is neither inline JSON nor an existing file path. __post_init__ first tries to parse the string as JSON if it starts with '{'; otherwise the raw string is treated as a file path, and load_extra_config raises when os.path.isfile fails. This runs at training launch, when the bridge arguments are constructed and extra config is loaded.

Solutions

1. Verify the path exists and is readable from the directory you launch the CLI from; prefer absolute paths
2. If you meant inline settings, pass a JSON object string starting with '{' (e.g. extra_config: '{"foo": 1}') or a YAML mapping that becomes a dict
3. Check for typos and stray whitespace in the path

Example fix

# before
extra_config: config/megatron_extra.json   # missing or wrong cwd

# after
extra_config: /abs/path/config/megatron_extra.json
# or inline:
extra_config: '{"optimizer": {"bar": 1}}'

import os
ec = cfg.get('extra_config')
if isinstance(ec, str) and not ec.strip().startswith('{'):
    ec = os.path.abspath(ec)
    assert os.path.isfile(ec), f'extra_config file missing: {ec}'
    cfg['extra_config'] = ec

def extra_config_resolvable(v: str | dict | None) -> bool:
    if v is None or isinstance(v, dict):
        return True
    return v.strip().startswith('{') or os.path.isfile(v)

try:
    extra = bridge_args.load_extra_config()
except ValueError as e:
    if 'file not found' in str(e):
        cfg['extra_config'] = inline_json_dict  # fall back to inline config
        bridge_args = MegatronBridgeArguments(**cfg)
    else:
        raise

When it happens

Trigger: Passing extra_config: /path/that/does/not/exist.json; a relative path evaluated from a different working directory than expected; a typo'd path; passing plain key=value text that neither starts with '{' nor names a file.

Common situations: Running llamafactory-cli train from a different cwd so relative paths break; file created after config written but deleted/renamed; typos in long absolute paths; permission issues that make isfile return False in unusual container mounts.

Related errors

• `moe_token_dispatcher_type` must be 'allgather'…
• `recompute_granularity` must be 'full' or 'selective'.
• `recompute_method` must be 'uniform' or 'block'.
• `recompute_num_layers` must be >= 1 when set.
• `sequence_parallel` requires `tensor_model_parallel_size` >…

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/974c0b02c6ff7a31. Report an issue: GitHub.

Appendix: source

Thrown at src/llamafactory/hparams/megatron_bridge_args.py:191

            "alltoall",
            "flex",
        ):
            raise ValueError("`moe_token_dispatcher_type` must be 'allgather', 'alltoall', or 'flex'.")

        if isinstance(self.extra_config, str):
            config_str = self.extra_config.strip()
            if config_str.startswith("{"):
                self.extra_config = _convert_str_dict(json.loads(config_str))
            else:
                self.extra_config = config_str

    def load_extra_config(self) -> dict:
        if self.extra_config is None:
            return {}
        if isinstance(self.extra_config, dict):
            return self.extra_config
        if not os.path.isfile(self.extra_config):
            raise ValueError(f"`extra_config` file not found: {self.extra_config}")
        with open(self.extra_config, encoding="utf-8") as f:
            return json.load(f)

View on GitHub (pinned to f28afaf635)