hiyouga/LlamaFactory · error · ValueError

`extra_config` file not found: {self.extra_config}

Error message

`extra_config` file not found: {self.extra_config}

What it means

Raised by MegatronBridgeArguments.load_extra_config (megatron_bridge_args.py:191) when extra_config is a string that is neither inline JSON nor an existing file path. __post_init__ first tries to parse the string as JSON if it starts with '{'; otherwise the raw string is treated as a file path, and load_extra_config raises when os.path.isfile fails. This runs at training launch, when the bridge arguments are constructed and extra config is loaded.

Source

Thrown at src/llamafactory/hparams/megatron_bridge_args.py:191

            "alltoall",
            "flex",
        ):
            raise ValueError("`moe_token_dispatcher_type` must be 'allgather', 'alltoall', or 'flex'.")

        if isinstance(self.extra_config, str):
            config_str = self.extra_config.strip()
            if config_str.startswith("{"):
                self.extra_config = _convert_str_dict(json.loads(config_str))
            else:
                self.extra_config = config_str

    def load_extra_config(self) -> dict:
        if self.extra_config is None:
            return {}
        if isinstance(self.extra_config, dict):
            return self.extra_config
        if not os.path.isfile(self.extra_config):
            raise ValueError(f"`extra_config` file not found: {self.extra_config}")
        with open(self.extra_config, encoding="utf-8") as f:
            return json.load(f)

View on GitHub (pinned to f28afaf635)

Solutions

1. Verify the path exists and is readable from the directory you launch the CLI from; prefer absolute paths
2. If you meant inline settings, pass a JSON object string starting with '{' (e.g. extra_config: '{"foo": 1}') or a YAML mapping that becomes a dict
3. Check for typos and stray whitespace in the path

Example fix

# before
extra_config: config/megatron_extra.json   # missing or wrong cwd

# after
extra_config: /abs/path/config/megatron_extra.json
# or inline:
extra_config: '{"optimizer": {"bar": 1}}'

import os
ec = cfg.get('extra_config')
if isinstance(ec, str) and not ec.strip().startswith('{'):
    ec = os.path.abspath(ec)
    assert os.path.isfile(ec), f'extra_config file missing: {ec}'
    cfg['extra_config'] = ec

def extra_config_resolvable(v: str | dict | None) -> bool:
    if v is None or isinstance(v, dict):
        return True
    return v.strip().startswith('{') or os.path.isfile(v)

try:
    extra = bridge_args.load_extra_config()
except ValueError as e:
    if 'file not found' in str(e):
        cfg['extra_config'] = inline_json_dict  # fall back to inline config
        bridge_args = MegatronBridgeArguments(**cfg)
    else:
        raise

When it happens

Trigger: Passing extra_config: /path/that/does/not/exist.json; a relative path evaluated from a different working directory than expected; a typo'd path; passing plain key=value text that neither starts with '{' nor names a file.

Common situations: Running llamafactory-cli train from a different cwd so relative paths break; file created after config written but deleted/renamed; typos in long absolute paths; permission issues that make isfile return False in unusual container mounts.

Related errors

• `virtual_pipeline_model_parallel_size` must be >= 1 when set
• `sequence_parallel` requires `tensor_model_parallel_size` >
• `recompute_granularity` must be 'full' or 'selective'.
• `recompute_method` must be 'uniform' or 'block'.
• `recompute_num_layers` must be >= 1 when set.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/974c0b02c6ff7a31. Report an issue: GitHub.