sgl-project/sglang · error · ValueError

Failed to load mistral

Error message

Failed to load mistral '{config_file_name}' config for model {model}. Please check if the model is a mistral-format model and if the config file exists.

What it means

After reading params.json for a Mistral-format model, the parsed config came back None/empty, so the model is likely not Mistral-format or the params.json is malformed. Note the code path makes this reachable mainly when the JSON file parses to null.

Solutions

  1. Inspect params.json in the model dir and confirm it is a non-null JSON object
  2. Re-download the model snapshot (hf download / huggingface-cli) to repair corrupted files
  3. Use the HF-format variant of the model
Defensive patterns

Strategy: validation

Validate before calling

import json; d = json.load(open(Path(model)/'params.json'))
assert isinstance(d, dict) and d

Prevention

When it happens

Trigger: A params.json that literally contains 'null' or is emptied out, while --model-config-parser mistral is active.

Common situations: Corrupted or placeholder params.json in a partially downloaded model directory.

Related errors


AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/3be5255608c258d0. Report an issue: GitHub.

Appendix: source

Thrown at python/sglang/srt/utils/hf_transformers/mistral_utils.py:312

    return config


class MistralConfigParser:
    def get_hf_file_to_dict(
        self, file_name: str, model: str | Path, revision: str | None = "main"
    ):
        file_path = Path(model) / file_name
        if not file_path.is_file():
            raise FileNotFoundError(f"File not found {model}, {file_name}")

        with open(file_path) as file:
            return json.load(file)

    def _download_mistral_config_file(self, model, revision) -> dict:
        config_file_name = "params.json"
        config_dict = self.get_hf_file_to_dict(config_file_name, model, revision)
        if config_dict is None:
            raise ValueError(
                f"Failed to load mistral '{config_file_name}' config for model "
                f"{model}. Please check if the model is a mistral-format model "
                f"and if the config file exists."
            )
        assert isinstance(config_dict, dict)
        return config_dict

    def parse(
        self,
        model: str | Path,
        revision: str | None = None,
        **kwargs,
    ) -> tuple[dict, PretrainedConfig]:
        config_dict = self._download_mistral_config_file(model, revision)
        if config_dict.get("max_position_embeddings") is None:
            logger.warning(
                "The params.json file is missing 'max_position_embeddings'"
                " and could not get a value from the HF config."

View on GitHub (pinned to 0132848349)