mudler/LocalAI · error · ValueError

engine_args is not valid JSON: {e}

Error message

engine_args is not valid JSON: {e}

What it means

The vLLM backend accepts engine tuning as a JSON string in the `engine_args` model option and merges it over a default EngineArgs dataclass via dataclasses.replace. Malformed JSON fails json.loads and is re-raised as ValueError with the JSONDecodeError detail, distinguishing payload problems from the later unknown-key check.

Source

Thrown at backend/python/vllm/backend.py:136

    def _apply_engine_args(self, engine_args, engine_args_json):
        """Apply user-supplied engine_args (JSON object) onto an AsyncEngineArgs.

        Returns a new AsyncEngineArgs with the typed fields preserved and the
        user's overrides layered on top. Uses ``dataclasses.replace`` so vLLM's
        ``__post_init__`` re-runs and auto-converts dict-valued fields like
        ``compilation_config`` / ``attention_config`` into their dataclass form.
        ``speculative_config`` and ``kv_transfer_config`` are accepted as dicts
        directly (vLLM converts them at engine init).

        Unknown keys raise ValueError with the closest valid field as a hint.
        """
        if not engine_args_json:
            return engine_args
        try:
            extra = json.loads(engine_args_json)
        except json.JSONDecodeError as e:
            raise ValueError(f"engine_args is not valid JSON: {e}") from e
        if not isinstance(extra, dict):
            raise ValueError(
                f"engine_args must be a JSON object, got {type(extra).__name__}"
            )
        valid = {f.name for f in dataclasses.fields(type(engine_args))}
        for key in extra:
            if key not in valid:
                suggestion = difflib.get_close_matches(key, valid, n=1)
                hint = f" did you mean {suggestion[0]!r}?" if suggestion else ""
                raise ValueError(f"unknown engine_args key {key!r}.{hint}")
        return dataclasses.replace(engine_args, **extra)

    def _messages_to_dicts(self, messages):
        """Convert proto Messages to list of dicts suitable for apply_chat_template()."""
        result = []
        for msg in messages:
            d = {"role": msg.role, "content": msg.content or ""}
            if msg.name:

View on GitHub (pinned to 44413a9d06)

Solutions

1. Validate the string with a JSON linter, then re-embed it (use a YAML block scalar '|-' or proper quoting).
2. Use strict JSON: double quotes on keys and values, no trailing commas.
3. Build the option programmatically with json.dumps in client code instead of string concatenation.

Example fix

# before (model YAML)
engine_args: '{ gpu_memory_utilization: 0.9, }'
# after
engine_args: '{"gpu_memory_utilization": 0.9}'

import json

def engine_args_parse_ok(engine_args_json) -> bool:
    if not engine_args_json:
        return True
    try:
        v = json.loads(engine_args_json)
        return isinstance(v, dict)
    except json.JSONDecodeError:
        return False

try:
    args = apply_engine_args(defaults, engine_args_json)
except ValueError as e:
    if "not valid JSON" in str(e):
        return config_error(field="engine_args", detail=str(e))
    raise  # unknown-key errors carry a 'did you mean' hint

When it happens

Trigger: Model YAML with engine_args: '{gpu_memory_utilization: 0.9}' (single quotes, unquoted keys — invalid JSON); trailing commas; newlines mangled by YAML block scalar handling; engine_args passed as a dict where a string is expected by this parse step.

Common situations: Writing JSON inside YAML model configs where YAML quoting rules corrupt it; hand-editing configs; clients serializing with repr() instead of json.dumps.

Related errors

• model snapshot does not exist: {model_ref}
• model snapshot must contain exactly one {suffix} file; found
• {raw!r} is not a boolean
• no insightface pack '{self.model_pack}' found — install via
• engine_args is not valid JSON: {e}

AI-assisted analysis of mudler/LocalAI@44413a9d06 (2026-08-15). Data as JSON: /api/errors/9a833882c530101a. Report an issue: GitHub.