mudler/LocalAI · error · ValueError

engine_args is not valid JSON: {e}

Error message

engine_args is not valid JSON: {e}

What it means

Raised by the sglang backend's engine-args parser when the model config's engine_args string is not parseable JSON. The value is expected to be a JSON object serialized as a string; any JSON syntax error (trailing commas, single quotes, unquoted keys) surfaces here with the json module's position info chained.

Source

Thrown at backend/python/sglang/backend.py:125

    def _apply_engine_args(self, engine_kwargs: dict, engine_args_json: str) -> dict:
        """Merge user-supplied engine_args (JSON object) into the kwargs dict
        that will be forwarded to ``sglang.Engine`` (which constructs a
        ``ServerArgs`` from them).

        Mirrors ``backend/python/vllm/backend.py::_apply_engine_args`` but
        operates on the kwargs dict because sglang's ``Engine.__init__``
        accepts ``**kwargs`` directly rather than a pre-built dataclass.
        Validation happens against ``ServerArgs`` fields so a typo fails
        early with a close-match suggestion instead of producing a confusing
        ``TypeError`` deep inside engine startup.
        """
        if not engine_args_json:
            return engine_kwargs
        try:
            extra = json.loads(engine_args_json)
        except json.JSONDecodeError as e:
            raise ValueError(f"engine_args is not valid JSON: {e}") from e
        if not isinstance(extra, dict):
            raise ValueError(
                f"engine_args must be a JSON object, got {type(extra).__name__}"
            )
        valid = {f.name for f in dataclasses.fields(ServerArgs)}
        for key in extra:
            if key not in valid:
                suggestion = difflib.get_close_matches(key, valid, n=1)
                hint = f" did you mean {suggestion[0]!r}?" if suggestion else ""
                raise ValueError(f"unknown engine_args key {key!r}.{hint}")
        engine_kwargs.update(extra)
        return engine_kwargs

    def _messages_to_dicts(self, messages) -> List[dict]:
        result: List[dict] = []
        for msg in messages:
            d = {"role": msg.role, "content": msg.content or ""}
            if msg.name:

View on GitHub (pinned to 44413a9d06)

Solutions

1. Use strict double-quoted JSON: '{"tp_size": 2}' as a single-line string
2. Validate the string with a JSON linter or python -c 'import json,sys; json.load(open(sys.argv[1]))' before deploying
3. Remove trailing commas and use double quotes for keys and values

Example fix

# before (YAML)
engine_args: "{'tp_size': 2,}"

# after (YAML)
engine_args: '{"tp_size": 2}'

import json
raw = cfg.get('engine_args', '')
if raw:
    try:
        json.loads(raw)
    except json.JSONDecodeError as e:
        raise ValueError(f'fix engine_args in model YAML: {e}') from e

def is_valid_engine_args_json(s: str) -> bool:
    try:
        json.loads(s)
        return True
    except (TypeError, json.JSONDecodeError):
        return False

try:
    kwargs = _apply_engine_args(base, engine_args_json)
except ValueError as err:
    fail_config(f'model config error: {err}')  # abort startup with clear message

When it happens

Trigger: Setting engine_args in YAML config to single-quoted Python-dict syntax ("{'tp_size': 2}"), leaving a trailing comma, or breaking quoting across YAML lines so the string is truncated.

Common situations: Writing engine_args in model YAML where YAML and JSON quoting interact (nested quotes), hand-editing configs, or passing a dict where a JSON string is expected.

Related errors

• engine_args must be a JSON object, got {type(extra).__name__
• unknown engine_args key {key!r}.{hint}
• onnx_direct engine requires both detector_onnx and recognize
• unknown engine: {name!r}
• resolution must be 480p or 720p

AI-assisted analysis of mudler/LocalAI@44413a9d06 (2026-08-15). Data as JSON: /api/errors/5cefb7a440edd6e5. Report an issue: GitHub.