mudler/LocalAI · error · ValueError

engine_args must be a JSON object, got {type(extra).__name__

Error message

engine_args must be a JSON object, got {type(extra).__name__}

What it means

Raised by the sglang backend's engine-args parser when the engine_args JSON parses successfully but is not an object — e.g. a JSON array, string, number, or boolean. Only a dict can be merged into the engine kwargs, so any other top-level JSON type is rejected with the offending type name in the message.

Source

Thrown at backend/python/sglang/backend.py:127

        """Merge user-supplied engine_args (JSON object) into the kwargs dict
        that will be forwarded to ``sglang.Engine`` (which constructs a
        ``ServerArgs`` from them).

        Mirrors ``backend/python/vllm/backend.py::_apply_engine_args`` but
        operates on the kwargs dict because sglang's ``Engine.__init__``
        accepts ``**kwargs`` directly rather than a pre-built dataclass.
        Validation happens against ``ServerArgs`` fields so a typo fails
        early with a close-match suggestion instead of producing a confusing
        ``TypeError`` deep inside engine startup.
        """
        if not engine_args_json:
            return engine_kwargs
        try:
            extra = json.loads(engine_args_json)
        except json.JSONDecodeError as e:
            raise ValueError(f"engine_args is not valid JSON: {e}") from e
        if not isinstance(extra, dict):
            raise ValueError(
                f"engine_args must be a JSON object, got {type(extra).__name__}"
            )
        valid = {f.name for f in dataclasses.fields(ServerArgs)}
        for key in extra:
            if key not in valid:
                suggestion = difflib.get_close_matches(key, valid, n=1)
                hint = f" did you mean {suggestion[0]!r}?" if suggestion else ""
                raise ValueError(f"unknown engine_args key {key!r}.{hint}")
        engine_kwargs.update(extra)
        return engine_kwargs

    def _messages_to_dicts(self, messages) -> List[dict]:
        result: List[dict] = []
        for msg in messages:
            d = {"role": msg.role, "content": msg.content or ""}
            if msg.name:
                d["name"] = msg.name
            if msg.tool_call_id:

View on GitHub (pinned to 44413a9d06)

Solutions

1. Use a JSON object with ServerArgs field names as keys: '{"tp_size": 2}'
2. Convert CLI flags to object form (--tp-size 2 -> "tp_size": 2)
3. Drop outer quotes that turn the object into a string

Example fix

# before
engine_args: '["--tp-size", "2"]'

# after
engine_args: '{"tp_size": 2}'

import json
extra = json.loads(cfg['engine_args'])
if not isinstance(extra, dict):
    raise ValueError(f'engine_args must be a JSON object, got {type(extra).__name__}')

def is_engine_args_object(s: str) -> bool:
    try:
        return isinstance(json.loads(s), dict)
    except (TypeError, json.JSONDecodeError):
        return False

try:
    kwargs = _apply_engine_args(base, engine_args_json)
except ValueError as err:
    fail_config(str(err))

When it happens

Trigger: engine_args set to '["--tp-size", "2"]' (array of CLI-style flags) or '"tp_size=2"' (a plain JSON string) instead of an object.

Common situations: Users porting CLI-style argument lists from sglang/vllm command lines into engine_args, or wrapping the whole thing in quotes so it parses as a JSON string.

Related errors

• engine_args is not valid JSON: {e}
• unknown engine_args key {key!r}.{hint}
• onnx_direct engine requires both detector_onnx and recognize
• unknown engine: {name!r}
• resolution must be 480p or 720p

AI-assisted analysis of mudler/LocalAI@44413a9d06 (2026-08-15). Data as JSON: /api/errors/4541bb2972aebf65. Report an issue: GitHub.