{"record":{"id":"0380cfa2b071c673","repo":"mudler/LocalAI","slug":"engine-args-must-be-a-json-object-got-type-extra-0380cf","errorCode":null,"errorMessage":"engine_args must be a JSON object, got {type(extra).__name__}","messagePattern":"engine_args must be a JSON object, got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"backend/python/vllm/backend.py","lineNumber":138,"sourceCode":"        \"\"\"Apply user-supplied engine_args (JSON object) onto an AsyncEngineArgs.\n\n        Returns a new AsyncEngineArgs with the typed fields preserved and the\n        user's overrides layered on top. Uses ``dataclasses.replace`` so vLLM's\n        ``__post_init__`` re-runs and auto-converts dict-valued fields like\n        ``compilation_config`` / ``attention_config`` into their dataclass form.\n        ``speculative_config`` and ``kv_transfer_config`` are accepted as dicts\n        directly (vLLM converts them at engine init).\n\n        Unknown keys raise ValueError with the closest valid field as a hint.\n        \"\"\"\n        if not engine_args_json:\n            return engine_args\n        try:\n            extra = json.loads(engine_args_json)\n        except json.JSONDecodeError as e:\n            raise ValueError(f\"engine_args is not valid JSON: {e}\") from e\n        if not isinstance(extra, dict):\n            raise ValueError(\n                f\"engine_args must be a JSON object, got {type(extra).__name__}\"\n            )\n        valid = {f.name for f in dataclasses.fields(type(engine_args))}\n        for key in extra:\n            if key not in valid:\n                suggestion = difflib.get_close_matches(key, valid, n=1)\n                hint = f\" did you mean {suggestion[0]!r}?\" if suggestion else \"\"\n                raise ValueError(f\"unknown engine_args key {key!r}.{hint}\")\n        return dataclasses.replace(engine_args, **extra)\n\n    def _messages_to_dicts(self, messages):\n        \"\"\"Convert proto Messages to list of dicts suitable for apply_chat_template().\"\"\"\n        result = []\n        for msg in messages:\n            d = {\"role\": msg.role, \"content\": msg.content or \"\"}\n            if msg.name:\n                d[\"name\"] = msg.name\n            if msg.tool_call_id:","sourceCodeStart":120,"sourceCodeEnd":156,"githubUrl":"https://github.com/mudler/LocalAI/blob/44413a9d06bf5bc52ce088ba8ca74e5a2e8bee26/backend/python/vllm/backend.py#L120-L156","documentation":"Raised by _apply_engine_args in the LocalAI vLLM backend when the YAML model config's engine_args setting parses as valid JSON but is not a JSON object (dict). The value is meant to be overlaid onto a vllm AsyncEngineArgs dataclass via dataclasses.replace, which only works with key/value pairs. Any JSON array, string, number, or boolean triggers this ValueError.","triggerScenarios":"Setting engine_args: '[1,2,3]' or engine_args: '\"auto\"' or engine_args: '0.9' in a model YAML. The string is json.loads-ed successfully but isinstance(extra, dict) is False, so the backend raises before loading the model.","commonSituations":"Operators pasting a bare scalar (e.g. a dtype or tensor_parallel_size value) into engine_args instead of an object; wrapping keys in quotes so the whole thing parses as one string; copying a JSON array from docs; YAML unquoted scalars that look like numbers.","solutions":["Change engine_args in the model YAML to a JSON object of AsyncEngineArgs field names, e.g. engine_args: '{\"dtype\": \"auto\", \"max_model_len\": 8192}'","Quote the whole JSON string in YAML if it contains special characters (single quotes are safest)","Remove the engine_args key entirely if you only need one of the typed fields (Quantization, MaxModelLen, GPUMemoryUtilization, ...) which have dedicated YAML keys","Check backend stderr — the backend prints 'engine_args error: ...' and exits the Load flow"],"exampleFix":"# before (model.yaml)\nengine_args: 0.9\n# after\nengine_args: '{\"gpu_memory_utilization\": 0.9}'","handlingStrategy":"validation","validationCode":"import json\ncfg = model_config.get('engine_args')\nif cfg is not None:\n    v = json.loads(cfg)\n    assert isinstance(v, dict), f'engine_args must be a JSON object, got {type(v).__name__}'","typeGuard":"def is_engine_args_object(s: str) -> bool:\n    try:\n        return isinstance(json.loads(s), dict)\n    except json.JSONDecodeError:\n        return False","tryCatchPattern":"try:\n    load_model(cfg)\nexcept ValueError as e:\n    if 'engine_args must be a JSON object' in str(e):\n        fix_engine_args_in_yaml(cfg)  # make it an object, reload\n    else:\n        raise","preventionTips":["Always write engine_args as a JSON object string in YAML, e.g. engine_args: '{\"key\": value}'","Lint model YAML: json.loads(engine_args) must yield a dict before submitting to the backend","Prefer the dedicated typed YAML fields for single options"],"tags":["vllm","config","json","yaml","backend"],"backgroundTag":null,"analyzedSha":"44413a9d06bf5bc52ce088ba8ca74e5a2e8bee26","analyzedAt":"2026-08-15T10:13:50.291Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}