{"record":{"id":"9a833882c530101a","repo":"mudler/LocalAI","slug":"engine-args-is-not-valid-json-e-9a8338","errorCode":null,"errorMessage":"engine_args is not valid JSON: {e}","messagePattern":"engine_args is not valid JSON: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"backend/python/vllm/backend.py","lineNumber":136,"sourceCode":"\n    def _apply_engine_args(self, engine_args, engine_args_json):\n        \"\"\"Apply user-supplied engine_args (JSON object) onto an AsyncEngineArgs.\n\n        Returns a new AsyncEngineArgs with the typed fields preserved and the\n        user's overrides layered on top. Uses ``dataclasses.replace`` so vLLM's\n        ``__post_init__`` re-runs and auto-converts dict-valued fields like\n        ``compilation_config`` / ``attention_config`` into their dataclass form.\n        ``speculative_config`` and ``kv_transfer_config`` are accepted as dicts\n        directly (vLLM converts them at engine init).\n\n        Unknown keys raise ValueError with the closest valid field as a hint.\n        \"\"\"\n        if not engine_args_json:\n            return engine_args\n        try:\n            extra = json.loads(engine_args_json)\n        except json.JSONDecodeError as e:\n            raise ValueError(f\"engine_args is not valid JSON: {e}\") from e\n        if not isinstance(extra, dict):\n            raise ValueError(\n                f\"engine_args must be a JSON object, got {type(extra).__name__}\"\n            )\n        valid = {f.name for f in dataclasses.fields(type(engine_args))}\n        for key in extra:\n            if key not in valid:\n                suggestion = difflib.get_close_matches(key, valid, n=1)\n                hint = f\" did you mean {suggestion[0]!r}?\" if suggestion else \"\"\n                raise ValueError(f\"unknown engine_args key {key!r}.{hint}\")\n        return dataclasses.replace(engine_args, **extra)\n\n    def _messages_to_dicts(self, messages):\n        \"\"\"Convert proto Messages to list of dicts suitable for apply_chat_template().\"\"\"\n        result = []\n        for msg in messages:\n            d = {\"role\": msg.role, \"content\": msg.content or \"\"}\n            if msg.name:","sourceCodeStart":118,"sourceCodeEnd":154,"githubUrl":"https://github.com/mudler/LocalAI/blob/44413a9d06bf5bc52ce088ba8ca74e5a2e8bee26/backend/python/vllm/backend.py#L118-L154","documentation":"The vLLM backend accepts engine tuning as a JSON string in the `engine_args` model option and merges it over a default EngineArgs dataclass via dataclasses.replace. Malformed JSON fails json.loads and is re-raised as ValueError with the JSONDecodeError detail, distinguishing payload problems from the later unknown-key check.","triggerScenarios":"Model YAML with engine_args: '{gpu_memory_utilization: 0.9}' (single quotes, unquoted keys — invalid JSON); trailing commas; newlines mangled by YAML block scalar handling; engine_args passed as a dict where a string is expected by this parse step.","commonSituations":"Writing JSON inside YAML model configs where YAML quoting rules corrupt it; hand-editing configs; clients serializing with repr() instead of json.dumps.","solutions":["Validate the string with a JSON linter, then re-embed it (use a YAML block scalar '|-' or proper quoting).","Use strict JSON: double quotes on keys and values, no trailing commas.","Build the option programmatically with json.dumps in client code instead of string concatenation."],"exampleFix":"# before (model YAML)\nengine_args: '{ gpu_memory_utilization: 0.9, }'\n# after\nengine_args: '{\"gpu_memory_utilization\": 0.9}'","handlingStrategy":"validation","validationCode":"import json\n\ndef engine_args_parse_ok(engine_args_json) -> bool:\n    if not engine_args_json:\n        return True\n    try:\n        v = json.loads(engine_args_json)\n        return isinstance(v, dict)\n    except json.JSONDecodeError:\n        return False","typeGuard":null,"tryCatchPattern":"try:\n    args = apply_engine_args(defaults, engine_args_json)\nexcept ValueError as e:\n    if \"not valid JSON\" in str(e):\n        return config_error(field=\"engine_args\", detail=str(e))\n    raise  # unknown-key errors carry a 'did you mean' hint","preventionTips":["Generate engine_args with json.dumps, never by hand or str(dict).","Validate model YAML with a linter that parses embedded JSON fields.","Reuse the error's did-you-mean hints to fix unknown keys quickly."],"tags":["vllm","engine-args","json","configuration","localai"],"backgroundTag":null,"analyzedSha":"44413a9d06bf5bc52ce088ba8ca74e5a2e8bee26","analyzedAt":"2026-08-15T10:13:50.291Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}