{"record":{"id":"683024b5926d9047","repo":"huggingface/transformers","slug":"prompt-must-be-a-string","errorCode":null,"errorMessage":"prompt must be a string.","messagePattern":"prompt must be a string\\.","errorType":"http","errorClass":"HTTPException","httpStatus":400,"severity":"error","filePath":"src/transformers/cli/serving/completion.py","lineNumber":91,"sourceCode":"\n    _valid_params_class = TransformersTextCompletionCreateParams\n    _unused_fields = UNUSED_LEGACY_COMPLETION_FIELDS\n\n    async def handle_request(self, body: dict, request_id: str) -> \"StreamingResponse | JSONResponse\":\n        \"\"\"Validate the request, load the model, and dispatch to streaming or non-streaming.\n\n        Args:\n            body (`dict`): The raw JSON request body (OpenAI legacy completions format).\n            request_id (`str`): Unique request identifier (from header or auto-generated).\n\n        Returns:\n            `StreamingResponse | JSONResponse`: SSE stream or JSON depending on ``body[\"stream\"]``.\n        \"\"\"\n        self._validate_request(body)\n\n        prompt = body.get(\"prompt\", \"\")\n        if not isinstance(prompt, str):\n            raise HTTPException(status_code=400, detail=\"prompt must be a string.\")\n\n        model_id, model, processor = self._resolve_model(body)\n        modality = self.model_manager.get_model_modality(model, processor=processor)\n        use_cb = self.generation_state.use_continuous_batching(model, modality)\n        logger.warning(f\"[Request received] Model: {model_id}, CB: {use_cb}\")\n        gen_manager = self.generation_state.get_manager(model_id, use_cb=use_cb)\n\n        tokenizer = getattr(processor, \"tokenizer\", processor)\n        inputs = tokenizer(prompt, return_tensors=None if use_cb else \"pt\")\n        if not use_cb:\n            inputs = inputs.to(model.device)\n\n        gen_config = self._build_generation_config(body, model.generation_config, use_cb=use_cb)\n        if use_cb:\n            gen_manager.init_cb(model, gen_config)\n\n        suffix = body.get(\"suffix\")\n        streaming = body.get(\"stream\")","sourceCodeStart":73,"sourceCodeEnd":109,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/cli/serving/completion.py#L73-L109","documentation":"In the legacy/OpenAI-style completions endpoint of `transformers serve`, the handler reads body['prompt'] (defaulting to '') and requires it to be a Python str. Anything else — most commonly a list of prompt strings (allowed by the OpenAI API for multi-prompt batching) or a list of token ids — triggers HTTP 400 'prompt must be a string.'.","triggerScenarios":"POSTing to /v1/completions with \"prompt\": [\"a\", \"b\"] (OpenAI multi-prompt form); sending token-id arrays like [50256, 11]; sending an int/None/null prompt; using a client SDK that auto-encodes prompts as lists.","commonSituations":"Porting code from the OpenAI API where batched string prompts were accepted; sending pre-tokenized ids; a client serializing a single-element list.","solutions":["Send a single string: {\"model\": ..., \"prompt\": \"Once upon a time\"}","Loop client-side over multiple prompts, issuing one request per prompt","Pre-decode token ids to text before sending","Ensure the JSON field is a plain str, not a list or null"],"exampleFix":"# before\ncurl -X POST localhost:8000/v1/completions -d '{\"model\": \"gpt2\", \"prompt\": [\"a\", \"b\"]}'  # 400\n\n# after\ncurl -X POST localhost:8000/v1/completions -d '{\"model\": \"gpt2\", \"prompt\": \"a\"}'","handlingStrategy":"type-guard","validationCode":"prompt = body.get(\"prompt\", \"\")\nif not isinstance(prompt, str):\n    return JSONResponse(status_code=400, content={\"error\": \"prompt must be a string\"})","typeGuard":"def coerce_prompt(body: dict) -> str:\n    p = body.get(\"prompt\", \"\")\n    if isinstance(p, list) and len(p) == 1 and isinstance(p[0], str):\n        return p[0]\n    if not isinstance(p, str):\n        raise ValueError(\"prompt must be a string\")\n    return p","tryCatchPattern":"from fastapi import HTTPException\n\ntry:\n    resp = await client.post(\"/v1/completions\", json={\"model\": mid, \"prompt\": prompt})\n    resp.raise_for_status()\nexcept HTTPException as e:\n    if e.status_code == 400 and \"prompt must be a string\" in e.detail:\n        prompt = prompt[0] if isinstance(prompt, list) else str(prompt)\n        resp = await client.post(\"/v1/completions\", json={\"model\": mid, \"prompt\": prompt})\n    else:\n        raise","preventionTips":["Always send prompt as a plain string","Batch client-side: one request per prompt","Decode token ids to text before calling the API"],"tags":["serving","api","completions","validation","http-400"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}