huggingface/transformers · error · HTTPException

prompt must be a string.

Error message

prompt must be a string.

What it means

In the legacy/OpenAI-style completions endpoint of `transformers serve`, the handler reads body['prompt'] (defaulting to '') and requires it to be a Python str. Anything else — most commonly a list of prompt strings (allowed by the OpenAI API for multi-prompt batching) or a list of token ids — triggers HTTP 400 'prompt must be a string.'.

Source

Thrown at src/transformers/cli/serving/completion.py:91

    _valid_params_class = TransformersTextCompletionCreateParams
    _unused_fields = UNUSED_LEGACY_COMPLETION_FIELDS

    async def handle_request(self, body: dict, request_id: str) -> "StreamingResponse | JSONResponse":
        """Validate the request, load the model, and dispatch to streaming or non-streaming.

        Args:
            body (`dict`): The raw JSON request body (OpenAI legacy completions format).
            request_id (`str`): Unique request identifier (from header or auto-generated).

        Returns:
            `StreamingResponse | JSONResponse`: SSE stream or JSON depending on ``body["stream"]``.
        """
        self._validate_request(body)

        prompt = body.get("prompt", "")
        if not isinstance(prompt, str):
            raise HTTPException(status_code=400, detail="prompt must be a string.")

        model_id, model, processor = self._resolve_model(body)
        modality = self.model_manager.get_model_modality(model, processor=processor)
        use_cb = self.generation_state.use_continuous_batching(model, modality)
        logger.warning(f"[Request received] Model: {model_id}, CB: {use_cb}")
        gen_manager = self.generation_state.get_manager(model_id, use_cb=use_cb)

        tokenizer = getattr(processor, "tokenizer", processor)
        inputs = tokenizer(prompt, return_tensors=None if use_cb else "pt")
        if not use_cb:
            inputs = inputs.to(model.device)

        gen_config = self._build_generation_config(body, model.generation_config, use_cb=use_cb)
        if use_cb:
            gen_manager.init_cb(model, gen_config)

        suffix = body.get("suffix")
        streaming = body.get("stream")

View on GitHub (pinned to a597f97485)

Solutions

1. Send a single string: {"model": ..., "prompt": "Once upon a time"}
2. Loop client-side over multiple prompts, issuing one request per prompt
3. Pre-decode token ids to text before sending
4. Ensure the JSON field is a plain str, not a list or null

Example fix

# before
curl -X POST localhost:8000/v1/completions -d '{"model": "gpt2", "prompt": ["a", "b"]}'  # 400

# after
curl -X POST localhost:8000/v1/completions -d '{"model": "gpt2", "prompt": "a"}'

prompt = body.get("prompt", "")
if not isinstance(prompt, str):
    return JSONResponse(status_code=400, content={"error": "prompt must be a string"})

def coerce_prompt(body: dict) -> str:
    p = body.get("prompt", "")
    if isinstance(p, list) and len(p) == 1 and isinstance(p[0], str):
        return p[0]
    if not isinstance(p, str):
        raise ValueError("prompt must be a string")
    return p

from fastapi import HTTPException

try:
    resp = await client.post("/v1/completions", json={"model": mid, "prompt": prompt})
    resp.raise_for_status()
except HTTPException as e:
    if e.status_code == 400 and "prompt must be a string" in e.detail:
        prompt = prompt[0] if isinstance(prompt, list) else str(prompt)
        resp = await client.post("/v1/completions", json={"model": mid, "prompt": prompt})
    else:
        raise

When it happens

Trigger: POSTing to /v1/completions with "prompt": ["a", "b"] (OpenAI multi-prompt form); sending token-id arrays like [50256, 11]; sending an int/None/null prompt; using a client SDK that auto-encodes prompts as lists.

Common situations: Porting code from the OpenAI API where batched string prompts were accepted; sending pre-tokenized ids; a client serializing a single-element list.

Related errors

• 'input' must be a string or list
• Unsupported input item type: {item_type!r}
• Missing `model` field in the request body.
• Unexpected fields in the request: {unexpected}
• Expected file upload, got string

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/683024b5926d9047. Report an issue: GitHub.