huggingface/transformers · error · HTTPException

Server is pinned to ' '; requested ' '.

Error message

Server is pinned to '{self.model_manager.force_model}'; requested '{requested}'.

What it means

HTTP 400 from _resolve_model when the server was launched pinned to a single model (force_model) and the request explicitly names a different one. A pinned server rewrites body['model'] to the forced model for consistency, but refuses silently serving the wrong model when the client asked for another.

Solutions

1. Set the request's model field to the pinned model id, matching the server's launch flag
2. Or omit the model field entirely — the server fills in the forced model
3. Or restart the server without the pin if multi-model dispatch is actually required

Example fix

# before
$ transformers serve --model openai-community/gpt2
client.post(body={'model': 'meta-llama/Llama-3.1-8B', ...})
# after
client.post(body={'model': 'openai-community/gpt2', ...})

if pinned_model is not None:  # discovered from server config/docs endpoint
    body['model'] = pinned_model
resp = client.post('/v1/chat/completions', json=body)

if resp.status_code == 400 and 'pinned to' in resp.text:
    pinned = re.search(r"pinned to '([^']+)'", resp.text).group(1)
    body['model'] = pinned
    resp = client.post('/v1/chat/completions', json=body)

When it happens

Trigger: Start the CLI with a forced model flag (e.g. transformers serve --model <id> pinning the server), then POST a chat/completions body whose 'model' field is a different model id.

Common situations: Reusing an OpenAI-compatible client configured with 'gpt-3.5-turbo' against a pinned local server; load balancers routing requests from mixed clients to one pinned replica; forgetting the deployment is pinned.

Related errors

• prompt must be a string.
• Audio transcription requires sequential generation (not CB)
• Expected file upload, got string
• Expected model name as string
• 'input' must be a string or list

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/edc93eca445faff3. Report an issue: GitHub.

Appendix: source

Thrown at src/transformers/cli/serving/utils.py:1114

    @staticmethod
    def chunk_to_sse(chunk: "str | pydantic.BaseModel") -> str:
        """Format a pydantic model or string as an SSE ``data:`` line."""
        if isinstance(chunk, str):
            return chunk if chunk.startswith("data: ") else f"data: {chunk}\n\n"
        return f"data: {chunk.model_dump_json(exclude_none=True)}\n\n"

    def _resolve_model(self, body: dict) -> tuple[str, "PreTrainedModel", "ProcessorMixin | PreTrainedTokenizerFast"]:
        """Apply force_model, load model + processor.

        Returns ``(model_id, model, processor)``.
        """
        from fastapi import HTTPException

        if self.model_manager.force_model is not None:
            requested = body.get("model")
            if requested is not None and requested != self.model_manager.force_model:
                raise HTTPException(
                    status_code=400,
                    detail=(f"Server is pinned to '{self.model_manager.force_model}'; requested '{requested}'."),
                )
            body["model"] = self.model_manager.force_model

        model_id = self.model_manager.process_model_name(body["model"])
        model, processor = self.model_manager.load_model_and_processor(model_id)

        return model_id, model, processor

    def _build_generation_config(
        self, body: dict, model_generation_config: "GenerationConfig", use_cb: bool = False
    ) -> "GenerationConfig":
        """Build a GenerationConfig from shared params (temperature, top_p, seed, generation_config JSON).

        Subclasses should call ``super()._build_generation_config(...)`` then apply
        endpoint-specific params (``max_tokens``, ``max_output_tokens``, etc.).

View on GitHub (pinned to a597f97485)