huggingface/smolagents · error · ValueError

InferenceClientModel only supports structured outputs with t

Error message

InferenceClientModel only supports structured outputs with these providers:, '.join(STRUCTURED_GENERATION_PROVIDERS)

What it means

InferenceClientModel.generate only supports structured outputs (response_format) with a whitelist of providers (STRUCTURED_GENERATION_PROVIDERS, e.g. together, hyperbolic, etc. on the HF Inference Providers route). Passing response_format while client_kwargs['provider'] is outside that list raises ValueError before any request is made.

Source

Thrown at src/smolagents/models.py:1562

        }
        super().__init__(model_id=model_id, custom_role_conversions=custom_role_conversions, **kwargs)

    def create_client(self):
        """Create the Hugging Face client."""
        from huggingface_hub import InferenceClient

        return InferenceClient(**self.client_kwargs)

    def generate(
        self,
        messages: list[ChatMessage | dict],
        stop_sequences: list[str] | None = None,
        response_format: dict[str, str] | None = None,
        tools_to_call_from: list[Tool] | None = None,
        **kwargs,
    ) -> ChatMessage:
        if response_format is not None and self.client_kwargs["provider"] not in STRUCTURED_GENERATION_PROVIDERS:
            raise ValueError(
                "InferenceClientModel only supports structured outputs with these providers:"
                + ", ".join(STRUCTURED_GENERATION_PROVIDERS)
            )
        completion_kwargs = self._prepare_completion_kwargs(
            messages=messages,
            stop_sequences=stop_sequences,
            tools_to_call_from=tools_to_call_from,
            # response_format=response_format,
            convert_images_to_image_urls=True,
            custom_role_conversions=self.custom_role_conversions,
            **kwargs,
        )
        self._apply_rate_limit()
        response = self.retryer(self.client.chat_completion, **completion_kwargs)
        content = response.choices[0].message.content
        if stop_sequences is not None and not self.supports_stop_parameter:
            content = remove_content_after_stop_sequences(content, stop_sequences)
        return ChatMessage(

View on GitHub (pinned to 30bb116109)

Solutions

Set provider to one of the supported ones (check STRUCTURED_GENERATION_PROVIDERS in your installed version), e.g. provider="together" with a compatible model.
If you need structured output on an unsupported provider, drop response_format and enforce the schema yourself by prompting for JSON and parsing/validating the reply.
Upgrade smolagents — the supported provider list grows over releases.

Example fix

# before
model = InferenceClientModel(model_id="meta-llama/Llama-3.1-8B-Instruct")
out = model(messages, response_format={"type": "json_object"})  # ValueError

# after
model = InferenceClientModel(model_id="meta-llama/Llama-3.1-8B-Instruct", provider="together")
out = model(messages, response_format={"type": "json_object"})

Defensive patterns

Strategy: validation

Validate before calling

from smolagents.models import STRUCTURED_GENERATION_PROVIDERS
provider = "together"
assert provider in STRUCTURED_GENERATION_PROVIDERS, (
    f"structured output needs one of {STRUCTURED_GENERATION_PROVIDERS}")

Type guard

def supports_structured(provider: str) -> bool:
    from smolagents.models import STRUCTURED_GENERATION_PROVIDERS
    return provider in STRUCTURED_GENERATION_PROVIDERS

Try / catch

try:
    out = model(messages, response_format=fmt)
except ValueError as e:
    if "structured outputs" in str(e):
        out = model(messages)  # parse/validate JSON yourself
    else:
        raise

Prevention

Check STRUCTURED_GENERATION_PROVIDERS (from your installed version) before requesting response_format.
Have a prompt-based JSON + local schema-validation fallback for unsupported providers.
Pin/upgrade smolagents when new providers gain structured-output support.

When it happens

Trigger: Calling generate(..., response_format={...}) on an InferenceClientModel whose provider= is not in STRUCTURED_GENERATION_PROVIDERS (e.g. auto, hf-inference, or an unsupported third-party provider).

Common situations: Porting OpenAIModel structured-output code to InferenceClientModel without changing provider; leaving provider unset (defaults to 'auto') and assuming JSON mode works; provider gaining support in a newer smolagents release than the installed one.

Related errors

AI-assisted analysis of huggingface/smolagents@30bb116109 (2026-08-28). Data as JSON: /api/errors/ba8ba0b9e5870398. Report an issue: GitHub.