{"record":{"id":"ba8ba0b9e5870398","repo":"huggingface/smolagents","slug":"inferenceclientmodel-only-supports-structured-outp","errorCode":null,"errorMessage":"InferenceClientModel only supports structured outputs with these providers:, '.join(STRUCTURED_GENERATION_PROVIDERS)","messagePattern":"InferenceClientModel only supports structured outputs with these providers:, '\\.join\\(STRUCTURED_GENERATION_PROVIDERS\\)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/smolagents/models.py","lineNumber":1562,"sourceCode":"        }\n        super().__init__(model_id=model_id, custom_role_conversions=custom_role_conversions, **kwargs)\n\n    def create_client(self):\n        \"\"\"Create the Hugging Face client.\"\"\"\n        from huggingface_hub import InferenceClient\n\n        return InferenceClient(**self.client_kwargs)\n\n    def generate(\n        self,\n        messages: list[ChatMessage | dict],\n        stop_sequences: list[str] | None = None,\n        response_format: dict[str, str] | None = None,\n        tools_to_call_from: list[Tool] | None = None,\n        **kwargs,\n    ) -> ChatMessage:\n        if response_format is not None and self.client_kwargs[\"provider\"] not in STRUCTURED_GENERATION_PROVIDERS:\n            raise ValueError(\n                \"InferenceClientModel only supports structured outputs with these providers:\"\n                + \", \".join(STRUCTURED_GENERATION_PROVIDERS)\n            )\n        completion_kwargs = self._prepare_completion_kwargs(\n            messages=messages,\n            stop_sequences=stop_sequences,\n            tools_to_call_from=tools_to_call_from,\n            # response_format=response_format,\n            convert_images_to_image_urls=True,\n            custom_role_conversions=self.custom_role_conversions,\n            **kwargs,\n        )\n        self._apply_rate_limit()\n        response = self.retryer(self.client.chat_completion, **completion_kwargs)\n        content = response.choices[0].message.content\n        if stop_sequences is not None and not self.supports_stop_parameter:\n            content = remove_content_after_stop_sequences(content, stop_sequences)\n        return ChatMessage(","sourceCodeStart":1544,"sourceCodeEnd":1580,"githubUrl":"https://github.com/huggingface/smolagents/blob/30bb1161095dbae2271e6bc3cc4c219cc3897a57/src/smolagents/models.py#L1544-L1580","documentation":"InferenceClientModel.generate only supports structured outputs (response_format) with a whitelist of providers (STRUCTURED_GENERATION_PROVIDERS, e.g. together, hyperbolic, etc. on the HF Inference Providers route). Passing response_format while client_kwargs['provider'] is outside that list raises ValueError before any request is made.","triggerScenarios":"Calling generate(..., response_format={...}) on an InferenceClientModel whose provider= is not in STRUCTURED_GENERATION_PROVIDERS (e.g. auto, hf-inference, or an unsupported third-party provider).","commonSituations":"Porting OpenAIModel structured-output code to InferenceClientModel without changing provider; leaving provider unset (defaults to 'auto') and assuming JSON mode works; provider gaining support in a newer smolagents release than the installed one.","solutions":["Set provider to one of the supported ones (check STRUCTURED_GENERATION_PROVIDERS in your installed version), e.g. provider=\"together\" with a compatible model.","If you need structured output on an unsupported provider, drop response_format and enforce the schema yourself by prompting for JSON and parsing/validating the reply.","Upgrade smolagents — the supported provider list grows over releases."],"exampleFix":"# before\nmodel = InferenceClientModel(model_id=\"meta-llama/Llama-3.1-8B-Instruct\")\nout = model(messages, response_format={\"type\": \"json_object\"})  # ValueError\n\n# after\nmodel = InferenceClientModel(model_id=\"meta-llama/Llama-3.1-8B-Instruct\", provider=\"together\")\nout = model(messages, response_format={\"type\": \"json_object\"})","handlingStrategy":"validation","validationCode":"from smolagents.models import STRUCTURED_GENERATION_PROVIDERS\nprovider = \"together\"\nassert provider in STRUCTURED_GENERATION_PROVIDERS, (\n    f\"structured output needs one of {STRUCTURED_GENERATION_PROVIDERS}\")","typeGuard":"def supports_structured(provider: str) -> bool:\n    from smolagents.models import STRUCTURED_GENERATION_PROVIDERS\n    return provider in STRUCTURED_GENERATION_PROVIDERS","tryCatchPattern":"try:\n    out = model(messages, response_format=fmt)\nexcept ValueError as e:\n    if \"structured outputs\" in str(e):\n        out = model(messages)  # parse/validate JSON yourself\n    else:\n        raise","preventionTips":["Check STRUCTURED_GENERATION_PROVIDERS (from your installed version) before requesting response_format.","Have a prompt-based JSON + local schema-validation fallback for unsupported providers.","Pin/upgrade smolagents when new providers gain structured-output support."],"tags":["huggingface","structured-output","response-format","provider-support"],"backgroundTag":"provider-unsupported-feature","analyzedSha":"30bb1161095dbae2271e6bc3cc4c219cc3897a57","analyzedAt":"2026-08-28T18:52:54.169Z","schemaVersion":2},"datasetVersion":"2026-08-28T21:17:43.275Z"}