HKUDS/DeepTutor · error · ValueError

openai_sdk adapter does not support multimodal `contents`. P

Error message

openai_sdk adapter does not support multimodal `contents`. Pick a multimodal-capable provider (cohere, aliyun).

What it means

The openai_sdk adapter (official OpenAI/Azure SDK path) is text-only by design; calling embed() with request.contents (multimodal/image inputs) is rejected up front with a pointer to multimodal-capable providers (cohere, aliyun).

Source

Thrown at deeptutor/services/embedding/adapters/openai_sdk.py:62

    def _build_client(self) -> AsyncOpenAI:
        # OpenRouter / custom gateways often don't validate the key, but the
        # SDK refuses to construct without one. Use a placeholder when empty.
        return AsyncOpenAI(
            api_key=self.api_key or "sk-no-key-required",
            base_url=self.base_url,
            timeout=max(self.request_timeout, 60),
            default_headers=(
                {str(k): str(v) for k, v in self.extra_headers.items()}
                if self.extra_headers
                else None
            ),
            max_retries=2,
            **openai_client_kwargs(timeout=max(self.request_timeout, 60)),
        )

    async def embed(self, request: EmbeddingRequest) -> EmbeddingResponse:
        if request.contents:
            raise ValueError(
                "openai_sdk adapter does not support multimodal `contents`. "
                "Pick a multimodal-capable provider (cohere, aliyun)."
            )

        model = request.model or self.model
        kwargs: Dict[str, Any] = {
            "model": model,
            "input": request.texts,
            # Unlike the gateway adapter (which omits `encoding_format` to avoid
            # HTTP 400s), the official OpenAI/Azure API accepts it and callers
            # expect float vectors, so pin "float" when none is set explicitly.
            "encoding_format": request.encoding_format or "float",
        }
        dim_value = request.dimensions or self.dimensions
        if dim_value and self._should_send_dimensions(model):
            kwargs["dimensions"] = dim_value

        client = self._build_client()

View on GitHub (pinned to 3e82f13042)

Solutions

Switch the embedding binding's provider/adapter to a multimodal-capable one (cohere, aliyun)
Or use openai_compatible adapter with a model name that looks multimodal (clip/vl markers)
Or drop contents and embed text only

Example fix

# before
provider = "openai_sdk"; model = "text-embedding-3-small"  # + contents -> raises
# after
provider = "cohere"; model = "embed-english-v3.0"  # or aliyun multimodal embedding

Defensive patterns

Strategy: type-guard

Validate before calling

if request.contents and adapter.provider == "openai_sdk":
    raise ConfigurationError("switch to a multimodal-capable embedding provider (cohere/aliyun)")

Type guard

def is_multimodal_adapter(adapter) -> bool:
    return getattr(adapter, "provider", "") not in ("openai_sdk",)

Try / catch

try:
    resp = await adapter.embed(req)
except ValueError as e:
    if "multimodal" in str(e):
        adapter = get_multimodal_adapter()  # cohere / aliyun
        resp = await adapter.embed(req)
    else:
        raise

Prevention

Document which embedding bindings are text-only
Gate image-indexing features on adapter multimodal capability checks

When it happens

Trigger: Any embed() call on the openai_sdk adapter where EmbeddingRequest.contents is truthy — e.g. image/document indexing configured against an OpenAI/Azure embedding binding.

Common situations: KB with image attachments wired to the default OpenAI text-embedding binding; config copied from a multimodal setup but adapter left as openai_sdk.

Related errors

AI-assisted analysis of HKUDS/DeepTutor@3e82f13042 (2026-08-27). Data as JSON: /api/errors/4e7c158d25c61df9. Report an issue: GitHub.