HKUDS/DeepTutor · error · ValueError

OpenAI-compatible embedding model '{model}' does not support

Error message

OpenAI-compatible embedding model '{model}' does not support multimodal `contents`.

What it means

embed() was called with multimodal `contents` (image inputs) while the configured model name does not look like a multimodal embedding model. The adapter refuses to forward image payloads to ordinary text-embedding models, which would either fail cryptically or silently embed garbage.

Source

Thrown at deeptutor/services/embedding/adapters/openai_compatible.py:184

            dimension=self.dimensions or 1,
            send_dimensions=self.send_dimensions,
        )

    async def embed(self, request: EmbeddingRequest) -> EmbeddingResponse:
        import asyncio

        headers = {"Content-Type": "application/json"}
        api_key = self._auth_api_key()
        self._set_auth_header(headers, api_key)
        headers.update({str(k): str(v) for k, v in self.extra_headers.items()})

        # Multimodal: pass `contents` through as `input` only for model names
        # that clearly advertise image/vision embedding support. This prevents
        # image indexing from accidentally hitting ordinary text-embedding
        # models just because the provider family has some multimodal models.
        model = request.model or self.model
        if request.contents and not looks_like_multimodal_embedding_model(model):
            raise ValueError(
                f"OpenAI-compatible embedding model '{model}' does not support "
                "multimodal `contents`."
            )
        input_payload: Any = request.contents if request.contents else request.texts

        payload = {
            "input": input_payload,
            "model": model,
        }
        # `encoding_format` is opt-in: omit it by default (request default is
        # None) because several OpenAI-compatible gateways (e.g. SiliconFlow)
        # reject the param with HTTP 400. Only forward an explicit choice.
        # Do not add a default here for the gateways that require the param —
        # that trades #934 for #651. The retry below recovers those from the
        # provider's own refusal, leaving every working config untouched.
        if request.encoding_format:
            payload["encoding_format"] = request.encoding_format

View on GitHub (pinned to 3e82f13042)

Solutions

  1. Switch the KB's embedding binding to a multimodal embedding model (name containing clip/vl/image/multimodal markers)
  2. Or keep the text model and index only text — do not pass contents
  3. If the model genuinely is multimodal but named without recognized markers, rename/alias it so the heuristic matches or route via a multimodal-capable provider (cohere, aliyun)

Example fix

# before
model = "text-embedding-3-small"  # + request.contents -> raises
# after
model = "Qwen3-VL-Embedding"  # recognized as multimodal, contents forwarded as input
Defensive patterns

Strategy: validation

Validate before calling

from deeptutor.services.embedding.adapters.base import looks_like_multimodal_embedding_model

def can_embed_contents(adapter, model):
    return looks_like_multimodal_embedding_model(model or adapter.model)

Type guard

def supports_multimodal_contents(model: str) -> bool:
    return looks_like_multimodal_embedding_model(model)

Try / catch

null

Prevention

When it happens

Trigger: EmbeddingRequest.contents is non-empty AND looks_like_multimodal_embedding_model(model) is False — e.g. indexing images against text-embedding-3-small or a Qwen text-embedding model.

Common situations: Image/PDF indexing pipeline accidentally wired to a text-only embedding binding; model name lacks vision/VL/multimodal markers (e.g. 'jina-clip', 'Qwen3-VL-Embedding' pass; 'bge-m3' fails).

Related errors

AI-assisted analysis of HKUDS/DeepTutor@3e82f13042 (2026-08-27). Data as JSON: /api/errors/aeb2eb1ad77440ac. Report an issue: GitHub.