sgl-project/sglang · warning

Warning: system prompt is not supported in VertexAI.

Error message

Warning: system prompt is not supported in VertexAI.

What it means

VertexAI's Gemini-style API historically has no dedicated system role in this integration, so when converting chat messages any system message is rewritten as a user message prefixed with 'System prompt: '. The warning tells you the system instruction loses its privileged position.

Source

Thrown at python/sglang/lang/backend/vertexai.py:109

            if text_seg != "":
                input.append(text_seg)
            input.append(Image.from_bytes(image_base64_data))
        text_seg = text_segs.pop(0)
        if text_seg != "":
            input.append(text_seg)
        return input

    def messages_to_vertexai_input(self, messages):
        vertexai_message = []
        # from openai message format to vertexai message format
        for msg in messages:
            if isinstance(msg["content"], str):
                text = msg["content"]
            else:
                text = msg["content"][0]["text"]

            if msg["role"] == "system":
                warnings.warn("Warning: system prompt is not supported in VertexAI.")
                vertexai_message.append(
                    {
                        "role": "user",
                        "parts": [{"text": "System prompt: " + text}],
                    }
                )
                vertexai_message.append(
                    {
                        "role": "model",
                        "parts": [{"text": "Understood."}],
                    }
                )
                continue
            if msg["role"] == "user":
                vertexai_msg = {
                    "role": "user",
                    "parts": [{"text": text}],
                }

View on GitHub (pinned to 0132848349)

Solutions

  1. Keep the system prompt but verify output quality, since it is downgraded to user text
  2. Move critical instructions into the first user message yourself for cleaner conditioning
  3. Switch to a backend/endpoint that supports a native system role if strict system behavior is needed

Example fix

# before
sgl.set_system("You are a helpful assistant.")
# after (vertexai)
gen = sgl.gen("reply", prompt="You are a helpful assistant.\n" + user_query)
Defensive patterns

Strategy: validation

Validate before calling

messages = [m for m in messages if m["role"] != "system"] if backend is vertexai else messages
# fold system text into first user message first

Prevention

When it happens

Trigger: Calling generate/generate_stream on the VertexAI backend with a chat transcript that includes a message with role 'system' (e.g. set_system prompt in the lang frontend).

Common situations: Reusing a system-prompt-heavy agent app across OpenAI and VertexAI backends; few-shot setups where the system message carries instructions the model then partially ignores.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/aa59984bed262f70. Report an issue: GitHub.