sgl-project/sglang · error · Exception

GenerativeModel

Error message

GenerativeModel

What it means

In the VertexAI backend's __init__, a module-level import of GenerativeModel was wrapped in try/except that stores the exception; if the import failed (e.g. google-cloud-aiplatform not installed), __init__ re-raises that stored import error. It indicates the optional VertexAI dependency is missing or broken.

Source

Thrown at python/sglang/lang/backend/vertexai.py:25

from sglang.lang.ir import SglSamplingParams

try:
    import vertexai
    from vertexai.preview.generative_models import (
        GenerationConfig,
        GenerativeModel,
        Image,
    )
except ImportError as e:
    GenerativeModel = e


class VertexAI(BaseBackend):
    def __init__(self, model_name, safety_settings=None):
        super().__init__()

        if isinstance(GenerativeModel, Exception):
            raise GenerativeModel

        project_id = os.environ["GCP_PROJECT_ID"]
        location = os.environ.get("GCP_LOCATION")
        vertexai.init(project=project_id, location=location)

        self.model_name = model_name
        self.chat_template = get_chat_template("default")
        self.safety_settings = safety_settings

    def get_chat_template(self):
        return self.chat_template

    def generate(
        self,
        s: StreamExecutor,
        sampling_params: SglSamplingParams,
    ):
        if s.messages_:

View on GitHub (pinned to 0132848349)

Solutions

1. pip install google-cloud-aiplatform (and shirley-vertexai or the vertexai package version the code expects)
2. Check the stored exception: run `import vertexai.generative_models` in the same interpreter to see the real import error
3. Pin compatible versions of google-auth / google-cloud-aiplatform if there is a dependency conflict

Example fix

# before
backend = VertexAI("gemini-1.5-pro")
# after
pip install google-cloud-aiplatform
backend = VertexAI("gemini-1.5-pro")

try:
    import vertexai  # noqa
    from vertexai.generative_models import GenerativeModel  # noqa
    vertexai_ok = True
except Exception:
    vertexai_ok = False
if not vertexai_ok:
    raise SystemExit("Install google-cloud-aiplatform to use the VertexAI backend")

def has_vertexai() -> bool:
    try:
        import vertexai.generative_models  # noqa: F401
        return True
    except ImportError:
        return False

try:
    backend = VertexAI(model_name)
except Exception as e:
    raise RuntimeError(f"VertexAI backend unavailable: {e}") from e

When it happens

Trigger: Instantiating VertexAI backend (e.g. lang backend selection for Gemini models) without google-cloud-aiplatform / vertexai installed, or with an incompatible version where `from vertexai.generative_models import GenerativeModel` fails.

Common situations: Using sglang's lang frontend with backend="vertexai" in an environment lacking the extra dependency; dependency conflicts after upgrading google libraries; CI environments that only install core requirements.

Related errors

• Can not import FA3 in sgl_kernel. Please check your installa
• cuda.bindings.driver is required for CUDA VMM operations
• Failed to load serve backend {name!r} from {self._entry_poin
• {_METALLIB_NAME} not found next to the native Metal extensio
• FlashAttention-4 CUTE is not available. Install flash-attn-4

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/e815591a9543fe3c. Report an issue: GitHub.