sgl-project/sglang · error · ValueError

The original encoder only has

Error message

The original encoder only has {num_hidden_layers} layers, but you requested {len(self.encoder.layers)} layers.

What it means

SiglipVisionModel builds its encoder with an optional layer-count override; if the constructed encoder ends up with more layers than config.num_hidden_layers, the checkpoint is inconsistent with the request and init aborts. This mirrors HF's check that you cannot request more layers than the pretrained encoder has.

Solutions

  1. Set the layer override to a value <= num_hidden_layers (layer trimming only reduces)
  2. Drop the override entirely to use all pretrained layers
  3. Check config.json num_hidden_layers matches the checkpoint you loaded

Example fix

# before
override = 40  # checkpoint only has 27
# after
override = 16  # or None to use all layers
Defensive patterns

Strategy: validation

Validate before calling

n = config.num_hidden_layers
assert override is None or override <= n, f"override {override} > {n} layers"

Prevention

When it happens

Trigger: Passing a num_hidden_layers_override (or a config where the override exceeds num_hidden_layers) when constructing SiglipVisionModel, making len(self.encoder.layers) > config.num_hidden_layers at python/sglang/srt/models/siglip.py:268.

Common situations: Server args like --num-hiddenLayers-override style vision-layer trimming features, or a mismatched config.json where num_hidden_layers was edited down; typically an override value typo (larger instead of smaller).

Related errors


AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/f92459b3de02e001. Report an issue: GitHub.

Appendix: source

Thrown at python/sglang/srt/models/siglip.py:268

        embed_dim = config.hidden_size

        self.embeddings = SiglipVisionEmbeddings(
            config, use_data_parallel=use_data_parallel
        )

        self.encoder = SiglipEncoder(
            config=config,
            qkv_backend=qkv_backend,
            act_layer=act_layer,
            flatten_batch=flatten_batch,
            use_data_parallel=use_data_parallel,
            quant_config=quant_config,
            prefix=add_prefix("encoder", prefix),
        )

        num_hidden_layers = config.num_hidden_layers
        if len(self.encoder.layers) > config.num_hidden_layers:
            raise ValueError(
                f"The original encoder only has {num_hidden_layers} "
                f"layers, but you requested {len(self.encoder.layers)} layers."
            )

        # VisionAttention in SiglipEncoderLayer is multihead attention
        self.post_layernorm = nn.LayerNorm(embed_dim, eps=config.layer_norm_eps)

    @property
    def device(self) -> torch.device:
        return self.embeddings.patch_embedding.weight.device

    def forward(
        self,
        pixel_values: torch.Tensor,
    ) -> torch.Tensor:
        hidden_states = self.embeddings(pixel_values.to(self.device)).to(
            self.post_layernorm.weight.dtype
        )

View on GitHub (pinned to 0132848349)