{"record":{"id":"0c564212e8d94a12","repo":"sgl-project/sglang","slug":"the-original-encoder-only-has-num-hidden-layers","errorCode":null,"errorMessage":"The original encoder only has {num_hidden_layers} layers, but you requested {len(self.encoder.layers)} layers.","messagePattern":"The original encoder only has (.+?) layers, but you requested (.+?) layers\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/models/encoders/clip.py","lineNumber":286,"sourceCode":"        self.config = config\n        embed_dim = config.hidden_size\n\n        self.embeddings = CLIPVisionEmbeddings(config)\n\n        # NOTE: This typo of \"layrnorm\" is not fixed on purpose to match\n        # the original transformers code and name of the model weights.\n        self.pre_layrnorm = nn.LayerNorm(embed_dim, eps=config.layer_norm_eps)\n\n        self.encoder = CLIPEncoder(\n            config=config,\n            quant_config=quant_config,\n            num_hidden_layers_override=num_hidden_layers_override,\n            prefix=f\"{prefix}.encoder\",\n        )\n\n        num_hidden_layers = config.num_hidden_layers\n        if len(self.encoder.layers) > config.num_hidden_layers:\n            raise ValueError(\n                f\"The original encoder only has {num_hidden_layers} \"\n                f\"layers, but you requested {len(self.encoder.layers)} layers.\"\n            )\n\n        # If possible, skip post_layernorm to conserve memory\n        if require_post_norm is None:\n            require_post_norm = len(self.encoder.layers) == num_hidden_layers\n\n        if require_post_norm:\n            self.post_layernorm = nn.LayerNorm(embed_dim, eps=config.layer_norm_eps)\n        else:\n            self.post_layernorm = None\n\n    def forward(\n        self,\n        pixel_values: torch.Tensor,\n        output_hidden_states: Optional[bool] = None,\n        feature_sample_layers: list[int] | None = None,","sourceCodeStart":268,"sourceCodeEnd":304,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/models/encoders/clip.py#L268-L304","documentation":"Raised in CLIPTextModel.__init__ when the constructed encoder has more layers than the checkpoint's config.num_hidden_layers. This guard prevents requesting a deeper text tower than the original model was trained with, which would leave new layers randomly initialized.","triggerScenarios":"Setting num_hidden_layers_override to a value greater than config.num_hidden_layers when instantiating the CLIP text model.","commonSituations":"Trying to increase encoder depth for experiments; copying a config with a small num_hidden_layers but a large override; mismatches after editing a model config JSON.","solutions":["Set num_hidden_layers_override <= config.num_hidden_layers (or omit it to use the config value)","If you truly need fewer layers, use the override with a smaller number — truncation is allowed, growth is not","Verify the loaded config's num_hidden_layers matches the checkpoint you intend to use"],"exampleFix":"# before\nmodel = CLIPTextModel(config, num_hidden_layers_override=48)  # config has 24\n# after\nmodel = CLIPTextModel(config, num_hidden_layers_override=24)","handlingStrategy":"validation","validationCode":"assert num_hidden_layers_override is None or num_hidden_layers_override <= config.num_hidden_layers, \"cannot exceed checkpoint layer count\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Validate overrides against config before model construction","Keep checkpoint config.json authoritative"],"tags":["clip","config","model-init","layer-count"],"backgroundTag":"config-value-out-of-range","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}