sgl-project/sglang · critical · ValueError

Only 'absolute' position_embedding_type is supported

Error message

Only 'absolute' position_embedding_type is supported

What it means

The BERT embeddings module only implements absolute position embeddings; config.position_embedding_type must be 'absolute'. Other types like sinusoidal or relative_key raise here.

Source

Thrown at python/sglang/srt/models/bert.py:47

        super().__init__()
        self.size = config.hidden_size
        self.word_embeddings = VocabParallelEmbedding(
            config.vocab_size, config.hidden_size
        )
        self.position_embeddings = VocabParallelEmbedding(
            config.max_position_embeddings, config.hidden_size
        )
        self.token_type_embeddings = VocabParallelEmbedding(
            config.type_vocab_size, config.hidden_size
        )
        self.LayerNorm = nn.LayerNorm(config.hidden_size, eps=config.layer_norm_eps)
        self.position_ids = nn.Parameter(
            torch.empty((1, config.max_position_embeddings)),
        )

        self.position_embedding_type = config.position_embedding_type
        if self.position_embedding_type != "absolute":
            raise ValueError(
                "Only 'absolute' position_embedding_type" + " is supported"
            )

    def forward(
        self,
        input_ids: torch.Tensor,
        positions: torch.Tensor,
        forward_batch: ForwardBatch,
    ) -> torch.Tensor:
        input_shape = input_ids.size()

        # Input embeddings.
        inputs_embeds = self.word_embeddings(input_ids)

        # Position embeddings.
        position_embeddings = self.position_embeddings(positions)

        token_type_ids = forward_batch.token_type_ids

View on GitHub (pinned to 0132848349)

Solutions

1. Set "position_embedding_type": "absolute" in config.json
2. Use a BERT checkpoint that was trained with absolute position embeddings

Example fix

// before
"position_embedding_type": "sinusoidal"
// after
"position_embedding_type": "absolute"

assert config.position_embedding_type == "absolute"

When it happens

Trigger: Loading a BERT config with position_embedding_type set to "sinusoidal", "relative_key", etc.

Common situations: Using a non-standard BERT variant checkpoint; configs produced by conversions that default to a different embedding type.

Related errors

• bad compress_ratio {compress_ratio}
• The requested FlashAttention forward configuration exceeds S
• flashinfer_sparse_mla supports only GLM DSA with FP8 KV cach
• GLM DSA with FP8 KV cache on NVIDIA SM120/SM121 supports onl
• kv-canary: RealKvSource.page_size must be >= 1, got {self.pa

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/78a888946f9e6007. Report an issue: GitHub.