{"record":{"id":"a1099617c35343af","repo":"huggingface/candle","slug":"only-alibi-is-supported-as-a-position-embedding-ty","errorCode":null,"errorMessage":"only alibi is supported as a position-embedding-type","messagePattern":"only alibi is supported as a position-embedding-type","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-transformers/src/models/jina_bert.rs","lineNumber":350,"sourceCode":"            .take(n_heads)\n            .cloned()\n            .collect::<Vec<f32>>()\n    };\n    let slopes = Tensor::new(slopes, &Device::Cpu)?.reshape((1, (), 1, 1))?;\n    alibi_bias.to_dtype(DType::F32)?.broadcast_mul(&slopes)\n}\n\n#[derive(Clone, Debug)]\nstruct BertEncoder {\n    alibi: Tensor,\n    layers: Vec<BertLayer>,\n    span: tracing::Span,\n}\n\nimpl BertEncoder {\n    fn new(vb: VarBuilder, cfg: &Config) -> Result<Self> {\n        if cfg.position_embedding_type != PositionEmbeddingType::Alibi {\n            candle::bail!(\"only alibi is supported as a position-embedding-type\")\n        }\n        let layers = (0..cfg.num_hidden_layers)\n            .map(|index| BertLayer::new(vb.pp(format!(\"layer.{index}\")), cfg))\n            .collect::<Result<Vec<_>>>()?;\n        let span = tracing::span!(tracing::Level::TRACE, \"encoder\");\n        let alibi = build_alibi_bias(cfg)?.to_device(vb.device())?;\n        Ok(Self {\n            alibi,\n            layers,\n            span,\n        })\n    }\n}\n\nimpl Module for BertEncoder {\n    fn forward(&self, xs: &Tensor) -> Result<Tensor> {\n        let _enter = self.span.enter();\n        let seq_len = xs.dim(1)?;","sourceCodeStart":332,"sourceCodeEnd":368,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-transformers/src/models/jina_bert.rs#L332-L368","documentation":"BertEncoder::new in jina_bert only supports ALiBi positional embeddings; any other cfg.position_embedding_type (e.g. absolute/rotary) is rejected at encoder construction. This keeps the implementation scoped to the Jina BERT ALiBi variant.","triggerScenarios":"Instantiating a Jina BERT model with a Config whose position_embedding_type != PositionEmbeddingType::Alibi — e.g. loading a standard BERT config.json lacking \"position_embedding_type\": \"alibi\".","commonSituations":"Feeding a vanilla BERT checkpoint/config into the jina_bert model code; config.json missing the field so it defaults to non-Alibi; using jina_bert for a different Jina model version.","solutions":["Set \"position_embedding_type\": \"alibi\" in the config","Use the candle bert model implementation instead for standard BERT checkpoints","Confirm you're loading an actual Jina BERT (jinaai/jina-bert-*) ALiBi checkpoint"],"exampleFix":"// before (config.json)\n{\"position_embedding_type\": \"absolute\", ...}\n// after\n{\"position_embedding_type\": \"alibi\", ...}","handlingStrategy":"validation","validationCode":"if cfg.position_embedding_type != jina_bert::PositionEmbeddingType::Alibi {\n    return Err(\"jina_bert requires position_embedding_type == alibi\");\n}","typeGuard":"fn uses_alibi(c: &jina_bert::Config) -> bool {\n    c.position_embedding_type == jina_bert::PositionEmbeddingType::Alibi\n}","tryCatchPattern":"match jina_bert::BertModel::new(&vb, cfg) {\n    Err(e) if e.to_string().contains(\"only alibi\") =>\n        Err(anyhow!(\"use candle's bert impl for non-ALiBi checkpoints\")),\n    r => r.map_err(Into::into),\n}","preventionTips":["Only load jinaai/jina-bert-* (ALiBi) checkpoints with this model","Standard BERT checkpoints go through candle's bert module instead","Check position_embedding_type in config.json before constructing the encoder"],"tags":["config","jina-bert","position-embedding"],"backgroundTag":"unsupported-architecture-config","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}