{"record":{"id":"400457eb38b10c61","repo":"huggingface/tokenizers","slug":"cannot-build-piece-from-string-s","errorCode":null,"errorMessage":"Cannot build Piece from string \"{s}\"","messagePattern":"Cannot build Piece from string \"(.+?)\"","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"tokenizers/src/processors/template.rs","lineNumber":156,"sourceCode":"\n    fn with_type_id(self, type_id: u32) -> Self {\n        match self {\n            Self::Sequence { id, .. } => Self::Sequence { id, type_id },\n            Self::SpecialToken { id, .. } => Self::SpecialToken { id, type_id },\n        }\n    }\n}\n\nimpl TryFrom<String> for Piece {\n    type Error = String;\n\n    fn try_from(s: String) -> StdResult<Self, Self::Error> {\n        let parts = s.split(':').collect::<Vec<_>>();\n\n        let err = || format!(\"Cannot build Piece from string \\\"{s}\\\"\");\n        match parts.as_slice() {\n            [id, type_id] => {\n                let type_id: u32 = type_id.parse().map_err(|_| err())?;\n                let piece = Self::extract_id(id).ok_or_else(err)?;\n                Ok(piece.with_type_id(type_id))\n            }\n            [id] => Self::extract_id(id).ok_or_else(err),\n            _ => Err(err()),\n        }\n    }\n}\n\nimpl TryFrom<&str> for Piece {\n    type Error = String;\n\n    fn try_from(s: &str) -> StdResult<Self, Self::Error> {\n        Piece::try_from(s.to_owned())\n    }\n}\n\n/// Represents a bunch of tokens to be used in a template.","sourceCodeStart":138,"sourceCodeEnd":174,"githubUrl":"https://github.com/huggingface/tokenizers/blob/6cfd9d385ca0ed91c10b49f0ce97d02cfde1b607/tokenizers/src/processors/template.rs#L138-L174","documentation":"This error comes from `TryFrom<String> for TemplateProcessing`'s `Piece` in the tokenizers library. A `Piece` in a post-processor template must be a `<id>` like `$A`, `$B`, or a special token like `[CLS]`, optionally with `:<type_id>`. When the `id:type_id` form is given, the part after the colon must parse as a `u32`; if it does not, the library rejects the whole string with `Cannot build Piece from string \"{s}\"`.","triggerScenarios":"Calling `Piece::try_from` (directly or while building a `TemplateProcessing` template string like `\"$A:0 $B:1\"`) where the segment after `:` is not a valid unsigned integer, e.g. `\"$A:first\"`, `\"$B:-1\"`, `\"$A:1.5\"`, or `\"$A: \"`.","commonSituations":"Common when writing post-processor JSON configs by hand — typos in the type id, using a 1-based index when 0-based is expected plus a non-numeric value, pasting templates from other tokenizers (e.g. sentencepiece style), or quoting/whitespace sneaking into the type id.","solutions":["Make the type id a plain non-negative integer with no spaces or quotes inside the segment: `$A:0 $B:1`.","If you intended only a sequence id, drop the colon entirely (`$A`) instead of supplying a non-numeric type id.","Validate the template before feeding it to `TemplateProcessing::new`/config loading by splitting on whitespace and `:` and checking `u32::from_str` on the second part.","Check that the template values correspond to declared `single`/`pair` sequence ids (`A`, `B`) and special tokens in your tokenizer."],"exampleFix":"// before\nTemplateProcessing::builder().single(\"$A:first $B:1\")\n\n// after\nTemplateProcessing::builder().single(\"$A:0 $B:1\")","handlingStrategy":"validation","validationCode":"import re\nPIECE_RE = re.compile(r\"^(\\$[A-Za-z]|\\[[^\\]]+\\])(?::(\\d+))?$\")\ndef valid_piece(piece: str) -> bool:\n    return bool(PIECE_RE.match(piece))","typeGuard":null,"tryCatchPattern":"try:\n    proc = TemplateProcessing.builder().single(template).build()\nexcept Exception as e:\n    raise ValueError(f\"bad template piece in {template!r}: {e}\") from e","preventionTips":["Always write type ids as plain non-negative integers (`$A:0`), never strings, negatives, or floats.","Keep templates in version-controlled config and validate them in tests at load time.","Use 0-based type ids; document that the colon part is a u32, not a label."],"tags":["rust","tokenizers","template-processing","parsing"],"backgroundTag":"invalid-argument-format","analyzedSha":"6cfd9d385ca0ed91c10b49f0ce97d02cfde1b607","analyzedAt":"2026-09-09T11:43:25.027Z","contentChangedAt":"2026-09-09T11:43:25.027Z","schemaVersion":2},"datasetVersion":"2026-09-16T04:17:20.429Z"}