{"record":{"id":"f9435d027d5749b9","repo":"OpenBMB/VoxCPM","slug":"expected-string-input-got-type-text","errorCode":null,"errorMessage":"Expected string input, got {type(text)}","messagePattern":"Expected string input, got (.+?)","errorType":"validation","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"src/voxcpm/model/utils.py","lineNumber":96,"sourceCode":"            self.multichar_tokens = multichar_tokens\n\n        def tokenize(self, text: str, **kwargs) -> List[str]:\n            \"\"\"Tokenize text and split multi-character Chinese tokens into single characters.\n\n            Args:\n                text: Input text to tokenize\n                **kwargs: Additional arguments passed to the base tokenizer\n\n            Returns:\n                List of processed tokens with multi-character Chinese tokens split\n\n            Example:\n                >>> wrapper = CharTokenizerWrapper(tokenizer)\n                >>> tokens = wrapper.tokenize(\"你好世界\")\n                >>> # Returns [\"你\", \"好\", \"世\", \"界\"] instead of [\"你好\", \"世界\"]\n            \"\"\"\n            if not isinstance(text, str):\n                raise TypeError(f\"Expected string input, got {type(text)}\")\n\n            tokens = self.tokenizer.tokenize(text, **kwargs)\n            processed = []\n\n            for token in tokens:\n                # Remove possible subword prefix\n                clean_token = token.replace(\"▁\", \"\")\n\n                if clean_token in self.multichar_tokens:\n                    # Split multi-character token into single characters\n                    chars = list(clean_token)\n                    processed.extend(chars)\n                else:\n                    processed.append(token)\n\n            return processed\n\n        def __call__(self, text: str, **kwargs) -> List[int]:","sourceCodeStart":78,"sourceCodeEnd":114,"githubUrl":"https://github.com/OpenBMB/VoxCPM/blob/f5a1c6a6b901bc732e20f0d59a369f6829ad717a/src/voxcpm/model/utils.py#L78-L114","documentation":"CharTokenizerWrapper.tokenize requires a Python str; any other type (None, bytes, list) raises TypeError. This wrapper splits Hugging Face subword tokens into per-character tokens.","triggerScenarios":"Calling the wrapped tokenizer's tokenize/__call__ with non-string input, which propagates from _generate when text is mistyped (though core.py usually catches empty/None first).","commonSituations":"Programmatic pipelines feeding None or pre-tokenized lists into the tokenizer, or bytes from file reads not decoded.","solutions":["Ensure text is decoded to str before tokenization","Add an isinstance(text, str) guard upstream","Decode bytes with .decode('utf-8') before passing"],"exampleFix":"# before\ntokens = tokenizer.tokenize(raw_bytes)\n# after\ntokens = tokenizer.tokenize(raw_bytes.decode(\"utf-8\"))","handlingStrategy":"type-guard","validationCode":"assert isinstance(text, str) and text != \"\"","typeGuard":"def is_str_input(x) -> bool:\n    return isinstance(x, str)","tryCatchPattern":null,"preventionTips":["Decode bytes at IO boundaries","Type-annotate pipeline functions with str"],"tags":["tokenizer","type-error","input-validation"],"backgroundTag":"wrong-argument-type","analyzedSha":"f5a1c6a6b901bc732e20f0d59a369f6829ad717a","analyzedAt":"2026-08-27T05:54:30.617Z","schemaVersion":2},"datasetVersion":"2026-08-27T08:17:20.692Z"}