{"record":{"id":"94be830c467ded2f","repo":"huggingface/tokenizers","slug":"async-decode-batch-sequences-can-t-be-none","errorCode":null,"errorMessage":"async_decode_batch: `sequences` can't be `None`","messagePattern":"async_decode_batch: `sequences` can't be `None`","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"bindings/python/py_src/tokenizers/implementations/base_tokenizer.py","lineNumber":354,"sourceCode":"\n        return self._tokenizer.decode_batch(sequences, skip_special_tokens=skip_special_tokens)\n\n    async def async_decode_batch(\n        self,\n        sequences: List[List[int]],\n        skip_special_tokens: bool = True,\n    ) -> List[str]:\n        \"\"\"Asynchronously decode a batch of sequences.\n\n        Args:\n            sequences: A list of sequences of ids to decode.\n            skip_special_tokens: Whether to remove special tokens from output.\n\n        Returns:\n            A list of decoded strings.\n        \"\"\"\n        if sequences is None:\n            raise ValueError(\"async_decode_batch: `sequences` can't be `None`\")\n        return await self._tokenizer.async_decode_batch(sequences, skip_special_tokens)\n\n    def token_to_id(self, token: str) -> Optional[int]:\n        \"\"\"Convert the given token to its corresponding id\n\n        Args:\n            token: str:\n                The token to convert\n\n        Returns:\n            The corresponding id if it exists, None otherwise\n        \"\"\"\n        return self._tokenizer.token_to_id(token)\n\n    def id_to_token(self, id: int) -> Optional[str]:\n        \"\"\"Convert the given token id to its corresponding string\n\n        Args:","sourceCodeStart":336,"sourceCodeEnd":372,"githubUrl":"https://github.com/huggingface/tokenizers/blob/6cfd9d385ca0ed91c10b49f0ce97d02cfde1b607/bindings/python/py_src/tokenizers/implementations/base_tokenizer.py#L336-L372","documentation":"`Tokenizer.async_decode_batch` raises this `ValueError` when `sequences` is `None`. The async wrapper validates that the batch of id sequences is a list before awaiting the Rust implementation, giving a clear message naming the offending parameter.","triggerScenarios":"Calling `await tokenizer.async_decode_batch(None)`, or awaiting with a variable that a prior async stage set to `None` (failed encode batch, cancelled aggregation).","commonSituations":"Async serving stacks where decode is chained after async encode and an earlier failure left the batch as `None`; pipeline frameworks that represent 'no data' as `None` instead of `[]`.","solutions":["Pass a list of id lists, or `[]` for an empty batch.","Guard the await: `if seqs is not None: texts = await tokenizer.async_decode_batch(seqs)`.","Fix the upstream async stage to return `[]` instead of `None`."],"exampleFix":"// before\ntexts = await tokenizer.async_decode_batch(seqs)\n// after\ntexts = await tokenizer.async_decode_batch(seqs or [])","handlingStrategy":"validation","validationCode":"if seqs is None:\n    seqs = []\ntexts = await tokenizer.async_decode_batch(seqs)","typeGuard":"def is_batch_of_ids(value) -> bool:\n    return isinstance(value, list) and all(isinstance(s, list) for s in value)","tryCatchPattern":"try:\n    texts = await tokenizer.async_decode_batch(seqs)\nexcept ValueError as e:\n    if \"can't be `None`\" in str(e):\n        texts = []\n    else:\n        raise","preventionTips":["Chain async encode/decode stages so a failed stage short-circuits instead of passing None onward.","Coalesce `seqs or []` before awaiting.","Type-hint pipeline stages with non-Optional list returns."],"tags":["python","async","null-argument","tokenizers"],"backgroundTag":"null-argument","analyzedSha":"6cfd9d385ca0ed91c10b49f0ce97d02cfde1b607","analyzedAt":"2026-09-09T11:43:25.027Z","contentChangedAt":"2026-09-09T11:43:25.027Z","schemaVersion":2},"datasetVersion":"2026-09-16T04:17:20.429Z"}